Firepower, url filtering for https.

HugePerch

Plan to implement URL filtering via FirePower and a 5506. The blocking works fine, it blocks http with a nice page saying the site is blocked the https sites dont get that page at all, it just timeout.

Is there any way around this?

What is the best way to block https sites 2016?

YFZblu

What you likely need here is SSL Inspection. I would start by finding out if your device supports it.

Iristheangel

You can reset the connection but you can't return a splash page with Firepower for SSL pages (unless you decrypt on the ASA which I don't recommend or have an appliance offload SSL) or you just have it reset the connection without the splash page

YFZblu

Why don't you recommend decrypting on the ASA? Resource consumption?

Iristheangel

Yep. Pretty much resources. Every single firewall out there that does it has a big performance hit. You can create custom rules to decrypt only certain things or maybe like a web server in your DMZ but I don't recommend with ANY vendor to try to do SSL decryption + all the other NGIPS/NGFW/Anti-Malware/etc solutions on the same box. From what I've seen in performance testing on various vendors and in real life, you get pages with PFS and other things and it'll sink your performance. It's better to offload that to dedicated appliance if you absolutely MUST and there's still some legal issues when it comes to decryption of everything (i.e. users visiting their banking sites, SSNs, etc).

If you absolutely MUST decrypt, get an appliance dedicated to offloading decryption and also make sure you understand the legal issues of doing so. I would probably try as hard as I can to safely protect at the network level but also harden and make sure endpoints have visibility and are secure.

636-555-3226

+1 for Iristheangel. Get Legal's sign-off, or if you don't have a Legal, HR. Lots of privacy concerns, esp. in places like Europe. Just logging a European's generic web activity (let alone decrypting things they thought were "SSL" secured) could be considered a violation of their privacy and lead to bad voodoo if any of them want to make a stink about it. Lots of murkiness around modern privacy and it won't get clearer any time soon....

Patdem

Iristheangel wrote: »

Yep. Pretty much resources. Every single firewall out there that does it has a big performance hit. You can create custom rules to decrypt only certain things or maybe like a web server in your DMZ but I don't recommend with ANY vendor to try to do SSL decryption + all the other NGIPS/NGFW/Anti-Malware/etc solutions on the same box. From what I've seen in performance testing on various vendors and in real life, you get pages with PFS and other things and it'll sink your performance. It's better to offload that to dedicated appliance if you absolutely MUST and there's still some legal issues when it comes to decryption of everything (i.e. users visiting their banking sites, SSNs, etc).

If you absolutely MUST decrypt, get an appliance dedicated to offloading decryption and also make sure you understand the legal issues of doing so. I would probably try as hard as I can to safely protect at the network level but also harden and make sure endpoints have visibility and are secure.

I was wondering if you had any ideas to offload the traffic to? The ASA 5515x SFR/FMC takes such a performance hit as you daid it just is not an option. Any advice would be great.

NotHackingYou

I have done this with Palo and Websense and it's not a big deal if the box is appropriately sized. You need an SSL proxy with a locally installed trusted root on every machine that you can issue certs from on the proxy.

Iristheangel

Still going to have some problems and there's going to be a growing number of items that can't be decrypted outright or packets that prevent your IPS from functioning as effectively. I did a TLS workshop recently and I'll sort of cut out some of the notes I took from it:

SSL Offloading vs. TLS Decryption
TLS Decryption
· Client to Internet
· Typically a forwarding proxy, NGFW
or dedicated appliance (i.e.A10)
· Certificate is “spoofed” and resigned

SSL Offloading
· Internet to Server
· A “known key” is exported from the server to a device to terminate TLS
· Typically used on a loadbalancer, reverse proxy, NGFW or dedicated appliance (i.e. A10)

Some things to be aware of:

PFS – Perfect Forward Secrecy
• Using RSA key for authentication and key generation is putting our traffic vulnerable to offline attacks
• Someone might record out traffic and years later, compromise our Server, get the private key and decrypt all traffic.
• Diffie-Hellman (DH) is a method to prevent this, but:
• With static DH, the DH parameters are actually again the private key…
• DHE is using unique keys for every session, this is called Perfect Forward Secrecy
• DHE Ciphers use PFS
• But are heavy to compute…L
• ECDHE Ciphers use Ecliptic Curves and provide PFS
• Much better performance J
• Especially in Software


Certificate Pinning – RFC 7469

• Method to actually compare the Certificate presented from the Server to a “stored” CA Certificate on the Client. Requires a method to ensure the Client is running the latest Version of your Software
• Applies to centrally updated Applications that connect to predictable Servers
• Two ways to do it:
• Incorporate a static list in the application, which of the CA Certificates is expected to be used for signing the server certificate
• Send a new “Header”(HPKP) to signal within the TLS Handshake that the client should PIN a certain public key for a certain amount of time
• Chrome connecting to gmail.com, twitter, FF connecting to mozilla.org

HTTP/2 – Features & Characteristics
• Header compression
• True multiplexing
• Re-Use of TCP connections
• Important with domain sharding (same ip, different domains)
• Browser limits connection per domain -> use subdomains: i.ytimg.com for icons, s.ytimg.com for stylesheets
• Server pushing content to the client
• Prioritization of streams
• Binary Format
HTTP2 and TLS
Multiplex requests and responses over single TCP connection
• More efficient object retrieval
• One TCP connection to each server (avoids TCP & TLS setup delays)
All browsers only attempt HTTP2 over TLS
• Chrome, Firefox, Safari
• Avoids difficult fallback code (like was necessary with HTTP 1.1 and middleboxes)
• Upgrades to HTTP2 using TLS extension
• Saves round trip of using HTTP’s “Upgrade:” header
Page load time: HTTP2-over-TLS is equivalent to (plaintext) HTTP
• Eliminates TLS page load time penalty
• Pages loading faster
• More usage of TLS
• Less TCP connections but longer lifetime
• HTTP/2 growing fast, websites change from SPDY to HTTP/2
• SPDY has proven that a new protocol can grow fast
HTTP/2 challenges on
proxy/ngfw/<insert nw-device here…>
• Browser only using HTTP/2 encrypted with TLS
• Binary Format & Header Compressions
need to be parsed (no more cleartext)
• Single TCP connection reuse (URI might change during one connection)
• Single TCP connection reuse
• Domain must be in certificate and resolve to same IP

TLS 1.3 – Draft - https://tools.ietf.org/html/draft-ietf-tls-tls13-11
• Remove of RSA authentication mode
• Using DHE / ECDHE instead for PFS
• Reducing overhead by using a 1-RTT handshake
• Fallback to normal handshake if client cannot handle it
• Working on a 0-RTT Session resumption
• Remove non-AEAD Ciphers (CBC), compression, RC4, MD5, SHA224
• Encrypting more values in the handshake
• Certificate Extensions such as CN & SAN
The requested hostname is usually detected by one of several methods:
Transparent Request (Transparent proxy, FW)

1. Check the SNI Name in the TLS Client hello
2. Check the SAN fields in the Server Certificate
3. Check the CN Field in the Subject of the Server Certificate

Explicit Proxy

1. Usually get the hostname from the CONNECT REQUEST
2. Check SNI, then SAN, then CN field

TLS 1.3 challenges on
proxy/ngfw/<insert nw-device here…>
• CN & SAN extensions are encrypted by DH
• We can only rely on SNI to know if we should decrypt or not
• SNI can be spoofed…

QUIC
• Google championed protocol to reduce latency
• UDP 80 & 443
• Encryption, congestion control and some HTTP/2 things (stream handling) move to QUIC
• 0-RTT connection handshake
• 0-RTT encryption handshake
• Connections survive IP address change
• Always encrypted
• Mostly fixes head of line blocking
• FEC (Forward Error Correction) data recovery
• Multiplexed streams
• Sharing connection across domains
• HPACK header compression
• Stream prioritization
• Flow Control
• Server*initiated streams
• HTTP response header
• Alternate-Protocol: 443:quic
• Client establishes QUIC connection in the
background
• Fully supported in Chrome browser
• Client’s can cache if server supports QUIC
QUIC Multipath
• QUIC using similar multipath concept
• Flows can be distributed among several UDP connections
• Can be established or torn down dynamically
• Can go different paths in the network
• Can use IPv4 and IPv6 on different connections

IPS, Proxy, and NGFW Challenges:
• QUIC is always encrypted
• QUIC is using multiplexed streams accross multiple paths
• Will happily use IPv4 and IPv6 concurrently
• If QUIC is not understood, connections look like unrelated UDP connections
• QUIC can be initiated from client and from server
• Where is now inbound and where is outbound?


Conclusion:
• Decryption of TLS is still possible in many cases…
• ....but new technologies like certificate pinning prevent decryption
• New Protocols like HTTP/2, TLS 1.3 and QUIC make it hard or even impossible for network devices to scan & detect threats in the network with deep packet inspection (DPI)

sucanushie

We do decryption on our WSA's. We don't decrypt categories such as Gov or Banking. We also have a custom category for sites that just don't play nice with the decryption. It works quite well.

The issue that we are running into now as we roll out new ASA's with firepower, all of our web traffic in the FMC is sourced by the WSA as expected but not desired.

In 6.1 they did introduce support for X-Forward-For header, however I have yet to get it working.

Iristheangel

X-Forward works. I've got it running in my lab, sucanushie

MitM

That's some good note taking

sucanushie

Iristheangel wrote: »

X-Forward works. I've got it running in my lab, sucanushie

Really? Hrmm...

I create a NAMP and made sure it was set up correctly, and enabled it on the WSA.

I have a TAC case open, just waiting to hear back

Iristheangel

PM me the TAC case. It's been working since at least 6.0 because I've had to set it up for POVs and such.

NotHackingYou

@Iris - great notes. Let's talk about these one at a time because I feel like the whole picture isn't represented here.

PFS - Yes, I agree that you can't do PFS when you have an SSL Forward proxy. The business must decide what they want to do here. I agree that you also must typically drop to TLS 1.1 - but some support TLS 1.2 (like Palo). Most sites support a variety of encryption protocols and ciphers and the firewall/proxy will be able to negotiate an acceptable set.

Pinning RFC 7469 - Pinning allows for a locally installed trusted root to override the pin. This is to accommodate for enterprise proxies. Pinning does not break a forward proxy

TLS 1.3, QUIC, IPv6, etc - Yes, these are challenges. Most sites will allow negotiation back to standard means. You can decide if you want to allow QUIC/SPDY in security policy. Like everything else, this is a decision for the big cheeses at the business.

So, in short, my opinion is that this is totally doable at an enterprise with the right gear. For the most part, users won't notice and it isn't any slower or less reliable. The insight into traffic for things like DLP and malware makes it more than worth it. This is my opinion because I have personally implemented and supported this technology more than once.

When it comes to big changes in IT, I firmly believe that technical experts must be able to describe the options to management in a way that management can choose what they feel is best for how the business functions. This is one of those that rests in that space.

Iristheangel

PFS and TLS 1.3, QUIC, etc - Agreed. It's a challenge and a pain in the arse for the business to decide on those.

Certificate pinning - Still some challenges when it comes to BYOD, non-corp devices, mobile, etc - pretty much either adding on some other technology to deliver the cert (NAC, MDM, etc) and make changes on the device or completely bypass which adds it's own layer.

Not saying that decryption doesn't have it's time and place - but I don't think it's the magic bullet of detection people hope it will be. I think we're going to have to shift away from it more and more in the future and start thinking about other ways of detecting compromise (i.e. moving the focus more on endpoint security, hardening, traffic anomaly detection, etc).

My point with the original post was that trying to put everything on your FW to offload SSL decryption is going to be a performance hit for sure. I've seen PA-7050's performance numbers tank in production because people were like "DECRYPT ALL THE THINGS!!!!!!!" Reality is that if you're going to try to shoot for that, you're probably going to have to go with something like a dedicated appliance and stop looking for magic boxes that do everything on them. It's more about building an architecture that supports the business' goals instead of trying to "silver bullet" it which is typically what I see when people are like "Does my firewall do... [insert 50,000 erroneous things it may do but it's not really what you necessarily want to use it for]." Not that I'm saying you're saying that - from your replies, you mention offloading it elsewhere.

NotHackingYou

@Iris, agree 100%

MitM

I've also been running decryption on a number of Palo Alto firewalls (PA-500, PA-3020) with no issues, pretty much for everything except for banking, health, government, shopping. No performance issues so far, but I also think it also depends on the amount of traffic passing through.

I know when we looked at CheckPoint, they said no way, you'd have to offload to a different device

NotHackingYou

MitM wrote: »

I've also been running decryption on a number of Palo Alto firewalls (PA-500, PA-3020) with no issues, pretty much for everything except for banking, health, government, shopping. No performance issues so far, but I also think it also depends on the amount of traffic passing through.

I know when we looked at CheckPoint, they said no way, you'd have to offload to a different device

The last time I had to do this with Checkpoint we used Websense (now Raytheon) as the proxy piece. This combo was less effective than Palo's approach at stopping malware. Websense had better reporting on what users were doing but allowed too much junk through. Checkpoint was largely ineffective, even when malware wasn't coming back over https, the IPS 'blade' never once caught it. I also found maintaining Checkpoint's IPS/IDS signatures was cumbersome and time consuming.

When Angler was new, I observed a site which would deliver Locky with almost every hit. It would sail through Checkpoint and Websense every time. I set up a PA-200 in virtual wire mode and it stopped it, first try.

Edited to add: I was using PA 5050 in production, PA 200 in lab and @home and PA Virtual in PA training. Never saw a performance hit but we are choosy about what to decrypt. I run a PA 200 at home.

Iristheangel

MitM wrote: »

I've also been running decryption on a number of Palo Alto firewalls (PA-500, PA-3020) with no issues, pretty much for everything except for banking, health, government, shopping. No performance issues so far, but I also think it also depends on the amount of traffic passing through.

I know when we looked at CheckPoint, they said no way, you'd have to offload to a different device

Yes, definitely depends on the amount of traffic. i.e. if a firewall is sized to be able to push 2GB through and then you add SSL decryption but are only pushing ~100MB on average anyways, you may never notice a performance hit because a 30-40% hit on a 2GB FW isn't going to be noticed at that traffic rate. I know NetworkWorld and NSS Labs did their own tests over the years. From the Network World Article:
"[FONT=&quot]SSL encryption is compute-intensive. Even with dedicated silicon for the task, the PA-5060, like virtually all other high-end firewalls, is a far slower performer when handling SSL traffic.[/FONT]

[FONT=&quot]


[/FONT]

[FONT=&quot]The PA-5060 generally moved traffic at around 7.5G to 7.6Gbps in every test case. We initially suspected that the nearly identical rates were caused by some limit in our test gear. But back-to-back tests of the Spirent Avalanche equipment without the PA-5060 in line moved traffic at around 8.6Gbps, faster than the firewall. So the test gear wasn't the bottleneck. (See our test methodology.)[/FONT]
[FONT=&quot]Rates for SSL traffic are higher than those for cleartext traffic, except in the firewall-only test case. This suggests the PA-5060 does less inspection of SSL traffic by default. Palo Alto's engineers confirmed this, but only for the particular traffic generated by Spirent Avalanche; in this case, the PA-5060 simply classified the traffic as type "SSL" and did no further inspection. Palo Alto says there are cases where the PA-5060 can detect certain attacks hidden in SSL traffic, but we did not attempt to verify that claim.[/FONT]
[FONT=&quot]The PA-5060 does support decryption of SSL traffic for deeper inspection, but that feature comes with a heavy performance cost. When doing SSL decryption, rates fell to 986Mbps when the PA-5060 acted as a firewall, and just 108Mbps with all UTM features enabled.[/FONT]
[FONT=&quot]Both numbers are a long way off from the 17-Gbps rates we saw in the cleartext tests, or even the 7.5-Gbps rates in the SSL tests without decryption. If higher-speed decryption of SSL is required, network managers might consider a purpose-built appliance such as those from Netronome and other vendors."

Also NSS Labs competitive testing for SSL Decryption: https://www.nsslabs.com/linkservid/13C7BD87-5056-9046-93FB736663C0B07A/[/FONT]

NotHackingYou

Agreed, it will be slower than not decrypting but this applies only to that specific rule. On Palo and Forti, decrypt is applied only to specific traffic. IPS/IDS is applied (or not applied) on each firewall rule. This allows you to put the right amount of inspection in a very granular place. Say you have trusted traffic from an application to a database server that needs 10gb throughpout. You probably aren't going to apply decryption and a protection profile there.

Meanwhile, slowing your internet pipe down at the firewall probably doesn't make a difference because you probably don't have a 10gb internet connection.

Edited to add some clarification:

On Palo and Forti, decrypt is a separate ruleset just like the security ruleset. Protection profiles (IDS/IPS) is applied based on each rule. You would be remiss to write a decrypt rule that tries to decrypt all traffic. Usually, the decrypt should match the security rule that allows web browsing.

On checkpoint, IDS/IPS is per firewall so ALL the traffic takes the IPS/IDS hit unless you write specific exemptions.

MitM

I agree with you both. In our case, we need the visibility.

Sorry to the OP for turning this into a Palo discussion.