privacy, security

Understanding the Limitations of HTTPS

A colleague recently forwarded me an article about the hazards of browsing on public WiFi with the question: “Doesn’t HTTPS fix this?” And the answer is, “Yes, generally.” As with most interesting questions, however, the complete answer is a bit more complicated.

HTTPS is a powerful technology for helping secure the web; all websites should be using it for all traffic.

If you’re not comfortable with nitty-gritty detail, stop reading here. If your takeaway upon reading the rest of this post is “HTTPS doesn’t solve anything, so don’t bother using it!” you are mistaken, and you should read the post again until you understand why.

HTTPS is a necessary condition for secure browsing, but it is not a sufficient condition.

There are limits to the benefits HTTPS provides, even when deployed properly. This post explores those limits.

Deployment Limitations

HTTPS only works if you use it.

In practice, the most common “exploit against HTTPS” is failing to use it everywhere.

Specify HTTPS:// on every URL, including URLs in documentation, email, advertisements, and everything else. Use Strict-Transport-Security (preload!) and Content-Security-Policy’s Upgrade-Insecure-Requests directive (and optionally Block-All-Mixed-Content) to help mitigate failures to properly set URLs to HTTPS.

Mixed Content – By default, browsers will block non-secure scripts and CSS (called “Active Mixed Content”) from secure pages. However, images and other “Passive Mixed Content” are requested and displayed; the page’s lock icon is silently hidden.

Non-secure Links – While browsers have special code to deal with Active and Passive mixed content, most browsers do nothing at all for Latent Mixed Content, where a secure page contains a link to a non-secure resource. Email trackers are the worst.

Privacy Limitations

SNI / IP-Address – When you connect to a server over HTTPS, the URL you’re requesting is encrypted and invisible to network observers. However, observers can see both the IP address you’re connecting to, and the hostname you’re requesting on that server (via the Server Name Indication ClientHello extension).

TLS 1.3 proposes a means of SNI-encryption (Encrypted SNI), but (unless you’re using something like Tor) an observer is likely to be able to guess which server you’re visiting using only the target IP address. In most cases, a network observer will also see the plaintext of the hostname when your client looks up its IP address via the DNS protocol (DNS over HTTPS aims to address).

image

Data Length – When you connect to a server over HTTPS, the data you send and receive is encrypted. However, in the majority of cases, no attempt is made to mask the length of data sent or received, meaning that an attacker with knowledge of the site may be able to determine what content you’re browsing on that site. Protocols like HTTP/2 offer built-in options to generate padding frames to mask payload length, and sites can undertake efforts (Twitter manually pads avatar graphics to fixed byte lengths) to help protect privacy. More generally, traffic analysis attacks make use of numerous characteristics of your traffic to attempt to determine what you’re up to; these are used by real-world attackers like the Great Firewall of China. Attacks like BREACH make use of the fact that when compression is in use, leaking just the size of data can also reveal the content of the data; mitigations are non-trivial.

Ticket Linking – TLS tickets can be used to identify the client. (Addressed in TLS1.3)

Referer Header – By default, browsers send a page’s URL via the Referer header (also exposed as the document.referrer DOM property) when navigating or making resource requests from one HTTPS site to another. HTTPS sites wishing to control leakage of their URLs should use Referrer Policy.

Server Identity Limitations

Certificate Verification – During the HTTPS handshake, the server proves its identity by presenting a certificate. Most certificates these days are issued after what’s called “Domain Validation”, a process by which the requestor proves that they are in control of the domain name listed in the certificate.

This means, however, that a bad guy can usually easily get a certificate for a domain name that “looks like” a legitimate site. While an attacker shouldn’t be able to get a certificate for https://paypal.com, there’s little to stop them from getting a certificate for https://paypal.co.com. Bad guys abuse this.

Some sites try to help users notice illegitimate sites by deploying Extended Validation (EV) certificates and relying upon users to notice if the site they’re visiting has not undergone that higher-level of vetting. Sadly, a number of product decisions and abysmal real-world deployment choices mean that EV certificates are of questionable value in the real-world.

Even more often, attackers rely on the fact that users don’t understand URLs at all, and are willing to enter their data into any page containing the expected logos:

image

One Hop – TLS often protects traffic for only one “hop.” For instance, when you connect to my https://fiddlerbook.com, you’ll see that it’s using HTTPS. Hooray!

What you didn’t know is that this domain is fronted by Cloudflare CDN’s free tier. While your communication with the Content Delivery Network is secure, the request from the CDN to my server (http://fiddlerbook.com) is over plain HTTP because my server doesn’t have a valid certificate[1]. A well-positioned attacker could interfere with your connection to the backend site by abusing that non-secure hop. Overall, using Cloudflare for HTTPS fronting improves security in my site’s scenario (protecting against some attackers), but browser UI limits mean that the protection probably isn’t as good as you expected. Here’s a nice video on this.

Multi-hop scenarios exist beyond CDNs; for instance, a HTTPS server might pull in a HTTP web service or use a non-secure connection to a remote database on the backend.

DOM Mixing – When you establish a connection to https://example.com, you can have a level of confidence that the top-level page was delivered unmolested from the example.com server. However, returned HTML pages often pull in third-party resources from other servers, whose certificates are typically not user-visible. This is especially interesting in cases where the top-level page has an EV certificate (“lighting up the green bar”), but scripts or other resources are pulled from a third-party with a domain-validated certificate.

Sadly, in many cases, third-parties are not worthy of the high-level of trust they are granted by inclusion in a first-party page.

Server Compromise – HTTPS only aims to protect the bytes in transit. If a server has been compromised due to a bug or a configuration error, HTTPS does not help (and might even hinder detection of the compromised content, in environments where HTTP traffic is scanned for malware by gateway appliances, for instance). HTTPS does not stop malware.

Server Bugs – Even when not compromised, HTTPS doesn’t make server code magically secure. In visual form:
image

NoSilverBullets

Client Identity Limitations

Client Authentication – HTTPS supports a mode whereby the client proves their identity to the server by presenting a certificate during the HTTPS handshake; this is called “Client Authentication.” In practice, this feature is little used.

Client Tampering – Some developers assume that using HTTPS means that the bytes sent by the client have not been manipulated in any way. In practice, it’s trivial for a user to manipulate the outbound traffic from a browser or application, despite the use of HTTPS.

Features like Certificate Pinning could have made it slightly harder for a user to execute a man-in-the-middle attack against their own traffic, but browser clients like Firefox and Chrome automatically disable Certificate Pinning checks when the received certificate chains to a user-installed root certificate. This is not a bug.

In some cases, the human user is not a party to the attack. HTTPS aims to protect bytes in transit, but does not protect those bytes after they’re loaded in the client application. A man-in-the-browser attack occurs when the client application has been compromised by malware, such that tampering or data leaks are performed before encryption or after decryption. The spyware could take the form of malware in the OS, a malicious or buggy browser extension, etc.

Real-world Implementation Limitations

Early Termination Detection – The TLS specification offers a means for detecting when a data stream was terminated early to prevent truncation attacks. In practice, clients do not typically implement this feature and will often accept truncated content silently, without any notice to the user.

Validation Error Overrides – HTTPS deployment errors are so common that most user-agents allow the user to override errors reported during the certificate validation process (expired certificates, name mismatches, even untrusted CAs etc). Clients range in quality as to how well they present the details of the error and how effectively they dissuade users from making mistakes.

Further Reading

-Eric

[1] A few days after posting, someone pointed out that I can configure Cloudflare to use its (oddly named) “Full” HTTPS mode, which allows it to connect to my server over HTTPS using the (invalid) certificate installed on my server. I’ve now done so, providing protection from passive evesdroppers. But you, as an end-user, cannot tell the difference, which is the point of this post.

Standard
security

Seek and Destroy Non-Secure References Using the moarTLS Analyzer

tl;dr: I made a Chrome Extension that finds security vulnerabilities.
It’s now available for Firefox too!

To secure web connections, TLS-enabling servers is only half the battle; the other half is ensuring that TLS is used everywhere.

Unfortunately, many HTTPS sites today include insecure references that provide an network-based attacker the opportunity to break into the user’s experience as they interact with otherwise secure sites. For instance, consider the homepage of Fidelity Investments:

Fidelity Homepage

You can see that the site has got the green lock and it’s using an EV certificate such that the organization’s name and location are displayed. Looks great! Even if you loaded this page over a coffee shop’s WiFi network, you’d feel pretty good about interacting with it, right?

Unfortunately, there’s a problem. Hit F12 to open the Chrome Developer Tools, and use the tool to select the “Open An Account” link in the site’s toolbar. That’s the link you’d click to start giving the site all of your personal information.

href attribute containing HTTP

Oops. See that http:// hiding out there? Even though Fidelity delivered the entire homepage securely, if a user clicks that link, a bad guy on the network has the opportunity to supply his own page in response! He can either return a HTML response that asks the victim for their information, or even redirect to a phony server (e.g. https://newaccountsetup.com) so that a lock icon remains in the address bar.

Adding insult to injury, what happens if a bad guy doesn’t take advantage of this hole?

    GET http://www.fidelity.com/open-account/overview
    301 Redirect to https://www.fidelity.com/open-account/overview

That’s right, in the best case, the server just sends you over to the HTTPS version of the page anyway. It’s as if the teller at your bank carried cash deposits out the front door, walking them around the building before reentering the bank and carrying them to the vault!

Okay, so, what’s a security-conscious person to do?

First, recognize the problem. If you stumble across a “HTTP” reference, it’s a security bug. Either fix it, or complain to someone who can.

Next, actively seek-and-destroy non-secure references.

A New Category of Mixed Content?

Web developers are familiar with two categories of Mixed Content: Active Mixed-Content (e.g. script) which is blocked by default, and Passive Mixed-Content (images, etc), which browsers tend to allow by default, usually with the penalty of removing the lock from the address bar.

However, secure pages with non-secure links don’t trigger ANY warning in the browser.

For now, let’s call the problem described in this post Latent Mixed Content.

Finding Latent Mixed Content

Finding HTTP links isn’t hard, but it can be tedious. To that end, between late-night feedings of my newborn, I’ve been learning Chrome’s extension model– a wonderful breath of fresh air after years of hacking together COM extensions in IE. The result of that effort is now available for your bug-hunting needs.

Download the moarTLS Chrome Extension.

#icanhazthecodez? Sure.

The extension adds an unobtrusive button to Chrome’s toolbar. The extension is designed to have little-to-no impact on the performance or operation of Chrome unless you actively interact invoke the extension.

When the Button button is clicked, the extension analyzes the current page to find out which hyperlinks (<a> elements) are targeting a non-secure protocol. If the page is free of non-secure links, the report is green:

PayPal showing all green

If the current page’s host sends a HTTP Strict Transport Security directive, a green lock is shown next to the hostname at the top of the report: green lock. Click the hostname to launch the SSLLabs Server Test for the host to explore what secure protocols are supported and find any errors in the host’s certificate or TLS configuration.

If the page contains one or more non-secure links, the report gets a yellow background and the non-secure links are listed:

Yellow warning report, 4 non-secure links

The non-secure links in the content of the page are marked in red for ease-of-identification:

Four Red links in the page

Alt+Click (or Ctrl+Click) on any entry in the report to cause the extension to probe the target hostname to see whether a HTTPS connection to the listed hostname is possible. If a HTTPS connection attempt succeeds, a grey arrow is shown. If the connection attempt fails (indicating that the server is only accessible via HTTP), a red X is shown:

Three grey up-arrows, one red X

If the target is accessible over HTTPS and the response includes a HTTP Strict Transport Security header, the grey arrow is replaced with a green arrow:

Green Arrow

Note: Accepting HTTPS connections alone doesn’t necessarily indicate that the host completely supports HTTPS—the secure connection could result in error pages or redirections to a redirection back to HTTP. But a grey arrow indicates that at least the server has a valid certificate and is listening  for TLS connections on port 443. Before updating each link to HTTPS, verify that the expected page is returned.

Returning to our original example, Fidelity’s non-secure links are readily flagged:

Non-secure links

If you read your email in a web client like GMail or Hotmail, you can also check whether your HTML emails are providing secure links:

Insecure credit card

 

HTTP-Delivered Pages

The examples above presuppose that the current page was delivered over HTTPS. If the page itself was delivered non-securely (over HTTP), invoking the moarTLS extension colors the background of the page itself red. In the report flyout, the hostname shown at the top is prefixed with http/. The icon adjacent to the domain name will either be an up-arrow:

…indicating that the host accepts HTTPS connections, or a red-X, indicating that it does not:

The extension exposes the option (simply right-click the icon) to flip insecurely-delivered images:

moarTLS options

When this option is enabled, images delivered insecurely are flipped vertically, graphically demonstrating one of the least malicious actions a Man-in-the-Middle could undertake when exploiting a site’s failure to use HTTPS.

Upside-down images

The Warn on non-secure downloads option instructs the extension to warn you when a file download occurs if either the page linking to a download, or the download itself, used a non-secure protocol:

Non-secure download

Non-secure file downloads are extremely dangerous; we’ve already seen attacks in-the-wild where a MITM intercepts such requests and responds with malware-wrapped replacements. Authenticode-signing helps mitigate the threat, but it’s not available everywhere, and it should be bolstered with HTTPS.

Limitations

This extension has a number of limitations; some will be fixed in future updates.

False Negatives

  • moarTLS looks only at the links in the markup. JavaScript could intercept a link click and cause an non-secure navigation when the user clicks a link with an otherwise secure HREF.
  • moarTLS does not currently check the source of CSS background images.
  • moarTLS does not currently mangle insecurely-delivered fonts, audio, or video.
  • moarTLS only evaluates links currently in the page. If links are added later (e.g. via AJAX calls), they’re not marked unless you click the button again.

False Positives

  • moarTLS looks only at the links in the markup. JavaScript could intercept a link click and cause an secure navigation when the user clicks a link with an otherwise non-secure HREF. Arguably this isn’t a false-positive because the user may have JavaScript disabled.
  • moarTLS looks only at the link, and does not exempt links which are automatically upgraded by the browser due to a HSTS rule. Arguably this isn’t a false-positive because not all browsers support HSTS, and the user may copy a URL to a non-browser client (e.g. curl, wget, etc).
  • moarTLS isn’t aware of upgrade-insecure-requests, although that only helps for same-origin navigations. Arguably this isn’t a false-positive because not all browsers support this CSP directive, and the user may copy a URL to a non-browser client (e.g. curl, wget, etc).
  • moarTLS isn’t aware of block-all-mixed-content. Arguably this isn’t a false-positive because not all browsers support this CSP directive, and the user may copy a URL to a non-browser client (e.g. curl, wget, etc).

Q&A

Q1. Why is this an extension? Shouldn’t it be in the Developer Tools’ Security pane, which currently flags active and passive mixed content:

Security Report

A1. Great idea. :)

Q2. How do I examine “popups” which don’t show the Chrome toolbar?

No toolbar

A2. Use “Show as Tab” on the system menu:

System Menu command Show As Tab

 

Q3. How much of the Web is HTTPS today?

A3. See https://security.googleblog.com/2016/03/securing-web-together_15.html

Standard