CORS and Vary

Yesterday, I started looking a site compatibility bug where a page’s layout is intermittently busted. Popping open the F12 Tools on the failing page, we see that a stylesheet is getting blocked because it lacks a CORS Access-Control-Allow-Origin response header:

NoStylesheetCORS

We see that the client demands the header because the LINK element that references it includes a crossorigin=anonymous directive:

crossorigin="anonymous" href="//s.axs.com/axs/css/90a6f65.css?4.0.1194" type="text/css" />

Aside: It’s not clear why the site is using this directive. CORS is required to use  SubResource Integrity, but this resource does not include an integrity attribute. Perhaps the goal was to save bandwidth by not sending cookies to the “s” (static content) domain?

In any case, the result is that the stylesheet sometimes fails to load as you navigate back and forward.

Looking at the network traffic, we find that the static content domain is trying to follow the best practice Include Vary: Origin when using CORS for access control.

Unfortunately, it’s doing so in a subtly incorrect way, which you can see when diffing two request/response pairs for the stylesheet:

VaryDiff

As you can see in the diff, the Origin token is added only to the response’s Vary directive when the request specifies an Origin header. If the request doesn’t specify an Origin, the server returns a response that lacks the Access-Control-* headers and also omits the Vary: Origin header.

That’s a problem. If the browser has the variant without the Access-Control directives in its cache, it will reuse that variant in response to a subsequent request… regardless of whether or not the subsequent request has an Origin header.

The rule here is simple: If your server makes a decision about what to return based on a what’s in a HTTP header, you need to include that header name in your Vary, even if the request didn’t include that header.

If your response specifies Vary: Origin to ensure that the browser rechecks with you before reusing a response, your server had better not return a HTTP/304 Not Modified response to a request from a different Origin without *also* updating the Access-Control-Allow-Origin to match the new request context. Otherwise, your cached response is not going to be readable in that new context.

-Eric

PS: This seems to be a pretty common misconfiguration, which is mentioned in the fetch spec:

CORS protocol and HTTP caches

If CORS protocol requirements are more complicated than setting `Access-Control-Allow-Origin` to * or a static origin, `Vary` is to be used.

Vary: Origin

In particular, consider what happens if Vary is not used and a server is configured to send Access-Control-Allow-Origin for a certain resource only in response to a CORS request. When a user agent receives a response to a non-CORS request for that resource (for example, as the result of a navigation request), the response will lack Access-Control-Allow-Origin and the user agent will cache that response. Then, if the user agent subsequently encounters a CORS request for the resource, it will use that cached response from the previous non-CORS request, without Access-Control-Allow-Origin.

But if Vary: Origin is used in the same scenario described above, it will cause the user agent to fetch a response that includes Access-Control-Allow-Origin, rather than using the cached response from the previous non-CORS request that lacks Access-Control-Allow-Origin.

However, if Access-Control-Allow-Origin is set to * or a static origin for a particular resource, then configure the server to always send Access-Control-Allow-Origin in responses for the resource — for non-CORS requests as well as CORS requests — and do not use Vary.

Be skeptical of client-reported MIME Content-Types

Over the 14 years that I’ve been working on browsers and the web platform, I’ve seen a lot of bugs where the client’s configuration causes a problem with a website.

By default, Windows maintains File Extension to Content Type and Content Type to File Extension mappings mappings in the registry. You can find the former mappings in subkeys named for each file extension, e.g. HKEY_CLASSES_ROOT\.ext, and the latter as subkeys under the HKEY_CLASSES_ROOT\MIME\Database\Content Type key:

PDFMapping

These mappings are how Internet Explorer, Edge, and other browsers know that a file delivered as Content-Type: application/pdf should be saved with a .pdf extension, and that a local file named example.html ought to be treated as Content-Type: text/html.

Unfortunately, these mappings are subject to manipulation by locally-installed software, which means you might find that installing Microsoft Excel causes your .CSV file upload to have a Content-Type of application/vnd.ms-excel instead of the text/csv your website was expecting.

Similarly, you might be surprised to discover that some popular file extensions do not have a MIME type registered by default on Windows. Perhaps the most popular of these is files in JavaScript Object Notation format; these generally should have the file extension .json and a MIME type of application/json but Windows treats these as an unknown type by default.

Today, I looked at a site which allows the user to upload a JSON file containing data exported from some other service. The upload process fails in Edge with an error saying that the file must be JSON. Looking at the script in the site, it contains the following:

validateFile = function(file) {
  if (file.type !== "application/json") // BUG BUG BUG
    { alert('That is not a valid json file.'); return; }

This function fails in Edge– the file.type attribute is the empty string because Windows has no mapping between .json and application/json.

This site usually works in Chrome because Chrome has a MIME-type determination system which first checks a fixed list of mappings, then, if no fixed mapping was found, consults the system registry, and finally, if the registry does not specify a MIME type for a given extension, Chrome consults a “fallback” list of mappings (kSecondaryMappings), and .JSON is in that final fallback list. However, even Chrome users would be broken if the file had the wrong extension (e.g. data.jso) or if the user’s registry contained a different mapping (e.g. .json=>”text/json”).

As a consequence, client JavaScript and server-side upload processing logic should be very skeptical of the MIME type contained in the file.type attribute or Content-Type header, as the MIME value reported could easily be incorrect by accident (or malice!).

-Eric Lawrence
PS: End users can workaround the problem with sites that expect particular MIME types for JSON by importing the following Registry Script (save the text as FixJSON.reg and double-click the file):

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\.json]
"Content Type"="application/json"

[HKEY_CLASSES_ROOT\MIME\Database\Content Type\application/json]
"Extension"=".json"

 

Edge EV UI Requires SmartScreen

A user recently noticed that when loading Paypal.com in Microsoft Edge, the UI shown was the default HTTPS UI (a gray lock):

Non-EV-UI-For-Paypal

Instead of the fancier “green” UI shown for servers that present Extended Validation (EV) certificates:EV-for-Paypal

The user observed this on some Windows 10 machines but not others.

The variable that differed between those machines was the state of the Menu > Settings > Advanced > Windows Defender SmartScreen setting.

Edge only shows the green EV user interface when SmartScreen is enabled.

IE 11

Internet Explorer 11 on Windows 10 behaves the same way as prior versions of IE going back to IE7– the green EV UI requires either SmartScreen be enabled or that the option Tools > Internet Options > Advanced > Security > Check for Server Certificate Revocation be enabled.

Chrome

The Chrome team recently introduced a new setting, exposed via the chrome://flags/#simplify-https-indicator page, that controls how EV certificates are displayed in their Security Chip. A user (or a field trial) can configure sites with EV certificates to display using the default HTTPS UI.

ChromeEV

 

-Eric

Stop Spilling the Beans

I’ve written about Same Origin Policy a bunch over the years, with a blog series mapping it to the Read/Write/Execute mental model.

More recently, I wrote about why Content-Type headers matter for same-origin-policy enforcement.

I’ve just read a great paper on cross-origin infoleaks and current/future mitigations. If you’re interested in browser security, it’s definitely worth a read.

Building your .APP website with NameCheap and GitHub Pages–A Visual Guide

I recently bought a few new domain names under the brand new .app top-level-domain (TLD). The .app TLD is awesome because it’s on the HSTSPreload list, meaning that browsers will automatically use only HTTPS for every request on every domain under .app, keeping connections secure and improving performance.

I’m not doing anything terribly exciting with these domains for now, but I’d like to at least put up a simple welcome page on each one. Now, in the old days of HTTP, this was trivial, but because .app requires HTTPS, that means I must get a certificate for each of my sites for them to load at all.

Fortunately, GitHub recently started supporting HTTPS on GitHub Pages with custom domains, meaning that I can easily get a HTTPS site up in running in just a few minutes.

1. Log into GitHub, go to your Repositories page and click New:

2. Name your new repository something reasonable:

3. Click to create a simple README file:

4. Edit the file

5. Click Commit new file

6. Click Settings on the repository

7. Scroll to the GitHub Pages section and choose master branch and click Save:

8. Enter your domain name in the Custom domain box and click Save

9. Login to NameCheap (or whatever DNS registrar you used) and click Manage for the target domain name:

10. Click the Advanced DNS tab:

11. Click Add New Record:

 

12. Enter four new A Records for host of @ with the list of IP addresses GitHub pages use:

13. Click Save All Changes.

14. Click Add New Record and add a new CNAME Record. Enter the host www and a target value of username.github.io. Click Save All Changes: 

15. Click the trash can icons to delete the two default DNS entries that NameCheap had for your domain previously:

16. Try loading your new site.

  • If you get a connection error, wait a few minutes for DNS to propagate and re-verify the DNS records you just added.
  • If you get a certificate error, look at the certificate. It’s probably the default GitHub certificate. If so, look in the GitHub Pages settings page and you may see a note that your certificate is awaiting issuance by LetsEncrypt.org. If so, just wait a little while.

  • After the certificate is issued, your site without errors:

 

Go forth and build great (secure) things!

-Eric Lawrence

Fight Phish with Facebook (and Certificate Transparency)

As of April 30th 2018, Chrome now requires that all certificates issued by a public certificate authority be logged in multiple public Certificate Transparency (CT) logs, ensuring that anyone can audit all certificates that have been issued. (Update: Microsoft Edge 79+ also mandates CT).

CT logs allow site owners and security researchers to much more easily detect if a sloppy or compromised Certificate Authority has issued a certificate in error, because all certificates are published to a public, tamper-proof log that anyone can search. If you see a certificate for a site you own that you didn’t request, someone’s attacking you!

For instance, I own bayden.com, a site where I distribute freeware applications. I definitely want to hear about it if any CA issues a certificate for my site, because that’s a strong indication that my site’s visitors may be under attack. What’s cool is that CT also allows me to detect if someone got a certificate for a domain name that was suspiciously similar to my domain, for instance bȧyden.com.

There are some great tools to examine CT logs; my go-to is crt.sh which is simple and fast. For example, here’s the list of all of the certificates issued covering my domain.

Manual lookups are neat, but to improve security, I still have to actually pay attention to the CT logs, and who’s got time for that? Someone else’s computer, that’s who.

The folks over at Facebook Security have built an easy-to-use interface that allows you to subscribe to notifications any time a domain you care about has a new certificate issued. Just enter a hostname and decide what sorts of alerts you’d like:

CTMonitor

You can even connect their system into webhooks if you’re looking for something more elaborate than email, although mail works just fine for me:

Notification

Beyond Facebook, there will likely be many other CT Monitoring services coming online over the next few years. For instance, the good folks at Hardenize have already integrated one into their broader security monitoring platform.

The future is awesome.

-Eric

PS: If you want to reduce the risk of a rogue certificate in the first place, consider opting-in to Certificate Authority Authorization, a mechanism that declares which CAs are allowed to issue certificates for your site.

Going Offline with ServiceWorker

In the IE8 era, I had a brief stint as an architect on the IE team, trying to figure out a coherent strategy and a deployable set of technologies that would allow web developers to build offline-capable web applications. A few of those ideas turned into features, several turned into unimplemented patents, and a few went nowhere at all.

A decade later, it’s clear that ServiceWorker is going to be the core engine beneath all future top-tier web applications. ServiceWorker brings the power of Fiddler’s AutoResponder and FiddlerScript features to JavaScript code running directly within the user’s browser. Designed by real web developers for real web developers, it delivers upon scenarios that previously required native applications. And browser support is looking great:

ServiceWorker

As I started looking at ServiceWorker, I was concerned about its complexity but I was delighted to discover a straightforward, very approachable reference on designing a ServiceWorker-backed application: Going Offline by Jeremy Keith. The book is short (I’m busy), direct (“Here’s a problem, here’s how to solve it“), opinionated in the best way (landmine-avoiding “Do this“), and humorous without being confusing. As anyone who has received unsolicited (or solicited) feedback from me about their book knows, I’m an extremely picky reader, and I have no significant complaints on this one. Highly recommended.

Unfortunately, the book isn’t available at list price on Amazon, but buying directly from the publisher is straightforward. The EBook is $11 and the paperback+ebook bundle is $28.80+shipping.

-Eric

Google Chrome–Two(ish) Years In

My first year (2016) on Chrome was both exciting and challenging. Beyond the expected firehose of new things to learn, and foreseen challenges (a second son!), there were unforeseen challenges, like working as a SWE instead of a Developer Advocate, and a long illness.

Overall, my second year (2017) on Chrome was a bit smoother– I was still learning a ton every day, but I finally started to make non-trivial contributions as a developer.

I started to develop a sense of direction when navigating the millions of lines of code that make up Chrome’s multi-process architecture, and did the development work for HTTPBad Phase 2. Building that feature involved sending messages from Blink through Mojo to the Browser process and triggering UI updates, collecting metrics and rolling out the feature gradually via a Chrome Field Trial. Shipping the change on iOS (where Chrome must use WkWebView rather than Blink) was especially tricky as the architecture, toolchain, and testing are all quite different.

I briefly took ownership of maintaining Chrome’s HSTS Preload list and processes and worked with domain registry fTLD in getting the .bank and .insurance top-level-domains added to the preload list. Getting TLDs preloaded is a huge win for security and performance, and hopefully we’ll see more TLDs joining in the near future.

The single biggest productivity improvement of the year was moving from building locally to using Goma for dramatically faster builds. With the benefit of hindsight, I should’ve done this in 2016 and failing to do so was one of the biggest mistakes I’ve made.

Reproducing, reducing, and triaging security issues remained my favorite thing to do, and I got better at identifying which of Chrome’s many experts was best suited to investigate and develop patches. I also made a bunch of updates to Chrome’s public Security FAQ and other documentation, and published some notes on my research into Chrome’s XSS Auditor. I had the chance to do the pre-launch security review for a number of cool features around the product, and work with other Google teams on a number of HTTPS-related issues. I filed 161 issues.

The only constant when working on Chrome is change, and I spent a lot of time keeping up with changes in everything from architecture (Site Isolation!) to tooling, including the move from Reitveld to Gerrit for code review. I also kept the extensions I’ve developed up-to-date with changes and bugfixes, and code-reviewed a few publicly-available extensions, both good and evil.

Speaking of changes…

I’ve learned a ton over the last two years and I’m grateful for having had the opportunity to help keep users safe and work with such a talented group of passionate engineers. I still get dizzy when I think about the size and skillset of the Chrome team, and I’m super-excited about the improvements coming to Chrome in the near future. While progress is never as fast as I’d like, I’m proud to see the real-world web moving toward a more encrypted, secure, and powerful future.

May 29th will be my last day on the Chrome team and at Google. Starting in June, I’ll be heading back to Program Management, working together with a lot of old friends.

I Still ❤ The Web

I’ve been working on web security for a long time at this point, and spending most of my time looking at all of the bad stuff happening on the web can get pretty demoralizing. Fortunately, there’s also a lot of amazing stuff on the web that periodically reminds me of what an amazing tool it can be.

For instance, this afternoon a friend posted the following picture:

MysteryText

Now, I’ve loved mysteries for a long time, and this one seemed like it ought to be an easy one with all of the magical tech we have at our disposal these days.

I first gave it a shot with the Google Translate app on my phone, which frequently surprises me with the things it can do. However, while it can do translations via photo, that feature requires that you first specify the source language, and I don’t know it yet.

I gave this nice article on recognizing character sets a skim, but no perfect match leapt out at me, although it did eliminate some of my guesses. The answer’s actually in there, had I done more than skim it.

Fortunately, I remembered an amazing website that lets you draw a shape and it’ll tell you what Unicode characters you might be writing. I chose the most distinctive character I could and scrawled it in the box, and the first half of the puzzle fell into place:

Drawbox

After that, it was a simple matter of clicking through to the Georgian character set block and verifying that all of the characters were present. I then copied the text over to Google Translate, which reports that ელდარ translates to Eldar.

There’s a ton of amazing stuff out there on the web. Don’t let all the bad stuff get you down.

-Eric