browsers, security

SSLVersionMin Policy returns to Chrome 66

Chrome 66, releasing to stable this week, again supports the SSLVersionMin policy that enables administrators to control the minimum version of TLS that Chrome is willing to negotiate with a server.

If this policy is in effect and configured to permit, say, only TLS/1.2+ connections, attempting to connect to a site that only supports TLS/1.0 will result in an error page with the status code ERR_SSL_VERSION_OR_CIPHER_MISMATCH.

This policy existed until Chrome 52 and was brought back for Chrome 66 and later. It is therefore possible that your administrators configured it long ago, forgot about the setting, and will be impacted again with Chrome 66’s rollout.


browsers, security

HSTS Preload and Subdomains

In order to be eligible for the HSTS Preload list, your site must usually serve a Strict-Transport-Security header with a includeSubdomains directive.

Unfortunately, some sites do not follow the best practices recommended and instead just set a one-year preload header with includeSubdomains and then immediately request addition to the HSTS Preload list. The result is that any problems will likely be discovered too late to be rapidly fixed– removals from the preload list may take months.

The Mistakes

In running the HSTS preload list, we’ve seen two common mistakes for sites using includeSubdomains:

Mistake: Forgetting Intranet Hosts

Some sites are set up with a public site ( and an internal site only accessible inside the firewall ( When includeSubdomains is set, all sites underneath the specified domain must be accessible over HTTPS, including in this case Some corporations have different teams building internal and external applications, and must take care that the security directives applied to the registrable domain are compatible with all of the internal sites running beneath it in the DNS hierarchy. Following the best practices of staged rollout (with gradually escalating max-age directives) will help uncover problems before you brick your internal sites and applications.

Of course, you absolutely should be using HTTPS on all of your internal sites as well, but HTTPS deployments are typically smoother when they haven’t been forced by Priority-Zero downtime.

Mistake: Forgetting Delegated Hosts

Some popular sites and services use third party companies for advertising, analytics, and marketing purposes. To simplify deployments, they’ll delegate handling of a subdomain under their main domain to the vendor providing the service. For instance, will point to the company’s own servers, while the hostname will point to Experian or Marketo servers. A HSTS rule with includeSubdomains applied to will also apply to those delegated domains. If your service provider has not enabled HTTPS support on their servers, all requests to those domains will fail when upgraded to HTTPS. You may need to change service providers entirely in order to unbrick your marketing emails!

Of course, you absolutely should be using HTTPS on all of your third-party apps as well, but HTTPS deployments are typically smoother when they haven’t been forced by Priority-Zero downtime.


If you do find yourself in the unfortunate situation of having preloaded a TLD whose subdomains were not quite ready, you can apply for removal from the preload list, but, as noted previously, the removal can be expected to take a very long time to propagate. For cases where you only have a few domains out of compliance, you should be able to quickly move them to HTTPS. You might also consider putting a HTTPS front-end out in front of your server (e.g. Cloudflare’s free Flexible SSL option) to allow it to be accessed over HTTPS before the backend server is secured.

Deploy safely out there!


browsers, security, Uncategorized

Google Internet Authority G3

For some time now, operating behind the scenes and going mostly unnoticed, Google has been changing the infrastructure used to provide HTTPS certificates for its sites and services.

You’ll note that I said mostly. Over the last few months, I’ve periodically encountered complaints from users who try to load a Google site and get an unexpected error page:


Now, there are a variety of different problems that can cause errors like this one– in most cases, the problem is that the user has some software (security software or malware) installed locally that is generating fake certificates that are deemed invalid for various reasons.

However, when following troubleshooting steps, we’ve determined that a small number of users encountering this NET::ERR_CERT_AUTHORITY_INVALID error page are hitting it for the correct and valid Google certificates that chain through Google’s new intermediate Google Internet Authority G3. That’s weird.

What’s going on?

The first thing to understand is that Google operates a number of different certificate trust chains, and we have multiple trust chains deployed at the same time. So a given user will likely encounter some certificate chains that go through the older Google Internet Authority G2 chain and some that go through the newer Google Internet Authority G3 chain– this isn’t something the client controls.


You can visit this GIA G3-specific test page to see if the G3 root is properly trusted by your system.

More surprisingly, it’s also the case that you might be getting a G3 chain for a particular Google site (e.g. while some other user is getting a G2 chain for the same URL. You might even end up with a different chain simply based on what Google sites you’ve visited first, due to a feature called HTTP/2 connection coalescing.

In order to see the raw details of the certificate encountered on an error page, you can click the error code text on the blocking page. (If the site loaded without errors, you can view the certificate like so).

Google’s new certificate chain is certainly supposed to be trusted automatically– if your operating system (e.g. Windows 7) didn’t already have the proper certificates installed, it’s expected to automatically download the root certificate from the update servers (e.g. Microsoft WindowsUpdate) and install it so that the certificate chain is recognized as trusted. In rare instances, we’ve heard of this process not working– for instance, some network administrators have disabled root certificate updates for their enterprise’s PCs.

On modern versions of Windows, you can direct Windows to check its trusted certificate list against the WindowsUpdate servers by running the following from a command prompt:

certutil -f -verifyCTL AuthRootWU

Older versions of Windows might not support the -verifyCTL command. You might instead try downloading the R2 GlobalSign Root Certificate directly and then installing it in your Trusted Root Certification Authorities:


Overall, the number of users reporting problems here is very low, but I’m determined to help ensure that Chrome and Google sites work for everyone.


browsers, security

Chrome Field Trials

Back in April, we announced:

Beginning in October 2017, Chrome will show the “Not secure” warning in two additional situations: when users enter data on an HTTP page, and on all HTTP pages visited in Incognito mode.

This is true, but it’s perhaps a little misleading, based on some of the tweets we’ve seen:

Screen Shot 2017-10-18 at 8.05.25 AM

What isn’t mentioned in the blog post is exactly how this feature will roll out– many readers naturally assume that it’s as simple as: “If you have Chrome 62, then this feature is present.” After all, that’s how software usually works.

In Chrome, things are more interesting. Where possible, Chrome rolls out new features dynamically using the Field Trials platform. You can think of Field Trials as a set of server-controlled flags that allow Google to change Chrome’s behavior dynamically, at runtime, without shipping a new version.

We use Field Trials for two major purposes– for experimentation, and for feature rollouts.

Experimentally, we run many experiments where we create one or more experimental groups and then compare telemetry from those clients against a control group. If a feature isn’t performing as expected (e.g. its usage declines vs. the feature it replaces, or browser crashes increase, or memory usage increases or page load slows, etc), the feature is tuned or removed before it officially “ships.” Experiments are often conducted on the pre-release channels (e.g. Canary, Dev, and Beta) before deciding whether or not a feature should be rolled out to the Stable channel.

After a feature has proven itself via experiments, it’s ready to roll out to users in the Stable channel. Unfortunately, pre-release channels don’t get coverage nearly as broad as we’d like, so we have to take care when a feature is first rolled out to Stable. By using a field trial, we can enable the new feature for a huge number of Stable users while still keeping it at a low percentage of the Stable user base (e.g. 1% of a billion installs == 10 million clients). We keep an eagle eye on telemetry and user-feedback reports to spot any unexpected problems, and assuming we don’t find any, we can quickly ramp up the new feature rollout to 100% of users. If we discover the feature wasn’t fit to ship for whatever reason (e.g. introduces some serious bug), we can dial it back to 0% until a fix can be created.

Unfortunately, field trials are one of the very few inscrutable parts of the otherwise veryopen Chrome– Google does not publish information about the current percentage of users in a trial, and while chrome://version/ shows which trials are currently enabled for a given client, there’s no public mapping of the Variations tokens to the actual trials they control.

Rest assured that I’m eager to push the new Not Secure warnings to 100% and I expect to get to do so very soon. If you just can’t wait, you can override the field trial and turn it on yourself by changing chrome://flags/#mark-non-secure-as and restarting Chrome.

Screen Shot 2017-10-18 at 8.22.16 AM

Note that there’s not a 1:1 correspondence between Flags and Field Trials– while many trials can be overridden by flags, not all experiments have user-toggleable flags. If a feature is controlled by a flag and a field trial, when a flag hasn’t been manually configured, the field trial’s setting (if any) is not reflected in chrome://flags… the flag entry will just show Default.

Protecting your web traffic as fast as we can,


browsers, security, Uncategorized

Stealing your own password is not a vulnerability

By far, the most commonly-reported “vulnerability” reported to the Chrome Vulnerability Rewards program boils down to “I can steal my own password.” Despite having its very own FAQ entry, this gets reported to the VRP at varying levels of breathlessness, sometimes multiple times per day.

You can see this “attack” in action:


Yes, it’s true, you can use Chrome to steal your own password.

You can also grab a knife and stab yourself in the leg. I wonder how often knife-makers get letters to that effect?


PS: “But… but… what if I don’t want to use this easy trick and instead steal my own passwords from myself by installing some software on my own PC? Isn’t that a security bug?”

No, although “Software can read passwords if I give it full permission to run on my computer” is probably the third or fourth most common non-bug reported to Chromium’s Security@ alias. is one of many such filings.
Chrome does use encryption when storing passwords in most cases. However, Chrome has the decryption key (necessarily). You can read more about why an attacker who has physical access or has compromised your machine is outside of the browser’s threat model.
browsers, security

Chrome 59 on Mac and TeletexString Fields

Update: This change ended up getting backed out, after it was discovered that it impacted smartcard authentication. Thanks for self-hosting Chrome Dev builds, IT teams!

A change quietly went into Chrome 59 that may impact your certificates if they contain non-ASCII characters in a TeletexString field. Specifically, these certificates will fail to validate on Mac, resulting in either a ERR_SSL_SERVER_CERT_BAD_FORMAT error for server certificates or a ERR_BAD_SSL_CLIENT_AUTH_CERT error for client certificates. The change that rejects such certificates is presently only in the Mac version of Chrome, but it will eventually make its way to other platforms.

You can see whether your certificates are using teletexStrings using an ASN.1 decoder program, like this one. Simply upload the .CER file, and look for the TeletexString type in the output. If you find any such fields that contain non-ASCII characters, the certificate is impacted:

Non-ASCII character in string

Background: Certificates are encoded using a general-purpose data encoding scheme called ASN.1. ASN.1 specifies encoding rules, and strings may be encoded using any of a number of different data types (teletexString, printableString, universalString, utf8String, bmpString). Due to the complexity and underspecified nature of the TeletexString, as well as the old practice of shoving Latin1 strings in fields marked as TeletexString, the Chrome change takes a conservative approach to handling TeletexString, only allowing the ASCII subset. utf8String is a well-specified and well-supported standard and should be used in place of the obsolete teletexString type.

To correct the problem with the certificate, regenerate it using UTF8String fields to store non-ASCII data.

-Eric Lawrence


Inspecting Certificates in Chrome

With a check-in on Monday night, Chrome Canary build 60.0.3088 regained a quick path to view certificates from the top-level security UI. When the new feature is enabled, you can just click the lock icon to the left of the address box, then click the “Valid” link in the new Certificate section of the Page Information bubble to see the certificate:

Chrome 60 Page Info dropdown showing certificate section

In some cases, you might only be interested in learning which Certificate Authority issued the site’s certificate. If the connection security is Valid, simply hover over the link to see the issuer information in a tooltip:

Tooltip shows Issuer CA

The new link is also available on the blocking error page in the event of an HTTPS error, although no tooltip is shown:

The link also available at the blocking Certificate Error page

Note: For now, you must manually enable the new Certificate section. Type chrome://flags/#show-cert-link in Chrome’s address box and hit enter. Click the Enable link and relaunch Chrome.


This section is enabled by default in Chrome 63 along with some other work to simplify the Page Information bubble.

If you want more information about the HTTPS connection, or to see the certificates of the resources used in the page, hit F12 to open the Developer Tools and click to the Security tab:

Chrome DevTools Security tab shows more information

You can learn more about Chrome’s certificate UIs and philosophy in this post from Chrome Security’s Chris Palmer.

-Eric Lawrence