Get Help with HTTPS problems

Sometimes, when you try to load a HTTPS address in Chrome, instead of the expected page, you get a scary warning, like this one:

image

Chrome has found a problem with the security of the connection and has blocked loading the page to protect your information.

In a lot of cases, if you’re just surfing around, the easiest thing to do is just find a different page to visit. But what happens if this happens on an important site that you really need to see? You shouldn’t just “click through” the error, because this could put your device or information at risk.

In some cases, clicking the ADVANCED link might explain more about the problem. For instance, in this example, the error message says that the site is sending the wrong certificate; you might try finding a different link to the site using your favorite search engine.

image

Or, in this case, Chrome explains that the certificate has expired, and asks you to verify that your computer clock’s Date and Time are set correctly:

image

You can see the specific error code in the middle of the text:

image

Some types of errors are a bit more confusing. For instance, NET::ERR_CERT_AUTHORITY_INVALID means that the site’s certificate didn’t come from a company that your computer is configured to trust.

image

Errors Everywhere?

What happens if you start encountering errors like this on every HTTPS page that you visit, even major sites like https://google.com?

In such cases, this often means that you have some software on your device or network that is interfering with your secure connections. Sometimes this software is well-meaning (e.g. anti-virus software, ad-blockers, parental control filters), and sometimes it’s malicious (adware, malware, etc). But even buggy well-meaning software can break your secure connections.

If you know what software is intercepting your traffic (e.g. your antivirus) consider updating it or contacting the vendor.

Getting Help

If you don’t know what to do, you may be able to get help in the Chrome Help Forum. When you ask for help, please include the following information:

  • The error code (e.g. NET::ERR_CERT_AUTHORITY_INVALID).
    • To help the right people find your issue, consider adding this to the title of your posting.
  • What version of Chrome you’re using. Visit chrome://version in your browser to see the version number
  • The type of device and network (e.g. “I’m using a laptop on wifi on my school’s network.”)
  • The error diagnostic information.

You can get diagnostic information by clicking or tapping directly on the text of the error code: image. When you do so, a bunch of new text will appear in the page:

image

You should select all of the text:

image

…then hit CTRL+C (or Command ⌘+C on Mac) to copy the text to your clipboard. You can then paste the text into your post. The “PEM encoded chain” information will allow engineers to see exactly what certificate the server sent to your computer, which might shed light on what specifically is interfering with your secure connections.

With any luck, we’ll be able to help you figure out how to surf securely again in no time!

 

-Eric

Get Help with HTTPS problems

Chrome Deprecates Subject CN Matching

If you’re using a Self-Signed certificate for your HTTPS server, a deprecation coming to Chrome may affect your workflow.

Chrome 58 will require that certificates specify the hostname(s) to which they apply in the SubjectAltName field; values in the Subject field will be ignored. This follows a similar change in Firefox 48. If impacted, you’ll see something like this blocking page as you load your HTTPS site:

NET::ERR_CERT_COMMON_NAME_INVALID blocking page in Chrome

NET::ERR_CERT_COMMON_NAME_INVALID is an unfortunate error code, insofar as all common names are now ignored. Chrome is working to improve the debuggability of this experience, via:

Update: Both of these have landed. Chrome now shows [missing_subjectAltName] in the details on the Certificate Error page, and a Subject Alternative Name Missing warning in the Security panel of the Developer tools.

Notably, Windows’ ancient makecert.exe utility cannot set the SubjectAltName field in certificates, which means that if you’re using it to generate your self-signed certificates, you need to stop. Instead, users of modern Windows can use the New-SelfSignedCertificate command in PowerShell.

New-SelfSignedCertificate -DnsName "www.example.com", "example.com" -CertStoreLocation "cert:\CurrentUser\My"

Using openssl for self-signed certificate generation? See https://stackoverflow.com/a/27931596.

This new restriction may also impact users of very old versions of Fiddler (or FiddlerCore), or users who have configured Fiddler to use MakeCert for whatever reason. Fortunately, Fiddler offers a number of different certificate generators, so you just need to make a small configuration change. To switch away from MakeCert, click Tools > Fiddler Options > HTTPS and click the “Certificates generated by MakeCert engine” link. Change the dropdown to CertEnroll and click OK. Click Actions > Reset All Certificates and restart Fiddler.

image

If you’re building an application atop FiddlerCore, you’ll need to make sure you’re not using makecert; see the end of this post for help.

-Eric Lawrence

PS: There’s also a EnableCommonNameFallbackForLocalAnchors policy. You shouldn’t use it and you should just fix your certificates, or they’ll break when it’s removed in Chrome 65 or earlier.

Chrome Deprecates Subject CN Matching

The Trouble with Magic

“Magic” is great… except when it isn’t.

Software Design is largely about tradeoffs, and one of the more interesting tradeoffs is between user experience and predictability. This has come up repeatedly throughout my career and in two independent contexts yesterday that I’ll describe in this post.

Developer Magic

I’m working on a tiny UX change to Google Chrome to deemphasize the data component of data: URIs.

Chrome is a multi-platform browser that runs on Windows, Mac, Linux, ChromeOS, Android, and iOS, which means that I need to make the same change in a number of places. Four, to be precise: Views (our cross-platform UI that runs on Windows, Linux and ChromeOS), Cocoa (Mac), Bling (iOS) and Clank (Android). The change for Views was straightforward and I did it first; porting the change to Mac wasn’t too hard. With the Mac change in hand, I figured that the iOS change would be simple, as both are written in Objective C++. I don’t have a local Mac development box, so I have to upload my iOS changes to the Chromium build bots to see if they work. Unfortunately, my iOS build failed, with a complaint from the linker:

Undefined symbols for architecture arm7:

“gfx::Range::ToNSRange() const”, referenced from:

OmniboxViewIOS::SetEmphasis(bool, gfx::Range) in omnibox_view_ios.o

OmniboxViewIOS::UpdateSchemeEmphasis(gfx::Range) in omnibox_view_ios.o

ld: symbol(s) not found for architecture arm7

Hrm… that’s weird; the Mac build worked and the iOS build used the same APIs. Let’s go have a look at the definition of ToNSRange():

ToNSRange() inside an if(defined(OS_MACOSX)

Oh, weird. It’s in an OS_MACOSX block, so I guess it’s not there for iOS. But how did it compile and only fail at linking?

Turns out that the first bit of “magic” is that when OS_IOS is defined, OS_MACOSX is always also defined:

iOS gets both MacOS and iOS defined

I was relieved to learn that I’m not the only person who didn’t know this, both by asking around and by finding code blocks like this:

If defined(win)||defined(mac)||defined(ios)

Okay, so that’s why it compiled, but why didn’t it link? Let’s look at the build configuration file:

image

Hmmm… That’s a bit suspicious. There’s range_mac.mm and range_win.cc both listed within a single target. But it seems unlikely that the Mac build includes the Windows code, or that the Windows build includes the Mac code. Which suggests that maybe there’s some magic not shown in the build configuration that determines what actually gets built. And indeed, it turns out that such magic does exist.

The overall Build Configuration introduces its own incompatible magic, whereby filenames suffixed with _mac only compile on Mac… and that is limited to actual Mac, not including iOS:

sources_assignment_filters

This meant that the iOS compilation had a header file with no matching implementation, and I was the first lucky guy to stumble upon this by calling the missing code.

Magic handling of filenames is simultaneously great (“So convenient”) and awful—I spoke to a number of engineers who knew that the build does this, but had no idea how, or whether or not iOS builds would include _mac-suffixed files. My  instinct for fixing this would be to rename range_mac.mm to just range_apple.mm (because .mm files are compiled only for Mac and iOS), but instead I’ve been told that the right fix is to just temporarily disable the magic:

Add exclusion via set_sources_assignment_filter

Talking to some of the experts, I learned that the long term goal is to get rid of the sources_assignment_filters altogether (No more magic!) but doing so entails a bunch of boring work (No more magic!).

Magic is great, when it works.

When it doesn’t, I spend a lot of time investigating and writing blog posts. In this case, I ended up flailing about for a few hours (because sending my various fix attempts off to the bots isn’t fast) trying to figure out what was going on.

There’s plenty of other magic that happens throughout the Chromium developer toolchain; some of it visible and some of it invisible. For instance, consider what happens when I forget the name of the command that finds out what release a changelist went into:

git find-release

Git “magically” knows what I meant, and points out my mistake.

Elsewhere, however, Chromium’s git “magically” knows what I meant and just does it:

git cl upalod (typo)

Which approach is better? I suppose it depends. The code that suggests proper commands is irritating (“Dammit, if you knew what I meant, you could just do it!”) but it’s also predictable—only legal commands run and typos cannot go overlooked and propagate throughout scripts, documentation, etc.

This same type of tradeoff appeared in a different scenario by the end of the day.

End-User Magic

This repro won’t work forever, but try clicking this link: https://www.kubernetes.io. If you do this right now, you’ll find that the page works great in Chrome, but doesn’t work in IE or Edge:

Edge and IE show Certificate Error

If you pop the site into SSLLabs’ server test, you can see that the server indeed has a problem:

Certificate subject name mismatch

The certificate’s SubjectAltNames field contains kubernetes.io, but not http://www.kubernetes.io.

So, what gives? Why does the original www URL work in Chrome? If you open the Developer Tools console while following the link, you’ll see the following explanation of the magic:

Console warning about automagic redirection

Basically, Chrome saw that the certificate for http://www.kubernetes.io was misconfigured and recognized that sending the user to the bare domain kubernetes.io was probably the right thing to do. So, it just did that, which is great for the user. Right? Right??

Well, yes, it’s great for Chrome users, and maybe for HTTPS adoption– users don’t like certificate errors, and asking them to manually “fix” things the browser can fix itself is annoying.

But it’s less awesome for users of other browsers without this accommodation, especially when the site developers don’t know about Chrome’s magic behavior and close the bug as “fixed” because they tested in Chrome. So other browsers have to adopt this magic if they want to be as great as Chrome (no browser vendor likes bugs whining “Your browser doesn’t work but Chrome does!”). Then, after all the browsers have the magic in place, then other tools like curl and wfetch and wget etc need to adopt it. And the magic is now a hack that lives on for decades, increasing the development cost of all future web clients. Blargh.

It’s worth noting that this scenario was especially confusing for users of Microsoft Edge, because its address box has special magic that hides the “www.” prefix, even on error pages. The default address is a “lie”:

Edge hides the

Only by putting focus in the address bar can you see the “truth”:

WWW is showing now on focus

When you’re building magic into your software, consider carefully how you do so.

  • Do you make your magic invisible, or obvious?
  • Is there a way to scope it, or will you have to maintain it forever?
  • Are you training users to expect magic, or guiding them away from it?
  • If you’re part of an ecosystem, is your magic in line with your long-term ecosystem goals?

-Eric

The Trouble with Magic

Client Certificates on Android

Recently, this interesting tidbit crossed my Twitter feed:

Tweet: "Your site asks for a client certificate?"

Sure enough, if you visited the site in Chrome, you’d get a baffling prompt.

My hometown newspaper shows the same thing:

No Certificates Found prompt on Android

Weird, huh?

Client certificates are a way for a browser to supply a certificate to the server to verify the client’s identity (in the typical case, a HTTPS server only sends its certificate so that the client can validate that the server is what it says it is.

In the bad old days of IE6 and IE7, the default behavior was to show a similar prompt, but what’s going on with the latest Chrome on modern Android?

It turns out that this is a limitation of the Android security model, to which Chrome on Android is subject. In order for Chrome to interact with the system’s certificate store, the operating system itself shows a security prompt.

If your server has been configured to accept client certificates (in either require or want mode), you should be sure to test it on Android devices to verify that it behaves as you expect for your visitors (most of whom likely will not have any client certificates to supply).

-Eric

Client Certificates on Android

Extended Validation Certificates – The Introduction

In 2005, one of my first projects on the Internet Explorer team was improving the user-experience for HTTPS sites (“SSLUX”).

Our first task was to change the certificate error experience from the confusing and misleading modal dialog box:

Certificate errors UX

… to something that more clearly conveyed the risk and which more clearly discouraged users from accepting invalid certificates. We quickly settled upon using a blocking page for bad certificates, a pattern seen in all major browsers today.

Next, we wanted to elevate the security information from the lowly lock buried at the bottom of the window (at best, since the status bar could be hidden entirely):

IE6 status bar

As a UI element, the lock resonated with users, but it wasn’t well understood (“I look for the lock and if it’s there, I’m safe”). We felt it was key to ensure that users not only saw that the connection was secure, but also with whom a secure connection had been made. This was especially important as some enterprising phishers had started obtaining HTTPS certificates for their spoofing sites, with domain names like BankOfTheVVest.com. Less urgently, we also wanted to help users understand that a secure connection didn’t necessarily mean the site is safethe common refrain was that we’d happily set up a secure connection to a site run by the Russian Mafia, so long as the user recognized who they were talking to.

We decided to promote the HTTPS certificate information to a new UI element next to the address bar1. Called the “Trust Badge”, the button would prominently display the information about the owner and issuer of the HTTPS certificate, and clicking it would allow users to examine the certificate in full:

EV Certificate UI in IE7

Displaying the Issuer of the certificate was deemed especially important– we knew some CAs were doing a much better job than others. High-volume-Low-vetting CAs’ $20 certificates were, to users, indistinguishable from the certificates from CAs who did a much more thorough job of vetting their applicants (usually at a much higher price point). The hope was that the UI would both shame lazy CAs and also provide a powerful branding incentive for those doing a good job.

We were pretty excited to show off our new work in IE7 Beta 1, but five months before our beta shipped, Opera beat us to it with Opera 8 beta 2 with a UI that was nearly identical to what we were building.

During those five months, however, we spoke to some of the Certificate Authorities in the Microsoft Root CA program and mentioned that we’d be making some changes to IE’s certificate UI. They expressed excitement to hear that their names would no longer be buried in the depths of a secondary dialog, but cautioned: “Just so long as you don’t do what Opera did.

Why’s that?” we asked innocently.

Well, they show the Subject organization and location information in their UI.”

“And that’s a problem because…” we prompted.

Well, we don’t validate any of the information in the certificate beyond the domain name.” came the reply.

But you omit any fields you don’t validate, right?” we asked with growing desperation.

Nah, we just copy ‘em over.

After the SSLUX feature team picked our collective jaws off the floor, we asked around and determined that, yes, the ecosystem “race to the bottom” had been well underway over the preceding few years, and so-called “Domain validation” (DV) of certificates was extremely prevalent. While not all DV certificates contained inaccurate information, there was no consistent behavior across CAs.

Those CAs who were doing a good job of vetting certificates were eager to work with browsers to help users recognize their products, and even the “cheap” CAs felt that their vetting was better than that of their competitors2. Soon the group that evolved into the CA/Browser forum was born, pulling in stakeholders of all types (technologists, policy wonks, lawyers) from all over the industry (Microsoft, Mozilla, Konquerer, etc). Meetings were had. And more meetings. And calls. And much sniping and snarking. And more meetings. Eventually, the version 1.0 guidelines for a new sort of certificate were adopted. These Extended Validation (nee “Enhanced Validation”, nee “High Assurance”) certificates required specific validation steps that every CA would be required to undertake.

EV certificates were far from perfect, but we thought they were a great first step toward fixing the worst problems in the ecosystem.

Browsers would clearly identify when a connection was secured with EV (IE flood-filled the address bar with green) to improve user confidence and provide sites with a business reason to invest (time and money) in a certificate with more vetting. For the EV UI treatment, browsers could demand sites and CAs use stronger algorithms and support features like revocation checking. Importantly, this new class of certificates finally gave browsers a stick to wield against popular CAs who did a poor job—in the past, threats to remove a CA from the trust store rang hollow, because the CA knew that users would blame the browser vendor more than the CA (“Why do I care if bad.com got a certificate, good.com should work fine!”); with EV, browsers could strip the EV UX from a CA (leading their paying customers to demand refunds) without issuing an “Internet Death Sentence” for the entire CA itself.

Our feature was looking great. Then the knives really came out.

…to be continued…

-Eric

1 Other SSLUX investments, like improving the handling of Mixed Content, were not undertaken until later releases.

2 Multiple CAs who individually came to visit Redmond for meetings brought along fraudulent certificates they’d tricked their competitors to issue in our names, perhaps not realizing how shady this made them look.

Extended Validation Certificates – The Introduction

Reset Fiddler’s HTTPS certificates

I’ve made changes to the latest versions of Fiddler to improve the performance of certificate creation, and to avoid problems with new certificate validation logic coming to Chrome and Firefox. The biggest of the Fiddler changes is that CertEnroll is now the default certificate generator on Windows 7 and later.

Unfortunately, this change can cause problems for users who have previously trusted the Fiddler root certificate; the browser may show an error message like NET::ERR_CERT_AUTHORITY_INVALID or The certificate was not issued by a trusted certificate authority.

Please perform the following steps to recreate the Fiddler root certificate:

Fiddler 4.6.1.5+

  1. Click Tools > Fiddler Options.
  2. Click the HTTPS tab.
  3. Ensure that the text says Certificates generated by CertEnroll engine.
  4. Click Actions > Reset Certificates. This may take a minute.
  5. Accept all prompts

Fiddler 4.6.1.4 and earlier

  1. Click Tools > Fiddler Options.
  2. Click the HTTPS tab
  3. Uncheck the Decrypt HTTPS traffic checkbox
  4. Click the Remove Interception Certificates button. This may take a minute.
  5. Accept all of the prompts that appear (e.g. Do you want to delete these certificates, etc)
  6. (Optional) Click the Fiddler.DefaultCertificateProvider link and verify that the dropdown is set to CertEnroll
  7. Exit and restart Fiddler
  8. Click Tools > Fiddler Options.
  9. Click the HTTPS tab
  10. Re-check the Decrypt HTTPS traffic checkbox
  11. Accept all of the prompts that appear (e.g. Do you want to trust this root certificate)

image

If you are using Fiddler to capture secure traffic from a mobile device or Firefox, you will need to remove the old Fiddler root certificate from that device (or Firefox) and install the newly-generated Fiddler certificate.

I apologize for the inconvenience, but I believe that the new certificate generator will help ensure smooth debugging with current and future clients.

-Eric Lawrence

Reset Fiddler’s HTTPS certificates

Fiddler Certificate Generators

Fiddler and FiddlerCore offer three different choices for generating interception certificates:

  • MakeCert
  • CertEnroll
  • Bouncy Castle

If you’re so inclined, you can even write your own certificate generator (say, by wrapping OpenSSL) and expose it to Fiddler using the ICertificateProvider3 interface.

On Windows, Fiddler includes the MakeCert and CertEnroll certificate generators by default; you can download the Bouncy Castle Certificate Generator if you like. In contrast, when Fiddler is running on Linux and Mac, only the Bouncy Castle certificate generator is available, and it is included by default.

If you’re using Windows, however, you may wonder which Certificate Generator you should use in Fiddler or for your applications based on FiddlerCore.

In general, I recommend the Bouncy Castle generator, as it has better performance than the default MakeCert generator and it offers more configuration choices than the CertEnroll generator. Another advantage of the Bouncy Castle certificate generator is that the only certificate that (typically) goes in the Windows Certificate store is the root certificate. The server (end-entity) certificates generated for each website are kept in memory and discarded when Fiddler exits; because the Bouncy Castle generator reuses a single private key for all certificates by default, the performance impact of this behavior is minimal.

The only downside to the Bouncy Castle generator is its size: it is ~200KB when compressed, which is 25% larger than FiddlerCore itself.

The CertEnroll generator was added to Fiddler relatively recently; it offers better performance and standards-compliance than the legacy MakeCert generator but it is available only on Windows 7 and later. You can easily switch Fiddler to use CertEnroll inside Tools > Fiddler Options > HTTPS.

The MakeCert generator is the original certificate generator used by Fiddler and it remains the default on Windows today (mostly) for legacy compatibility reasons. It suffers from a number of shortcomings, including the fact that the certificates it generates are not compatible with iOS and (some) Android devices. It generates certificates with a 1024 bit RSA key (which may soon trigger warnings in some browsers) and each certificate has a unique key (meaning that each new secure site you visit triggers the somewhat costly key generation code).

Both the CertEnroll and MakeCert-based certificate generators must store all server certificates in the Windows Certificate store which some users may find confusing:

image

The storage of (potentially thousands of) server certificates in the user profile can also cause some problems for corporate users who have roaming user profiles, as these certificates are roamed to each workstation as the user logs in. To mitigate that, the Clear server certs on exit checkbox can be set inside the Tools > Fiddler Options > HTTPS > Certificate Provider dialog, or via:

    FiddlerApplication.Prefs.SetBoolPref("fiddler.certmaker.CleanupServerCertsOnExit", true);

… however, the downside of doing that is that Fiddler must then re-create the server certificates every time it starts. This performance penalty is smaller when using CertEnroll, which reuses a single 2048-bit RSA key, than for MakeCert, which generates unique 1024-bit RSA keys for each site.

FiddlerCore Considerations

To determine which Certificate Generator is in use, be sure to attach the following event handlers:

Fiddler.FiddlerApplication.OnNotification +=
  delegate(object sender, NotificationEventArgs oNEA) { Console.WriteLine(“** NotifyUser: ” + oNEA.NotifyString); };
Fiddler.FiddlerApplication.Log.OnLogString +=
  delegate(object sender, LogEventArgs oLEA) { Console.WriteLine(“** LogString: ” + oLEA.LogString); };

You can then view information about the Certificate Generator in the console when it loads.

Developers building applications atop FiddlerCore should keep the following in mind when deciding which Certificate Generator to use:

MakeCert

  • MakeCert.exe is a Microsoft Visual Studio 2008 redistributable file, meaning that you’re licensed to redistribute it if you have an appropriate license to that version of Visual Studio. Microsoft may offer MakeCert.exe as a redistributable in other circumstances, but licensing is provided by Microsoft, not Telerik.
  • To use MakeCert.exe, you must include it adjacent to your application’s .exe file.
  • MakeCert-generated certificates are not compatible with iOS and some Android devices.
  • MakeCert-generated certificates “pollute” the user’s Certificate Store and you should consider offering a mechanism to clear them.

CertEnroll

  • The CertEnroll API is available on Windows 7 and later.
  • Use CertEnroll by either omitting makecert.exe from the application’s folder or by explicitly setting the preference:
  •     FiddlerApplication.Prefs.SetBoolPref("fiddler.certmaker.PreferCertEnroll", true);

  • CertEnroll-generated certificates “pollute” the user’s Certificate Store and you should consider offering a mechanism to clear them.

Bouncy Castle

  • Bouncy Castle is an open-source PKI and crypto library distributed under the MIT license.

  • To use Bouncy Castle, you must include CertMaker.dll and BCMakeCert.dll adjacent to your application’s .exe file.
  • Bouncy Castle does not store certificates in the Windows Certificate Store (yay!) but this also means that your application needs to keep track of its root certificate and private key (unless you recreate and retrust it every time the application runs).

    Two preferences are used to hold the key and certificate, fiddler.certmaker.bc.key and fiddler.certmaker.bc.cert. After you first call createRootCert, you should retrieve these preferences using FiddlerApplication.Prefs.GetStringPref and store them somewhere within your application’s settings (registry, XML, etc); the private key should be considered sensitive data and protected as such.  When your application next runs, it should detect whether the key and certificate have already been created, and if so, they should be provided to the certificate generator using FiddlerApplication.Prefs.SetStringPref before any certificates are requested, lest you inadvertently create a new root certificate.

    Rick Strahl wrote a great blog post on this process, including some sample code.

 

-Eric

Fiddler Certificate Generators