authentication

While most HTTPS sites only authenticate the server (using a certificate sent by the website), HTTPS also supports a mutual authentication mode, whereby the client supplies a certificate that authenticates the visiting user’s identity. Such a certificate might be stored on a SmartCard, or used as a part of an OS identity feature like Windows Hello.

To request mutual authentication, servers send a CertificateRequest message to the client during the HTTPS handshake, specifying a criteria filter that the browser will use to find a client certificate to satisfy the server’s request.

If a client certificate is supplied in the browser’s Certificate response to the server’s challenge, the browser proves the user’s possession of that certificate using the private key that matches that client certificate’s public key.

A client may choose not to send a certificate (either because no matching certificate is available, or because the user declined to supply a certificate that it had)—in such cases, the server may terminate the handshake (showing a Client Certificate Required error message) or it may continue the handshake and attempt to authenticate the user via other means.

Certificate Selection

The CertificateRequest message allows the server to specify criteria for the certificates it is willing to accept from the client, including details such as the certificate’s issuer, and key/signature/hash types.

The browser consults the Operating System’s trust store (Keychain on Mac OS X, certmgr.msc on Windows) to find any candidate certificates (unexpired certificates with the Client Authentication purpose set and a private key available) that match the server-supplied filtering criteria:

The private key for a given certificate might be stored on a SmartCard — when a SmartCard is inserted, the certificate(s) on it are “virtually” propagated to the OS trust store for use by browsers and other applications.

Certificates that meet the server’s filtering criteria are shown in a prompt:

If the user hits “Cancel”, the handshake is completed without sending a certificate. However, if the user selects a certificate, the browser caches that decision for the lifetime of the browser instance. The selected certificate will be resent on all new connections to the target origin and the prompt will not be shown again.

Today, there’s no good way to clear the selection decision, short of restarting the browser entirely. In contrast, legacy IE offered two very awkward mechanisms, the Clear SSLState button in the Internet Control Panel, and the ClearAuthenticationCache web API.

Automatic Selection of Client Certificate

Internet Explorer and Edge Legacy offered a behavior (Don’t prompt for client certificate selection when only one certificate exists, URLACTION_CLIENT_CERT_PROMPT), on-by-default for the Local Intranet Zone:

…whereby the browser would not prompt the user to select a certificate if the user only has one certificate that matches the server’s request. In such cases, the client would automatically send the matching certificate without showing a prompt.

For other zones, IE and Edge Legacy do prompt the user to select a certificate before any certificate is sent. This is a privacy measure, because if the browser silently sends the user’s identity to any website that asks for it, this is a “super-cookie” that would allow tracking that user across sites. Also, the client’s certificate might directly contain personally identifiable information about the user (e.g. their email address, office phone number, home address, etc).

Chromium (and thus Chrome, Edge, Brave, Opera, Vivaldi) largely does not use the concept of Zones, so instead the AutoSelectCertificateForUrls policy exists. This policy allows an IT administrator to configure clients to automatically send certificates to specified websites that request them, which can be used to satisfy the need to have, say, the user’s Windows Hello certificate sent to *.login.microsoft.com sites.

Here are two examples: the first selects the first certificate issued by “Windows Hello PIN – MSIT1” and the second rule selects the certificate with a SubjectCN=”RSACSP”.

If you’re trying to set a rule whereby multiple client certificates are valid candidates and the client should just return the first found match, just add another rule with the same pattern and a different filter.

For instance, this set will use SubjectCN=”RSACP” if a matching certificate found, or a certificate with IssuerCN=”Windows Hello PIN – MSIT1” if not:

A screenshot of a cell phone

Description automatically generated

However, as you may have noticed, the AutoSelectCertificateForUrls policy has one significant limitation, which is that it always sends the user’s first matching certificate to the selected site. Some users might have more than one certificate that matches the policy (for instance, some enterprises have both “test” and “production” certificates.

To address this shortcoming, the Edge team introduced a new policy in Edge 81. The new ForceCertificatePromptsOnMultipleMatches policy which does as it says: If the client has multiple certificates that could be used to satisfy the {OriginFilter->CertificateFilter} policy specified by a AutoSelectCertificateForUrls policy, instead of simply sending the first matching certificate, the browser will instead show a certificate selection prompt filtered to the certificates that match the policy.

If you find that Microsoft Edge shows a client certificate selection prompt in one scenario where other browsers do not, one possibility is that the site in question is not actually requesting a client certificate from those other browsers for some reason. For instance, some web authentication flows, including Microsoft’s AAD login, take the browser’s User-Agent into account when deciding what authentication mechanisms to use with the client.

In order to understand exactly what’s going on with Client Authentication, collect Network Traffic logs. SSL_HANDSHAKE_MESSAGE_RECEIVED messages of type 13 represent the client certificate request.

-Eric

Bonus trivia

  1. Notably: Certificate selection policies apply across browser profiles, meaning that they are in force even when the user is in an Incognito browser session.
  2. PS: Client Certificate prompting behavior on Android is weird.

The general notion of “how Client Certificates were supposed to work” was that each user would have one certificate for each organization to which they belong, issued by that organization’s root certificate. When visiting that organization’s servers, the server would send in the CertificateRequest message the identifier(s) of the root certificate(s) to which acceptable client certificates chain (using the certificate_authorities structure). The visiting client would then filter the certificates available for selection to only those that chain to that root (hopefully one certificate).

So, say I have two certificates, e.g. USA-NationalID and Microsoft-EmployeeID. When I visit https://portal.microsoft.com, Microsoft sends a CertificateRequest with a MicrosoftRootCA in the certificate_authorities field. My browser automatically filters my client certificates list to just the Microsoft-EmployeeID certificate and then sends that. In contrast, when I visit https://irs.gov, the government sends a CertificateRequest with a USGovernmentRootCA in the certificate_authorities field. My browser automatically filters my client certificates list to just the USA-NationalID certificate and sends that.

In practice, unfortunately, things haven’t worked out that way. Most organizations have not had the infrastructure or discipline to configure things to work like that, and as a consequence you end up with varying client behavior.

Firefox doesn’t seem to filter the certificate list, but it does offer a “Remember this decision” checkbox which presumably reduces user annoyance:

Firefox does not respect the Windows Trust Store, so each client certificate must be manually loaded into Firefox’s configuration. This is a hassle, but it tends to result in a somewhat “cleaner” experience where the user isn’t distracted by random certificates that might be cluttering Windows’ cert store.

In some cases, organizations are generating invalid client certificates but expecting them to work, leading us to create compat accommodations like the FEATURE_CLIENTAUTHCERTFILTER Feature Control Key.

In the browser, SmartCards can be used for two ways: HTTPS Client Certificate Authentication, and Windows Integrated Authentication.

  • Straight TLS mutual authentication, as described above.
  • Windows Integrated Authentication that occurs when visiting a website that sends a WWW-Authenticate: Negotiate header. The client may automatically send the user’s login credentials (Intranet Zone). Or, if those creds do not work or the Zone is not configured for automatic credential release (Non-intranet), the user will be prompted for credentials to use. In Edge 79, the user would get a prompt with two blank fields (“Username” and “Password”). In Edge 80 or later, upon noticing that the user has configured Windows Hello, the user will be shown the Windows Hello auth dialog that allows the user to use their face, type a PIN, use a SmartCard, etc. So, now Edge 80 matches Edge Legacy (v18 and lower).

Low Level Details 1
Low Level Details 2

Nice discussion (with pictures) of setting up client cert auth on IIS.

In Windows 10 Apps, the AppContainer must have the sharedUserCertificates capability to use certificates from the trust store.

This is the first message the client can send after receiving a ServerHelloDone message. This message is only sent if the server requests a certificate. If no suitable certificate is available, the client MUST send a certificate message containing no certificates. That is, the certificate_list structure has a length of zero. If the client does not send any certificates, the server MAY at its discretion either continue the handshake without client authentication, or respond with a fatal handshake_failure alert. Also, if some aspect of the certificate chain was unacceptable (e.g., it was not signed by a known, trusted CA), the server MAY at its discretion either continue the handshake (considering the client unauthenticated) or send a fatal alert.

CertificateVerify signs using the client certificate’s private key.

CertOpenStore “my” store

https://docs.microsoft.com/en-us/windows/desktop/api/wincrypt/nf-wincrypt-certopenstore

ClientAuthIssuer trust store.

Hard Problems: Fetch in Serviceworker scenario — how can the user select a certificate when no UI is allowed?

All major browsers have a built-in password manager. So we should use them, right?

I Do

  • I use my browser’s password manager because it’s convenient: with sync, I get all of my passwords on all of my devices.
  • This convenience means that I can use a different password for every website, improving my security.
  • This convenience means that my passwords can be long and hard to type, because I never have to do so.
  • This means that I don’t even know my own passwords for many sites, and because I can rely on my password manager to only fill my passwords on the sites to which they belong, I cannot succumb to a phishing attack.

Should You?

The easy answer is “Yes, use your browser’s password manager!

The more nuanced answer begins: “Tell me about your threat model?

As when evaluating almost any security feature, my threat model might not match your threat model, and as a consequence, our security choices might be different.

Here are the most relevant questions to consider when thinking about whether you should use a password manager:

  • Is a password manager available for your platform(s)?
  • What sort of attackers are you worried about?
  • What sort of websites do you log into?
  • Do you select strong, unique passwords?
  • Are your accounts protected with 2FA?
  • What sort of attacks are most likely?
  • What sort of attacks are possible?
  • How do you protect your devices?
  • What’s your personal tolerance for inconvenience?
  • Are you confident in the security of your password manager’s vendor?
  • If you sync passwords, are you confident in the security of the design of the sync system?

The answers to these questions might change your decisions about whether to use a password manager, and if so, whether you want to use the built-in password manager or use a password manager provided by a third-party.

For instance, if you’re sharing a Windows/Mac OS login account with someone you don’t trust, you should stop. If you cannot or don’t want to, you should not use a password manager, because there are trivial ways for a local user steal your passwords one-at-a-time and simple ways to steal them all at once. Of course, even if you’re not using a password manager, a co-user can simply use a keylogger to steal your passwords one-by-one as you type them.

Lock (Win+L) your computer when you’re not using it.

While browser passwords are encrypted on disk, they’re encrypted using a key available to any process on your PC, including any locally-running malware. Even if passwords are encrypted in a “vault” by a master key, they’ll be decrypted when loaded in the browser’s memory space and can be harvested after you unlock the vault. Locally-running malware is particularly dire if your threat model includes the possibility of a worm running rampant within your enterprise– it could infect all of your employees’ machines and steal all of their passwords in bulk in seconds. (Yes, dear reader, I know that you’re thinking of clever mechanisms to mitigate these sorts of attacks. I assure you I can defeat every practical idea you have. It’s a fundamental law of computing.)

Concern about instantaneous bulk egress of credentials has led the authors of security configuration guidance to recommend disabling browser password managers. For instance, the Edge Security Baseline and the Chrome STIG both suggest preventing users from using the password manager. (I personally think this is a poor tradeoff that increases the higher risk of individual users getting phished, but I don’t write the configuration guidance.)

Some tech elites advocate for using a 3rd-party password manager, and some users really like them. Most 3rd-party password managers are designed with broader feature sets to satisfy alternative threat models (including using master passwords to help protect against limited local attackers). Many also include additional conveniences like automatic generation of strong passwords and roaming of passwords to mobile platforms and apps. On the other hand, many external password manager applications are themselves a source of security vulnerabilities, and these products often end up growing extremely complicated due to the “Checkbox Wars” endemic to the security products industry.

Parting Advice

Passwords are a poor security mechanism, and should be phased out wherever possible.

When that’s not yet possible (because you don’t control the website): choose strong passwords, use a password manager if it satisfies your threat model, and enable 2FA if available (especially on your email accounts to which password recovery emails are sent).

-Eric

Many websites offer a “Log in” capability where they don’t manage the user’s account; instead, they offer visitors the ability to “Login with <identity provider>.”

When the user clicks the Login button on the original relying party (RP) website, they are navigated to a login page at the identity provider (IP) (e.g. login.microsoft.com) and then redirected back to the RP. That original site then gets some amount of the user’s identity info (e.g. their Name & a unique identifier) but it never sees the user’s password.

Such Federated Identity schemes have benefits for both the user and the RP site– the user doesn’t need to set up yet another password and the site doesn’t have to worry about the complexity of safely storing the user’s password, managing forgotten passwords, etc.

In some cases, the federated identity login process (typically implemented as a JavaScript library) relies on navigating the user to a top-level page to log in, then back to the relying party website into which the library injects an IFRAME1 back to the identity provider’s website.

FederatedID

The authentication library in the RP top-level page communicates with the IP subframe (using postMessage or the like) to get the logged-in user’s identity information, API tokens, etc.

In theory, everything works great. The IP subframe in the RP page knows who the user is (by looking at its own cookies or HTML5 localStorage or indexedDB data) and can release to the RP caller whatever identity information is appropriate.

Crucially, however, notice that this login flow is entirely dependent upon the assumption that the IP subframe is accessing the same set of cookies, HTML5 storage, and/or indexedDB data as the top-level IP page. If the IP subframe doesn’t have access to the same storage, then it won’t recognize the user as logged in.

Unfortunately, this assumption has been problematic for many years, and it’s becoming even more dangerous over time as browsers ramp up their security and privacy features.

The root of the problem is that the IP subframe is considered a third-party resource, because it comes from a different domain (identity.example) than the page (news.example) into which it is embedded.

For privacy and security reasons, browsers might treat third-party resources differently than first-party resources. Examples include:

  1. The Block 3rd Party cookies option in most browsers
  2. The SameSite Cookie attribute
  3. P3P cookie blocking in Internet Explorer2
  4. Zone Partitioning in Internet Explorer and Edge Spartan3
  5. Safari’s Intelligent Tracking Protection
  6. Firefox Content Blocking
  7. Microsoft Edge Tracking Prevention

When a browser restricts access to storage for a 3rd party context, our theoretically simple login process falls apart. The IP subframe on the relying party doesn’t see the user’s login information because it is loaded in a 3rd party context. The authentication library is likely to conclude that the user is not logged in, and redirect them back to the login page. A frustrating and baffling infinite loop may result as the user is bounced between the RP and IP.

The worst part of all of this is that a site’s login process might usually work, but fail depending on the user’s browser choice, browser configuration, browser patch level, security zone assignments, or security/privacy extensions. As a result, a site owner might not even notice that some fraction of their users are unable to log in.

So, what’s a web developer to do?

The first task is awareness: Understand how your federated login library works — is it using cookies? Does it use subframes? Is the IP site likely to be considered a “Tracker” by popular privacy lists?

The second task is to build designs that are more resilient to 3rd-party storage restrictions:

  • Be sure to convey the expected state from the Identity Provider’s login page back to the Relying Party. E.g. if your site automatically redirects from news.example to identity.example/login back to news.example/?loggedin=1, the RP page should take note of that URL parameter. If the authentication library still reports “Not signed in”, avoid an infinite loop and do not redirect back to the Identity Provider automatically.
  • Authentication libraries should consider conveying identity information back to the RP directly, which will then save that information in a first party context.For instance, the IP could send the identity data to the RP via a HTTP POST, and the RP could then store that data using its own first party cookies.
  • For browsers that support it, the Storage Access API may be used to allow access to storage that would otherwise be unavailable in a 3rd-party context. Note that this API might require action on the part of the user (e.g. a frame click and a permission prompt).

The final task is verification: Ensure that you’re testing your site in modern browsers, with and without the privacy settings ratcheted up.

-Eric

[1] The call back to the IP might not use an IFRAME; it could also use a SCRIPT tag to retrieve JSONP, or issue a fetch/XHR call, etc. The basic principles are the same.
[2] P3P was removed from IE11 on Windows 10.
[3] In Windows 10 RS2, Edge 15 “Spartan” started sharing cookies across Security Zones, but HTML5 Storage and indexedDB remain partitioned.

You should enable “2-Step Verification” for logins to your Google account.

Google Authenticator is an app that runs on your iOS or Android phone and gives out 6 digit codes that must be entered when you log in on a device. This can’t really prevent phishing (because a phishing page will just ask you for a code from it and if you’re fooled, you’ll give it up) but it does prevent attacks if a bad guy has only your password. Authenticator is free and simple to use, and is supported by many sites, including GitHub. Microsoft offers a nearly identical Authenticator app too. How ToTP works.

YubiKeys (and similar) are small USB keys that you can configure your accounts to require. They are cheapish (~$18) and cannot be phished (even if you tap your key while on a phishing site, the attacker cannot use it due to how the crypto works). These are the best protection for your accounts (Googlers all use them) and are highly recommended for Chrome extension developers, journalists, activists, etc, etc.

Consider, for instance, this phishing email that this Chrome Extension developer received:

cwsphish

If the developer’s account were protected by a Yubikey, his credentials would be useless to a phisher because they would not have the required second factor necessary to log in and plant their malicious code in the developer’s extensions.

If the developer’s account were protected by a TOTP/Google Authenticator, it would require that the attacker collect the token value and be actively watching for victims such that the ephemeral token did not expire before they had a chance to replay it to the legitimate Google servers.

Note: This blog post was written before the new Chromium-based Microsoft Edge was announced. As a consequence, it mostly discusses the behavior of the Legacy Microsoft Edge browser. The new Chromium-based Edge behaves largely the same way as Google Chrome.

InPrivate Mode was introduced in Internet Explorer 8 with the goal of helping users improve their privacy against both local and remote threats. Safari introduced a privacy mode in 2005.

All leading browsers offer a “Private Mode” and they all behave in the same general ways.

HTTP Caching

While in Private mode, browsers typically ignore any previously cached resources and cookies. Similarly, the Private mode browser does not preserve any cached resources beyond the end of the browser session. These features help prevent a revisited website from trivially identifying a returning user (e.g. if the user’s identity were cached in a cookie or JSON file on the client) and help prevent “traces” that might be seen by a later user of the device.

In Firefox’s and Chrome’s Private modes, a memory-backed cache container is used for the HTTP cache, and its memory is simply freed when the browser session ends. Unfortunately, WinINET never implemented a memory cache, so in Internet Explorer InPrivate sessions, data is cached in a special WinINET cache partition on disk which is “cleaned up” when the InPrivate session ends.

Because this cleanup process may be unreliable, in 2017, Edge Legacy made a change to simply disable the cache while running InPrivate, a design decision with significant impact on the browser’s network utilization and performance. For instance, consider the scenario of loading an image gallery that shows one large picture per page and clicking “Next” ten times:

InPrivateVsRegular

Because the gallery reuses some CSS, JavaScript, and images across pages, disabling the HTTP cache means that these resources must be re-downloaded on every navigation, resulting in 50 additional requests and a 118% increase in bytes downloaded for those eleven pages. Sites that reuse even more resources across pages will be more significantly impacted.

Another interesting quirk of Edge Legacy’s InPrivate implementation is that the browser will not download FavIcons while InPrivate. Surprisingly (and likely accidentally), the suppression of FavIcon downloads also occurs in any non-InPrivate windows so long as any InPrivate window is open on the system.

Web Platform Storage

Akin to the HTTP caching and cookie behaviors, browsers running in Private mode must restrict access to HTTP storage (e.g. HTML5 localStorage, ServiceWorker/CacheAPI, IndexedDB) to help prevent association/identification of the user and to avoid leaving traces behind locally. In some browsers and scenarios, storage mechanisms are simply set to an “ephemeral partition” while in others the DOM APIs providing access to storage are simply configured to return “Access Denied” errors.

You can explore the behavior of various storage mechanisms by loading this test page in Private mode and comparing to the behavior in non-Private mode.

Within IE and Edge Legacy’s InPrivate mode, localStorage uses an in-memory store that behaves exactly like the sessionStorage feature. This means that InPrivate’s storage is (incorrectly) not shared between tabs, even tabs in the same browser instance.

Network Features

Beyond the typical Web Storage scenarios, browser’s Private Modes should also undertake efforts to prevent association of users’ Private instance traffic with non-Private instance traffic. Impacted features here include anything that has a component that behaves “like a cookie” including TLS Session Tickets, TLS Resumption, HSTS directives, TCP Fast Open, Token Binding, ChannelID, and the like.

Automatic Authentication

In Private mode, a browser’s AutoComplete features should be set to manual-fill mode to prevent a “NameTag” vulnerability, whereby a site can simply read an auto-filled username field to identify a returning user.

On Windows, most browsers support silent and automatic authentication using the current user’s Windows login credentials and either the NTLM and Kerberos schemes. Typically, browsers are only willing to automatically authenticate to sites on “the Intranet“. Some browsers behave differently when in Private mode, preventing silent authentication and forcing the user to manually enter or confirm an authentication request.

In Firefox Private Mode and Edge Legacy’s InPrivate, the browser will not automatically respond to a HTTP/401 challenge for Negotiate/NTLM credentials.

In Chrome Incognito, Brave Incognito, and IE InPrivate, the browser will automatically respond to a HTTP/401 challenge for Negotiate/NTLM credentials even in Private mode.

Notes:

  • In Edge Legacy and the new Chromium-based Edge, the security manager returns MustPrompt when queried for URLACTION_CREDENTIALS_USE.
  • Unfortunately Edge Legacy’s Kiosk mode runs InPrivate, meaning you cannot easily use Kiosk mode to implement a display that projects a dashboard or other authenticated data on your Intranet.
  • For Firefox to support automatic authentication at all, the
    network.negotiate-auth.allow-non-fqdn and/or network.automatic-ntlm-auth.allow-non-fqdn preferences must be adjusted.

Detection of Privacy Modes

While browsers generally do not try to advertise to websites that they are running inside Private modes, it is relatively easy for a website to feature-detect this mode and behave differently. For instance, some websites like the Boston Globe block visitors in Private Mode (forcing login) because they want to avoid circumvention of their “Non-logged-in users may only view three free articles per month” paywall logic.

Sites can detect privacy modes by looking for the behavioral changes that signal that a given browser is running in Private mode; for instance, indexedDB is disabled in Edge Legacy while InPrivate. Detectors have been built for each browser and wrapped in simple JavaScript libraries. Defeating Private mode detectors requires significant investment on the part of browsers (e.g. “implement an ephemeral mode for indexedDB”) and fixes lagged until mainstream news sites (e.g. Boston Globe, New York Times) began using these detectors more broadly.

See also:

Signaling Privacy Mode

Update: Perhaps surprisingly, the new Microsoft Edge sends a deliberate signal to the user’s default search engine when it is loaded in InPrivate mode. The HTTPS request header PreferAnonymous: 1 is sent on requests to the server to allow it to avoid caching any data related to the user’s use of the search engine.

This header is sent only to the search engine, and not to other sites.

“Guest” Profile

Browsers based on Chromium often also have a Guest Browsing mode, which has a superset of Private Mode behavior.

When you “Browse as Guest”, the browser session treats itself as running in Private Mode, but unlike a normal Private Mode session, the session also starts with an empty user profile. That means the browser will not

  • show history entries in the address box or History page
  • list your bookmarks
  • offer to fill in your account information or passwords
  • offer to fill in autocomplete information
  • etc.

When you close all Guest Profile windows, your history and other state generated in the session is deleted, just like in regular Private mode.

Using a Guest Profile helps prevent you from accidentally leaking information to sites (e.g. inadvertently triggering an autofill).

Advanced Private Modes

Generally, mainstream browsers have taken a middle ground in their privacy features, trading off some performance and some convenience for improved privacy. Users who are very concerned about maintaining privacy from a wider variety of threat actors need to take additional steps, like running their browser in a discardable Virtual Machine behind an anonymizing VPN/Proxy service, disabling JavaScript entirely, etc.

The Brave Browser offers a “Private Window with Tor” feature that routes traffic over the Tor anonymizing network; for many users this might be a more practical choice than the highly privacy-preserving Tor Browser Bundle, which offers additional options like built-in NoScript support to help protect privacy.

Related Links

-Eric