Determining OS Platform Version

In general, you should not care what Operating System visitors are using to visit your website. If you attempt to be clever, you will often get it wrong and cause problems that are an annoyance for users and a hassle for me to debug.

So avoid trying to be nosy/clever if at all possible.

That being said, some websites want to be able to distinguish Windows versions for whatever reason; I’m an engineer, not a cop.

The typical path to get this information from Windows-based browsers is to look at the dreaded User-Agent string, a wretched hive of lies, scum, and villainy.

However, this approach falls down with Windows 11, which reports itself as Windows NT 10.0 (almost certainly for compatibility reasons); in general, browsers are moving toward freezing the User-Agent string to limit passive fingerprinting. The modern mechanism for learning more information about the client is called Client Hints.

If you want to be able to distinguish between Windows 10 and Windows 11, starting around October 2021 in Microsoft Edge 95 (and v95 of most Chromium browsers), the Sec-Ch-UA-Platform-Version Client Hint is the way to go.

Available for request in v95, this value will indicate the UniversalApiContract “API Level” of the Windows platform. The mapping of values to version is

Windows VersionSec-CH-UA-Platform-Version
Windows 7 | 8 | 8.10.0.0
Windows 10 15071.0.0
Windows 10 15112.0.0
Windows 10 16073.0.0
Windows 10 17034.0.0
Windows 10 17095.0.0
Windows 10 18036.0.0
Windows 10 18097.0.0
Windows 10 1903 | 19098.0.0
Windows 10 2004 | 20H2 | 21H110.0.0
Windows 11 Previews13.0.0 | 14.0.0
Windows 11 Release15.0.0

You can request this Client Hint from JavaScript thusly:

navigator.userAgentData.getHighEntropyValues(['platformVersion'])
.then(uapv => { console.log(uapv.platformVersion); });

Note that the userAgentData API requires that your page be a secure context (Served by ~https or on localhost).

Or you can request it be sent to your server as a HTTP request header on subsequent requests by sending a response header:

Response.AddHeader("Accept-CH", "Sec-CH-UA-Platform-Version");
Response.AddHeader("Accept-CH-Lifetime", "86400");

As you may have noticed, the 0.0.0 value is shared across all pre-Windows 10 versions of Windows, so you’ll need to continue to use the User-Agent string to distinguish between Windows 7, 8.0, and 8.1 platforms.

As far as I know, Firefox does not plan to implement this Client Hint.

Bonus: Platform Architecture and Bitness

Distributors of native applications often would like to know the bitness and architecure of the client platform to ensure that they serve the correct installer for their native application. The sec-ch-ua-bitness and sec-ch-ua-arch hints are useful for this purpose.

Unfortunately, there’s presently a bug in Chromium whereby ARM64 devices like the Surface Pro X will return x86 as the architecture.

The Edge-proprietary window.external.getHostEnvironmentValue() API returns the truth {"os-architecture":"ARM64"} but this API is non-standard and you should not use it.

-Eric

PS: Native client apps should generally not check the registry key directly, instead it should query the appropriate “Is this API level supported” API. HKLM\SOFTWARE\Microsoft\WindowsRuntime\WellKnownContracts\Windows.Foundation.UniversalApiContract

Leaky Abstractions

In the late 1990s, the Windows Shell and Internet Explorer teams introduced a bunch of brilliant and intricate designs that allowed extension of the shell and the browser to handle scenarios beyond what those built by Microsoft itself. For instance, Internet Explorer supported the notion of pluggable protocols (“What if some protocol, say, FTPS, becomes as important as HTTP?”) and the Windows Shell offered an extremely flexible set of abstractions for browsing of namespaces, enabling third parties to build browsable “folders” not backed by the file system– everything from WebDAV (“your HTTP-server is a folder“) to CAB Folders (“your CAB archive is a folder“). As a PM on the clipart team in 2004, after I built a .NET-based application to browse clipart from the Office web services, I next sketched out an initial design for a Windows Shell extension that would make it look like Microsoft’s enormous web-based clipart archive were installed in a local folder on your system.

Perhaps the most popular (or infamous) example of a shell namespace extension is the Compressed Folders extension, which handles the exploration of ZIP files. First introduced in the Windows 98 Plus Pack and later included with Windows Me+ directly, Compressed Folders allows billions of Windows users to interact with ZIP files without downloading third-party software. Perhaps surprisingly, the feature was itself was acquired from two third-parties — Microsoft acquired the Explorer integration from Dave Plummer’s “side project”, while a company called InnerMedia claims credit for the “DynaZIP” engine underneath.

Unfortunately, the code hasn’t really been updated in a while. A long while. The timestamp in the module claims it was last updated on Valentine’s Day 1998, and while I suspect there may’ve been a fix here or there since then (and one feature, extract-only Unicode filename support), it’s no secret that the code is, as Raymond Chen says: “stuck at the turn of the century.” That means that it doesn’t support “modern” features like AES encryption, and its performance (runtime, compression ratio) is known to be dramatically inferior to modern 3rd-party implementations.

So, why hasn’t it been updated? Well, “if it aint broke, don’t fix it” accounts for part of the thinking– the ZIP Folders implementation has survived in Windows for 23 years without the howling of customers becoming unbearable, so there’s some evidence that users are happy enough.

Unfortunately, there are degenerate cases where the ZIP Folders support really is broken. I ran across one of those yesterday. I had seen an interesting Twitter thread about hex editors that offer annotation (useful for exploring file formats) and decided to try a few out (I decided I like ReHex best). But in the process, I downloaded the portable version of ImHex and tried to move it to my Tools folder.

I did so by double-clicking the 11.5mb ZIP to open it. I then hit CTRL+A to select all of the files within, then crucially (spoiler alert) CTRL+X to cut the files to my clipboard.

I then created a new subfolder in my C:\Tools folder and hit CTRL+V to paste. And here’s where everything went off the rails– Windows spent well over a minute showing “Calculating…” with no visible progress beyond the creation of a single subfolder with a single 5k file within:

Huh? I knew that the ZIP engine beneath ZIP Folders wasn’t well-optimized, but I’d never seen anything this bad before. After waiting a few more minutes, another file extracted, this one 6.5 mb:

This is bananas. I opened Task Manager, but nothing seemed to be using up much of my 12 thread CPU, my 64gb of memory, or my NVMe SSD. Finally, I opened up SysInternals’ Process Monitor to try to see what was going on, and the root cause of the problem was quickly seen.

After some small reads from the end of the file (where the ZIP file keeps its index), the entire 11 million byte file was being read from disk a single byte at a time:

Looking more closely, I realized that the reads were almost all a single byte, but every now and then, after a specific 1 byte read, a 15 byte read was issued:

What’s at those interesting offsets (330, 337)? The byte 0x50, aka the letter P.

Having written some trivial ZIP-recovery code in the past, I know what’s special about the character P in ZIP files– it’s the first byte of the ZIP format’s block markers, each of which start with 0x50 0x4B. So what’s plainly happening here is that the code is reading the file from start to finish looking for a particular block, 16 bytes in size. Each time it hits a P, it looks at the next 15 bytes to see if they match the desired signature, and if not, it continues scanning byte-by-byte, looking for the next P.

Is there something special about this particular ZIP file? Yes.

The ZIP Format consists of a series of file records, followed by a list (“Central Directory”) of those file records.

Each file record has its own “local file header” which contains information about the file, including its size, compressed size, and CRC-32; the same metadata is repeated in the Central Directory.

However, the ZIP format allows the local file headers to omit this metadata and instead write it as a “trailer” after each individual file’s DEFLATE-compressed data, a capability that is useful when streaming compression– you cannot know the final compressed size for each file until you’ve actually finished compressing its data. Most ZIP files probably don’t make use of this option, but my example download does. (The developer reports that this ZIP file was created by the GitHub CI.)

You can see the CRC and sizes are 0‘d in the header and instead appear immediately following the signature 0x08074b50 (Data Descriptor), just before the next file’s local header:

The 0x08 bit in the General Purpose flag indicates this option; users of 7-Zip can find it mentioned as Descriptor in the entry’s Characteristics column:

Based on the read size (1+15 bytes), I assume the code is groveling for the Data Descriptor blocks. Why it does that (vs. just reading the same data from the Central Directory), I do not know.

Making matters worse, this “read the file, byte by byte” crawl through the file doesn’t just happen once– it happens at least once for every file extracted. Making matters worse, this data is being read with ReadFile rather than fread() meaning that there’s no caching in userspace, requiring we go to the kernel for every byte read.

Eventually, after watching about 85 million single byte reads, Process Monitor hangs:

After restarting and configuring Process Monitor with Symbols, we can examine the one-byte reads and get a hint of what’s going on:

The GetSomeBytes function is getting hammered with calls passing a single byte buffer, in a tight loop inside the readzipfile function. But look down the stack and the root cause of the mess becomes clear– this is happening because after each file is “moved” from the ZIP to the target folder, the ZIP file must be updated to remove the file that was “moved.” This deletion process is inherently not fast (because it results in shuffling all of the subsequent bytes of the file and updating the index), and as implemented in the readzipfile function (with its one-byte read buffer) it is atrociously slow.

Back up in my repro steps, note that I hit CTRL+X to “Cut” the files, resulting in a Move operation. Had I instead hit CTRL+C to “Copy” the files, resulting in a Copy operation, the ZIP folder would not have performed a delete operation as each file was extracted. The time required to unpack the ZIP file drops from over thirty minutes to four seconds. For perspective, 7-Zip unpacks the file in under a quarter of a second, although it cheats a little.

And here’s where the abstraction leaks— from a user’s point-of-view, copying files out of a ZIP file (then deleting the ZIP) vs. moving the files from a ZIP file seems like it shouldn’t be very different. Unfortunately, the abstraction fails to fully paper over the reality that deleting from certain ZIP files is an extremely slow operation, while deleting a file from a disk is usually trivial. As a consequence, the Compressed Folder abstraction works well for tiny ZIPs, but fails for the larger ZIP files that are becoming increasingly common.

While it’s relatively easy to think of ways to dramatically improve the performance of this scenario, precedent suggests that the code in Windows is unlikely to be improved anytime soon. Perhaps for its 25th Anniversary? 🤞

– Eric

Offline NetLog Viewing

A while back, I explained how you can use Telerik Fiddler or the Catapult NetLog Viewer to analyze a network log captured from Microsoft Edge, Google Chrome, or another Chromium or Electron-based application.

While Fiddler is a native app that runs locally, the Catapult NetLog Viewer is a JavaScript application that runs in your browser. Because NetLogs can contain sensitive data, some users have worried about the privacy of the viewer– what if someday it started leaking sensitive data from logs, either unintentionally or maliciously?

Fortunately, the NetLog Viewer is a self-contained single page application that doesn’t need a network connection to run. You can use it entirely offline, either from a Virtual Machine with no network connection, or from a browser instance configured to override all network requests.

Your first step is to get an copy of the viewer as a file. You can do that by right-clicking this link and choosing “Save Link As”. Save the HTML file somewhere locally, e.g. C:\temp\NetLogView.html on Windows.

If you want to run it from a disconnected VM, simply copy the file into such a VM and you’re good to go.

If, however, you want the convenience of running the viewer from your Internet-connected PC without worrying about leaks, you can run it from a browser instance that won’t make network connections.

After saving the file, open it in a new browser window thusly:

msedge.exe --user-data-dir=C:\temp\profile --inprivate --host-rules="MAP * 0.0.0.0" --app=C:\temp\NetLogView.html

The command line arguments bear some explanation. In reverse order:

  • The app argument instructs Edge to open the supplied file with a minimal browser UI, as if it were a native app.
  • The host-rules argument tells the browser instance to direct all network requests to an IP address of 0.0.0.0. On Windows, such requests instantly fail. On Mac/Linux, the null IP points back at your own PC.
  • The inprivate argument directs the browser to discard all storage after the app exits (since it’s not needed). For Chrome, use --incognito instead.
  • The user-data-dir instructs the browser to use a temporary browser profile (which prevents the app’s window from being merged into an existing browser process, such that the host-rules argument would’ve been ignored.)


While none of this is strictly necessary (the NetLog Viewer doesn’t leak data), it’s always nice to be able to discard attack surface wherever possible.

-Eric

Download Blocking by File Type

I’ve previously spoken about the magic of the File Type Policies component — a mechanism that allows files to be classified by their level of “dangerousness”, such that harmless files (e.g. .txt files) can be downloaded freely, whilst potentially-dangerous files (e.g. .dll files) are subjected to a higher degree of vetting and a more security-conscious user-experience.

File Type Danger Level

Microsoft Edge inherits its file type policies from the upstream Chromium browser; you can view the current contents of the list here, and documentation of its format here.

Within the list, you’ll see that each type has a danger_level, which is one of three values: DANGEROUS, NOT_DANGEROUS, or ALLOW_ON_USER_GESTURE.

The first two are simple: NOT_DANGEROUS means Safe to download and open, even if the download was accidental. No additional warnings are necessary. DANGEROUS means Always warn the user that this file may harm their computer. Let users continue or discard the file. If [SmartScreen or Safe Browsing] returns a SAFE verdict, still warn the user before saving the file.

The third setting, ALLOW_ON_USER_GESTURE1 is more subtle. Such files are potentially dangerous, but likely harmless if the user is familiar with download site and if the download was intentional. Microsoft Edge will allow such downloads to proceed automatically if two conditions are both met:

  1. User Gesture: There is a user gesture associated with the network request that initiated the download (e.g. the user clicked a link to the download).
  2. Familiar Initiator: There is a recorded prior visit to the referring origin prior to the most recent midnight (i.e. yesterday or earlier). Such a visit implies that the user has at least some history of visiting the site that kicked off the download.

The download will also proceed automatically if the user explicitly initiated a download by using the Save link as context menu command, entered directly into the browser’s address bar the download’s URL, or if Microsoft Defender SmartScreen (in Edge, or Google Safe Browsing in Chrome) indicates that the file is known safe.

Update: Starting in version 91, Microsoft Edge will join Google Chrome in interrupting downloads that lack the required gesture.

User Experience for Downloads Lacking Gestures

Within Google Chrome, a download lacking a required gesture shows explicit buttons to allow the user to decide whether to proceed with the download or abandon it:

In contrast, Microsoft Edge states that the download “was blocked”, although the same options, titled Keep and Delete, are available from the … menu on the download item.

If you visit edge://downloads, you’ll see the same options:

Enterprise Controls

While users are somewhat unlikely to encounter download interruptions for sites they use every day, they might encounter them for legitimate downloads on sites that they use rarely or in sites that hit “Corner Cases” described in a section below.

To help streamline the user-experience for Enterprises, a Group Policy is available.

Enterprises can use ExemptDomainFileTypePairsFromFileTypeDownloadWarnings to specify the filetypes that are allowed to download from specific sites without interruption.

[{"file_extension":"xml","domains":["contoso.com", "woodgrovebank.com"]},
{"file_extension":"msg", "domains": ["*"]}]

If the SmartScreenForTrustedDownloadsEnabled (or equivalent policy for Chrome) is set to 0 (disabled), and the file download’s URL is Trusted (on Windows, in the Local Machine, Intranet, or Trusted zone) then the download will proceed without interruption (even without a gesture), regardless of danger_level. (Aside: This seems a bit strange, but feels more logical if you pretend that the file type warnings are a part of SmartScreen).

File Types Requiring a Gesture

The latest file types policies are published in the Chromium source code. As of May 2021, file types with a danger_level of ALLOW_ON_USER_GESTURE on at least one OS platform include:
accda, accdb, accde, accdr, action, ad, ade, adp, apk, app, application, appref-ms, as, asp, asx, bas, bash, bat, caction, cdr, cer, chi, chm, cmd, com, command, configprofile, cpgz, cpi, cpl, crt, crx, csh, dart, dc42, deb, definition, der, desktop, dex, diskcopy42, dmg, dmgpart, dvdr, dylib, efi, eml, exe, fon, fxp, hlp, htt, img, imgpart, inf, ins, internetconnect, inx, isp, isu, job, js, jse, ksh, lnk, mad, maf, mag, mam, maq, mar, mas, mat, mau, mav, maw, mda, mdb, mde, mdt, mdw, mdz, mht, mhtml, mmc, mobileconfig, mpkg, msc, msg, msh, msh1, msh1xml, msh2, msh2xml, mshxml, msi, msp, mst, ndif, networkconnect, ocx, ops, out, oxt, paf, partial, pax, pcd, pet, pif, pkg, pl, plg, prf, prg, ps1, ps1xml, ps2, ps2xml, psc1, psc2, pst, pup, py, pyc, pyo, pyw, rb, reg, rels, rgs, rpm, run, scr, sct, search-ms, service, settingcontent-ms, sh, shar, shb, shs, slk, slp, smi, sparsebundle, sparseimage, svg, tcsh, toast, u3p, udif, vb, vbe, vbs, vbscript, vdx, vsd, vsdm, vsdx, vsmacros, vss, vssm, vssx, vst, vstm, vstx, vsw, vsx, vtx, wflow, workflow, ws, wsc, wsf, wsh, xip, xml, xnk, xrm-ms, xsd, xsl

Other Fields in the File Type Policies

  • You’ll also note that some file types have an auto_open_hint which controls whether the user may configure that type of file to open automatically when the download completes.
  • File type settings sometimes vary depending on the client OS platform (an .exe is not dangerous on a Mac, while an .applescript is harmless on Windows). The platform attribute of an entry specifies on which OS the danger_level applies.
  • The max_file_size_to_analyze field controls how big of a file (.zip, .rar, etc) the browser will be willing to unpack to scan it for dangerous content.

Group Policies

DownloadRestrictions is a policy that makes a complicated browser behavior even more complicated. When you set DownloadRestrictions to 1, Edge won’t just interrupt the download, it will block it.

Enterprises can use ExemptDomainFileTypePairsFromFileTypeDownloadWarnings to specify the filetypes that are allowed to download from specific sites without blocking.

Corner Cases

  • If you put referrerpolicy=”no-referrer” on your download link (or otherwise suppress referrers), the Familiar Initiator check fails.
  • Prior to v94, if you initiate the download by dynamically creating an A element with a download attribute, then click it from JavaScript, User Gesture check fails.

As of August 2021, Microsoft Outlook Web Access’ email attachment file downloads encounter both of these issues.

Test cases for these conditions can be found here. (Note that you’ll have to have visited webdbg.com yesterday or earlier for the familiarity check to pass).


1 ALLOW_ON_USER_GESTURE_AND_FAMILIAR_INITIATOR would be the accurate name for the setting

Per-Site Permissions in Edge

Last year, I wrote about how the new Microsoft Edge browser mostly ignores Security Zones (except in very rare circumstances) to configure security and permissions decisions. Instead, in Chromium per-site permissions are controlled by settings and policies expressed using a simple syntax with limited wildcarding support.

Settings Page’s Site Permissions and Group Policy

Internet Explorer offered around 88 URLAction permissions, but the majority (62) of these settings have no equivalent; for instance, there are a dozen that control various features of ActiveX controls, a technology that does not exist in the new Edge.

Unfortunately, there’s no document mapping the old URLActions to the new equivalents (if any) available within the new Edge. 

When users open chrome://settings/content/siteDetails?site=https://example.com, they’ll find a long list of configuration switches and lists for various permissions. Users rarely use the Settings Page directly, instead making choices using various widgets and toggles in the Page Info dropdown (which appears when you click the lock) or via various prompts or buttons at the right-edge of the address bar/omnibox.

Enterprises can use Group Policy to provision site lists for individual policies that control the browser’s behavior. To find these policies, simply open the Edge Group Policy documentation and search for ForUrls to find the policies that allow and block behavior based on the loaded site’s URL. I recently wrote a post about Chromium’s URL Filter syntax, which doesn’t always work like one might expect. Most of the relevant settings are listed within the Group Policy for Content Settings.

There are also a number of policies whose names contain Default that control the default behavior for a given setting.

Here’s a list of Site Settings with information about their policies and behavior:

As you can see, some of these settings are very obscure (WebSerial, WebMIDI) while others will almost never be changed away from their defaults (Images).

-Eric

Specifying Per-Site Policy with Chromium’s URL Filter Format

Chromium-based browsers like Microsoft Edge make very limited use of Windows Security Zones. Instead, most permissions and features that offer administrators per-site configuration via policy rely on lists of rules in the URL Filter Format.

Filters are expressed in a syntax that is similar to other matching rules, but different enough to cause confusion. For instance, consider a URLBlocklist rule expressed as follows:

These filters don’t work as expected. The HTTPS rule should not have a trailing * in the path component (it won’t match anything), while the data: rule requires a trailing * to function.

The syntax has a few other oddities as well:

  • A leading dot before the host means exactly match; the filter .example.com matches just https://example.com but not https://www.example.com
  • You may specify a path prefix (example.com/foo) but you must not include a wildcard * anywhere in the path
  • You may specify wildcards in a query (example.com?bar=*); you may omit the path to have the querystring checked on all pages, or include a path to only check the querystring on pages within the path.
  • A rule of blob:* doesn’t seem to match blob URLs, while a rule of data:* does seem to match all data URLs.

Unfortunately, there’s not a great debugger for figuring out the proper syntax. You can use the chrome://policy page to see whether Chrome finds any glaring error in the policy:

…but short of testing your policy there’s not a great way to verify it does what you hope.

Q: The problem of special-URLs

There are a variety of special URLs (particularly blob and data) that do not directly express a hostname– instead, the URL exists within a security context that is not included in the URL itself. This can cause problems for Policies if the code implementing the policy does not check the URL of the security context and looks only at the blob/data URL directly. A system administrator might set a policy for downloads from https://example.com/download, but if download page on that site uses a script-generated file download (e.g. a blob), the policy check might overlook the rule for example.com because it checks just the blob: URL.

An example bug can be found here.

Q: Can filters match on a site’s IP?

The Permissions system’s “Site Lists” feature does not support specifying an IP-range for allow and block lists. Wildcards are not supported.

It does support specification of individual IP literals, but such rules are only respected if the user navigates to the site using said literal (e.g. http://127.0.0.1/). If a hostname is used (http://localhost), the IP Literal rule will not be respected even though the resolved IP of the host matches the filter-listed IP.

Q: Can filters match just dotless hostnames?

Not today, no. You must individually list each desired hostname, e.g. (https://payroll, https://stock, https://who, etc).

Chromium’s URL Filter Format is convenient if your intranet is structured under one private domain (e.g. *.intranet.example.com) but is much less convenient if your Intranet uses dotless hostnames (http://example) or many disjoint private domains.

The ability to match only hostnames not containing dots would be convenient to accommodate the old IE behavior whereby Windows would map dotless hostnames to the Local Intranet Zone by default. (To my surprise, there’s been no significant demand for this capability in the first year of Edge’s existence, so perhaps corporate intranets are no longer using dotless hostnames very much?)

References

Web Proxy Authentication

Last year, I wrote about how the new Microsoft Edge’s adoption of the Chromium stack changed proxy determination away from the Windows Service (WinHTTP Proxy Service) to similar but not identical code in Chromium. This change mostly goes unnoticed, but it can have performance and functionality implications.

In today’s post, I want to explore another side-effect of changing away from the Windows stack– the impact on Proxy Authentication.

Background

When the client is configured to route its traffic through an authenticating proxy server, the proxy responds to any request that does not contain a Proxy-Authorization request header with a HTTP/407 response that demands credentials, specifying the desired authentication scheme using a Proxy-Authenticate header:

The client is expected to resend the request with the requested credentials added in the Proxy-Authorization header.

Browsers tend to support four methods for authentication: the same four used for plain HTTP authentication: Basic, Digest, NTLM, and Negotiate (NTLM+Kerberos). The NTLM and Negotiate schemes are generally much preferred, as they can be configured to authenticate silently without showing a prompt to the user. However, depending on configuration (e.g. not in the right zone, or when the user’s Windows Login Credentials are not the ones the proxy requires), the user might see a proxy-authentication prompt in the browser.

Browser Credential UX

The prompt differs depending on the client; Google Chrome, and until recently Microsoft Edge, show an auth “bubble” anchored to the bottom of the browser address bar UI. In contrast, the latest versions of Edge and Internet Explorer show the battleship gray Windows Hello modal dialog box:

While these experiences are all conceptually similar, they differ in very important ways. First, note that Internet Explorer’s option shows a “Remember my credentials” checkbox while Chrome/Edge do not. Proxy credentials saved by Internet Explorer will be reused without prompting, across browsing sessions and across other applications that use the WinHTTP Proxy Service.

In contrast, in Chrome and older Edge, the proxy credentials prompt is integrated with the browser’s Password Manager. If the user accepts the followup prompt to save the proxy credentials, those credentials will autofill the dialog in future browser sessions. Notably, however, the proxy authentication is not automatic– the user must manually click the Sign in button once per browser session.

Storing a credential in your Edge profile’s password manager only applies to that one browser profile– other profiles and apps do not have access to the credential.

In Edge 88+, this password manager integration was lost in the switch to use the Windows Hello prompt for NTLM/Negotiate. A new policy (WindowsHelloForHTTPAuthEnabled) added to Edge 90 changes Edge’s proxy credentials dialog back to the older dialog.

Other App Credential UX

Support for manually-authenticating proxies outside of browsers has always been terrible, and many Win8 applications didn’t even bother to handle it. Sadly, that shortcoming remains in Windows 10, where you can’t even use the Windows Store when a proxy needs credentials that are not supplied automatically. An generic error page is shown:

However, as mentioned earlier, around IE11 a change was made to push proxy authentication creds entered in Internet Explorer’s proxy authentication prompt into the Windows Credential Manager (CredMan):

As seen in the Control Panel view (Persistence: Enterprise):

Even if you untick the “Remember my credentials” box on the prompt, the entered creds are still remembered for the Windows session (Persistence: Login Session):

As a consequence, users can “unbreak” various Windows scenarios by just using the browser to populate the Credential Manager with the proxy credentials.

We’ve lost that workaround in the new world, because Chromium’s password manager is not integrated with CredMan (for a number of sane1 reasons).

Surface Hub

The Microsoft Surface Hub is a “whiteboard computing” device designed for corporate conference rooms and the like. By default, the Windows logon session on the device uses an account named ComputerName\SurfaceHub, and that account is very unlikely to be supported by an authenticating proxy. To support it, Windows’ Settings applet on Surface Hub has a custom option:

However, this option doesn’t presently do anything for Chromium’s network stack. When Edge gets a Proxy-Authorization: Negotiate challenge from the proxy, it will respond with the credentials for the SurfaceHub account rather than the machine credentials, and the login will likely fail. At present, we don’t have a great workaround for this: it’s probably possible to write a browser extension (catching the onAuthRequired event and responding with non-default credentials) but this won’t pick up the Windows settings.

Other Brokenness

  • We recently fixed a bug in Edge Canary where the browser would crash on startup if the user was behind a manually-authenticated proxy server. A network request triggered very early in browser startup wasn’t prepared for an auth challenge.
  • If you have the Microsoft Office Chrome extension installed, it can (for reasons unknown) automatically cancel the manual proxy authentication dialog before the user has a chance to submit credentials.

In conclusion: Two decades in, support for proxy authentication remains a mess. The feature is required by the networks of some corporate titans (and thus support continues to limp along), but many corner cases remain broken. Consider carefully whether proxy authentication needs to be a part of your network strategy.

-Eric

1 There are several reasons, but one obvious one is that Chromium uses a “profiles” system whereby one Windows logon account may have multiple browser profiles, each of which is independent and does not share data with the others. This precludes direct use of system-level features that do not offer such partitioning.

window.close() Restrictions

Sometimes, Web Developers are surprised to find that the window.close() API doesn’t always close the browser window.

When looking at the Developer Tools console, they’ll see a message like:

Scripts may close only the windows that were opened by them.

Why Do Browsers Limit close()?

Before we dive into what factors govern what happens when close() is called, it’s important to understand why there’s a restriction at all.

While sometimes explained away as nebulous “Security reasons!“, the core reason is more aligned with simple user-experience— users could lose important state in their browser window or “back/forward” stack (in IE, we called this the TravelLog) if a tab or window went away unexpectedly. The user might’ve been using a tab to explore a set of search results, and if one of those results could blow away both itself and the backstack with the search results page, this would be pretty annoying.

There’s a minor anti-abuse/security argument as well– if a browser tab could blow itself away freely, this might be useful as a part of scareware or other user-experience abuses.

What Does the Standard Say?

Here’s what the dom-window-close section of the HTML Standard has to say:

A browsing context is script-closable if it is an auxiliary browsing context that was created by a script (as opposed to by an action of the user), or if it is a top-level browsing context whose session history contains only one Document.

This seems simple enough, although the parts I’ve bolded hide a lot of complexity and nuance. (Most obviously, “what should we do if the script was run in response to an action of the user?“)

What Do Browsers Do?

Unfortunately for us, each browser has a different set of behaviors (explore with this test page), partly because most were implemented before the standard was written.

Internet Explorer

In Internet Explorer, a tab/window will be closed silently if it was created as a result of a JavaScript window.open() call. There’s no attempt to check whether the back/forward stack contains only one document: a tab with a large TravelLog will still close silently if it was opened by script. (IE also allows HTA documents to close themselves without restriction.)

In all other circumstances, the tab/window will not silently close: instead, the user is presented with a one of two modal dialogs, depending on whether the page represents the only tab in the browser window:

Chromium (Microsoft Edge / Google Chrome / etc)

As of Chromium 88, window.close() succeeds if the new window/tab has an opener or if the back/forward stack contains fewer than two entries.

As you can see, there are subtle differences here between what the spec requires and what the browser implements.

First, notice that I said “has an opener” rather than “was created by a script.” Recall that the opener property allows a popup window to refer to the tab that created it.

  • If the user creates a new tab by clicking the frame button, hitting Ctrl+T, Shift+Clicking a link, or launching a URL from the Shell, the resulting new tab doesn’t have an opener set.
  • In contrast, if the tab is opened via open() or hyperlink with a named target (not _blank), then by default, the opener property is set.
  • Any hyperlink may include rel=opener or rel=noopener to specify whether the new tab has an opener set.
  • A open() JavaScript call can specify noopener in its windowFeatures string to set the new tab’s opener to null.

As you can see from the above list, both normal link clicks and JavaScript open() invocations can result in tabs with or without the opener set. This can be very confusing: Shift+click on a link might result in a tab that cannot self-close, while Left-click on that same link results in a tab that can always close itself.

Secondly, notice I said “entries” rather than “Documents.” In most cases, these are equivalent, but they’re not exactly the same. Consider the case where the new tab navigates to a HTML document with a Table of Contents at the top. The user clicks on a ToC link to #Section3 and the browser dutifully scrolls down to that section. The back/forward stack now contains two entries, both pointing to the same document. Chromium blocks window.close(), but it shouldn’t. This longstanding shortcoming was made more visible in Chromium 88, which now gives links targeting _blank the noopener behavior by default.

crbug.com/1170131 tracks fixing this issue by counting the number of Documents in the back/forward stack, but it will be tricky to fix because presently the renderer process running JavaScript has access only to the count of entries in the back/forward stack, not their URLs.

Chromium: User-Experience

When Chrome blocks close(), it emits a notice to the console:

Scripts may close only the windows that were opened by them.

…but there’s no notice to the user, who may be left confused if they clicked on a “Close” button or link inside the page. Recently filed crbug.com/1170034 suggests introducing a dialog box like Internet Explorer’s. (As an aside, it also sets a new standard for bug filing by including comic-book style art showing unhappy users converting to happy users if the proposed feature is landed. :)

Chromium: Exotic Bug Trivia

This is a super-obscure corner case. However, I’ve seen it independently reported against both Chrome and Edge in the span of five years, and it makes me laugh.

If you set Chromium’s On Startup option to “Continue where you left off”, navigate to a page that tries to close itself, and then close the window, the browser will subsequently kill itself on every startup.

It’s hard to get into this state, but it’s still possible in Chrome/Edge 90.

Repro steps: Visit https://webdbg.com/test/opener/. Click the ‘Page that tries to close itself‘ link. Hit Ctrl+Shift+Delete and Clear Browsing History (to clear the back/forward stack). Close the browser with the ‘X’ icon. Now, try to launch the browser from your start menu. Giggle wildly as your browser launches and then blows itself away.

Safari/WebKit

WebKit’s code is similar to Chromium’s (unsurprising, considering their shared lineage) except it does not equate navigations triggered with noopener with those created by browser UI. Thus, in Safari, the user may click between many different same-origin pages and close() is still permitted.

If close() is blocked, Safari’s (well-hidden) JavaScript console shows:

Can't close the window since it was not opened by JavaScript

Firefox

Unlike Chromium, Firefox correctly implements the “only one Document” part of the spec. Firefox calls IsOnlyTopLevelDocumentInSHistory which calls IsEmptyOrHasEntriesForSingleTopLevelPage() which enumerates the history list to check. If there is more than one entry, whether all of the entries are for the same Document. If they are all for the same document, the close() succeeds.

Firefox offers an about:config override setting named dom.allow_scripts_to_close_windows that bypasses the default restrictions.

When Firefox blocks close(), it emits a notice to the console:

Scripts may not close windows that were not opened by script.

There’s an 18 year-old feature request for Firefox to show a confirmation dialog instead of failing silently.

In Conclusion

umm…. browsers are wicked complicated?

-Eric

Sandboxing vs. Elevated Browsing (As Administrator)

The Web Browser is the most security-critical application on most users’ systems– it accepts untrusted input from servers anywhere in the world, parses that input using dozens to hundreds of parsers, and renders the result locally as fast as it can. For performance reasons, almost all code in almost all browsers is written in memory-unsafe languages.

To mitigate this insanity, web browsers perform this chainsaw juggling act inside nested layers of sandboxing. When an attacker inevitably manages to find an exploit, the resulting damage might be contained by the sandbox. Chromium calls this “The Rule of 2”:

alt text

As a consequence, most real-world browser attacks now must include exploits for two vulnerabilities: the original RCE, and a Sandbox Escape.

A web browser uses multiple mechanisms for sandboxing, from “Design sandboxing” (Same-Origin-Policy, limited JavaScript capabilities) to Process Sandboxing (AppContainers, UAC Integrity Levels, Restricted Tokens, etc) to optional VM sandboxing (WDAG).

Process sandboxing arose in the mid-200Xs as a defense-in-depth against Design Sandboxing mistakes – any error in implementation meant that an attacker could achieve arbitrary code execution (RCE) and take over the victim user’s computer.

Windows Vista introduced the notion of UAC Integrity Levels, whereby each system object (including processes) had an integrity level that limited the other objects it could read and write. Internet Explorer 7 built atop the UAC Integrity Levels system and introduced the notion of “Protected Mode” whereby Internet Zone content ran inside a tighter sandbox with decreased write-access to the system. Google Chrome and later versions of IE went much further, with AppContainers and broader process restrictions to make exploitation more challenging and reduce its impact.

Tools like SysInternals’ Process Explorer allow inspecting some of the sandboxing limitations applied to browsers’ processes. For instance, you can see the Windows Job restrictions applied to this Edge Renderer Process:

You’ll note that browsers typically use multiple processes with different sandboxing restrictions attached to each. For instance, in a default launch of Chrome, you’ll see that the main browser process launches at the default (“Medium”) Integrity Level, while the GPU process runs at Low Integrity, and the Renderer processes run at Untrusted Integrity (the lowest):

Looking at Internet Explorer, you can see that its main process runs at Medium, an Intranet Zone tab process runs at Medium, and an Internet Zone tab process runs at Low:

If you enable Enhanced Protected Mode, that Internet Zone tab process is further restricted to run in an AppContainer:

Run As Administrator

One interesting difference between Legacy IE and Microsoft Edge/Chrome is the behavior when you “Run As Administrator”: In the old IE, this resulted in the tab processes running unsandboxed at High IL:

For reasons that are hopefully obvious, running web content outside of a sandbox at High IL is an extremely dangerous configuration.

In contrast to IE behavior, in Edge/Chrome, the browser process, crash handler, and some utility processes run at High IL, but the sandboxed Renderers stay in Untrusted IL processes.

While this is clearly a safer configuration that what Internet Explorer had, the transitive nature of UAC Elevation means that many executables launched by this browser are launched at High Integrity, including Native Messaging Hosts, AppProtocol Handlers, downloaded executables, etc.

Functional Bugs

Beyond the increased security attack surface, there are some functional problems with running a Chromium-based browser at High IL. For instance, if you have AppLocker enabled, you may find that all of your sandboxed renderer processes crash (crbug.com/740132), or that link launches from non-admin processes are ignored (crbug.com/801539#c9). 

As a result of problems like these, Microsoft Edge will attempt to silently deelevate and relaunch itself when launched as an Administrator, running at Medium IL instead of High. If this feature is disabled (--do-not-de-elevate), Edge will show a warning to help users recognize the unexpected situation:

Administrator Mode Detected balloon in Edge

Generally speaking, there’s no legitimate reason to run your browser at High IL, and you shouldn’t ever do it.

But…

Impact on Windows Integrated Authentication

Recently, I was disappointed to learn that there are some scenarios where running at HighIL is still necessary. These scenarios are based on Windows Integrated Authentication, whereby the web server uses passthrough authentication to pass the user’s Elevated Authentication Token through to some backend service that requires membership in the Administrators group. If the web server receives and passes along the admin user’s non-elevated token (because the browser was run at Medium), the backend services will refuse to undertake the privileged operation.

The fact that such an application architecture ever worked is interesting but a bit bananas.

I’d be much happier with a design whereby this Elevated token passing scenario didn’t have to happen in the browser— either configuration should happen in some non-browser application, or the Service Account could invoke some native UI to collect the admin’s elevated credentials during the specific configuration operation where such credentials are needed. From a layering POV, I’m sure this would be complicated, because the ability of a background service to pop stuff on the admin user’s desktop is intentionally limited.

If you’re depending upon an architecture like this, please stop. Pretty please! It’s fragile: it behaves differently based on the user’s default browser, behaves differently depending on what browser windows the user already has open (launch msedge.exe as Admin doesn’t result in an Elevated browser if Edge was already open), and exposes the user to a greater attack surface if they use the Elevated browser for anything else after finishing with your scenario.

Stay safe (and de-elevated) out there!

-Eric