Leaky Abstractions

In the late 1990s, the Windows Shell and Internet Explorer teams introduced a bunch of brilliant and intricate designs that allowed extension of the shell and the browser to handle scenarios beyond what those built by Microsoft itself. For instance, Internet Explorer supported the notion of pluggable protocols (“What if some protocol, say, FTPS, becomes as important as HTTP?”) and the Windows Shell offered an extremely flexible set of abstractions for browsing of namespaces, enabling third parties to build browsable “folders” not backed by the file system– everything from WebDAV (“your HTTP-server is a folder“) to CAB Folders (“your CAB archive is a folder“). As a PM on the clipart team in 2004, after I built a .NET-based application to browse clipart from the Office web services, I next sketched out an initial design for a Windows Shell extension that would make it look like Microsoft’s enormous web-based clipart archive were installed in a local folder on your system.

Perhaps the most popular (or infamous) example of a shell namespace extension is the Compressed Folders extension, which handles the exploration of ZIP files. First introduced in the Windows 98 Plus Pack and later included with Windows Me+ directly, Compressed Folders allows billions of Windows users to interact with ZIP files without downloading third-party software. Perhaps surprisingly, the feature was itself was acquired from two third-parties — Microsoft acquired the Explorer integration from Dave Plummer’s “side project”, while a company called InnerMedia claims credit for the “DynaZIP” engine underneath.

Unfortunately, the code hasn’t really been updated in a while. A long while. The timestamp in the module claims it was last updated on Valentine’s Day 1998, and while I suspect there may’ve been a fix here or there since then (and one feature, extract-only Unicode filename support), it’s no secret that the code is, as Raymond Chen says: “stuck at the turn of the century.” That means that it doesn’t support “modern” features like AES encryption, and its performance (runtime, compression ratio) is known to be dramatically inferior to modern 3rd-party implementations.

So, why hasn’t it been updated? Well, “if it aint broke, don’t fix it” accounts for part of the thinking– the ZIP Folders implementation has survived in Windows for 23 years without the howling of customers becoming unbearable, so there’s some evidence that users are happy enough.

Unfortunately, there are degenerate cases where the ZIP Folders support really is broken. I ran across one of those yesterday. I had seen an interesting Twitter thread about hex editors that offer annotation (useful for exploring file formats) and decided to try a few out (I decided I like ReHex best). But in the process, I downloaded the portable version of ImHex and tried to move it to my Tools folder.

I did so by double-clicking the 11.5mb ZIP to open it. I then hit CTRL+A to select all of the files within, then crucially (spoiler alert) CTRL+X to cut the files to my clipboard.

I then created a new subfolder in my C:\Tools folder and hit CTRL+V to paste. And here’s where everything went off the rails– Windows spent well over a minute showing “Calculating…” with no visible progress beyond the creation of a single subfolder with a single 5k file within:

Huh? I knew that the ZIP engine beneath ZIP Folders wasn’t well-optimized, but I’d never seen anything this bad before. After waiting a few more minutes, another file extracted, this one 6.5 mb:

This is bananas. I opened Task Manager, but nothing seemed to be using up much of my 12 thread CPU, my 64gb of memory, or my NVMe SSD. Finally, I opened up SysInternals’ Process Monitor to try to see what was going on, and the root cause of the problem was quickly seen.

After some small reads from the end of the file (where the ZIP file keeps its index), the entire 11 million byte file was being read from disk a single byte at a time:

Looking more closely, I realized that the reads were almost all a single byte, but every now and then, after a specific 1 byte read, a 15 byte read was issued:

What’s at those interesting offsets (330, 337)? The byte 0x50, aka the letter P.

Having written some trivial ZIP-recovery code in the past, I know what’s special about the character P in ZIP files– it’s the first byte of the ZIP format’s block markers, each of which start with 0x50 0x4B. So what’s plainly happening here is that the code is reading the file from start to finish looking for a particular block, 16 bytes in size. Each time it hits a P, it looks at the next 15 bytes to see if they match the desired signature, and if not, it continues scanning byte-by-byte, looking for the next P.

Is there something special about this particular ZIP file? Yes.

The ZIP Format consists of a series of file records, followed by a list (“Central Directory”) of those file records.

Each file record has its own “local file header” which contains information about the file, including its size, compressed size, and CRC-32; the same metadata is repeated in the Central Directory.

However, the ZIP format allows the local file headers to omit this metadata and instead write it as a “trailer” after each individual file’s DEFLATE-compressed data, a capability that is useful when streaming compression– you cannot know the final compressed size for each file until you’ve actually finished compressing its data. Most ZIP files probably don’t make use of this option, but my example download does. (The developer reports that this ZIP file was created by the GitHub CI.)

You can see the CRC and sizes are 0‘d in the header and instead appear immediately following the signature 0x08074b50 (Data Descriptor), just before the next file’s local header:

The 0x08 bit in the General Purpose flag indicates this option; users of 7-Zip can find it mentioned as Descriptor in the entry’s Characteristics column:

Based on the read size (1+15 bytes), I assume the code is groveling for the Data Descriptor blocks. Why it does that (vs. just reading the same data from the Central Directory), I do not know.

Making matters worse, this “read the file, byte by byte” crawl through the file doesn’t just happen once– it happens at least once for every file extracted. Making matters worse, this data is being read with ReadFile rather than fread() meaning that there’s no caching in userspace, requiring we go to the kernel for every byte read.

Eventually, after watching about 85 million single byte reads, Process Monitor hangs:

After restarting and configuring Process Monitor with Symbols, we can examine the one-byte reads and get a hint of what’s going on:

The GetSomeBytes function is getting hammered with calls passing a single byte buffer, in a tight loop inside the readzipfile function. But look down the stack and the root cause of the mess becomes clear– this is happening because after each file is “moved” from the ZIP to the target folder, the ZIP file must be updated to remove the file that was “moved.” This deletion process is inherently not fast (because it results in shuffling all of the subsequent bytes of the file and updating the index), and as implemented in the readzipfile function (with its one-byte read buffer) it is atrociously slow.

Back up in my repro steps, note that I hit CTRL+X to “Cut” the files, resulting in a Move operation. Had I instead hit CTRL+C to “Copy” the files, resulting in a Copy operation, the ZIP folder would not have performed a delete operation as each file was extracted. The time required to unpack the ZIP file drops from over thirty minutes to four seconds. For perspective, 7-Zip unpacks the file in under a quarter of a second, although it cheats a little.

And here’s where the abstraction leaks— from a user’s point-of-view, copying files out of a ZIP file (then deleting the ZIP) vs. moving the files from a ZIP file seems like it shouldn’t be very different. Unfortunately, the abstraction fails to fully paper over the reality that deleting from certain ZIP files is an extremely slow operation, while deleting a file from a disk is usually trivial. As a consequence, the Compressed Folder abstraction works well for tiny ZIPs, but fails for the larger ZIP files that are becoming increasingly common.

While it’s relatively easy to think of ways to dramatically improve the performance of this scenario, precedent suggests that the code in Windows is unlikely to be improved anytime soon. Perhaps for its 25th Anniversary? 🤞

– Eric

Offline NetLog Viewing

A while back, I explained how you can use Telerik Fiddler or the Catapult NetLog Viewer to analyze a network log captured from Microsoft Edge, Google Chrome, or another Chromium or Electron-based application.

While Fiddler is a native app that runs locally, the Catapult NetLog Viewer is a JavaScript application that runs in your browser. Because NetLogs can contain sensitive data, some users have worried about the privacy of the viewer– what if someday it started leaking sensitive data from logs, either unintentionally or maliciously?

Fortunately, the NetLog Viewer is a self-contained single page application that doesn’t need a network connection to run. You can use it entirely offline, either from a Virtual Machine with no network connection, or from a browser instance configured to override all network requests.

Your first step is to get an copy of the viewer as a file. You can do that by right-clicking this link and choosing “Save Link As”. Save the HTML file somewhere locally, e.g. C:\temp\NetLogView.html on Windows.

If you want to run it from a disconnected VM, simply copy the file into such a VM and you’re good to go.

If, however, you want the convenience of running the viewer from your Internet-connected PC without worrying about leaks, you can run it from a browser instance that won’t make network connections.

After saving the file, open it in a new browser window thusly:

msedge.exe --user-data-dir=C:\temp\profile --inprivate --host-rules="MAP * 0.0.0.0" --app=C:\temp\NetLogView.html

The command line arguments bear some explanation. In reverse order:

  • The app argument instructs Edge to open the supplied file with a minimal browser UI, as if it were a native app.
  • The host-rules argument tells the browser instance to direct all network requests to an IP address of 0.0.0.0. On Windows, such requests instantly fail. On Mac/Linux, the null IP points back at your own PC.
  • The inprivate argument directs the browser to discard all storage after the app exits (since it’s not needed). For Chrome, use --incognito instead.
  • The user-data-dir instructs the browser to use a temporary browser profile (which prevents the app’s window from being merged into an existing browser process, such that the host-rules argument would’ve been ignored.)


While none of this is strictly necessary (the NetLog Viewer doesn’t leak data), it’s always nice to be able to discard attack surface wherever possible.

-Eric

Download Blocking by File Type

I’ve previously spoken about the magic of the File Type Policies component — a mechanism that allows files to be classified by their level of “dangerousness”, such that harmless files (e.g. .txt files) can be downloaded freely, whilst potentially-dangerous files (e.g. .dll files) are subjected to a higher degree of vetting and a more security-conscious user-experience.

File Type Danger Level

Microsoft Edge inherits its file type policies from the upstream Chromium browser; you can view the current contents of the list here, and documentation of its format here.

Within the list, you’ll see that each type has a danger_level, which is one of three values: DANGEROUS, NOT_DANGEROUS, or ALLOW_ON_USER_GESTURE.

The first two are simple: NOT_DANGEROUS means Safe to download and open, even if the download was accidental. No additional warnings are necessary. DANGEROUS means Always warn the user that this file may harm their computer. Let users continue or discard the file. If [SmartScreen or Safe Browsing] returns a SAFE verdict, still warn the user before saving the file.

The third setting, ALLOW_ON_USER_GESTURE1 is more subtle. Such files are potentially dangerous, but likely harmless if the user is familiar with download site and if the download was intentional. Microsoft Edge will allow such downloads to proceed automatically if two conditions are both met:

  1. User Gesture: There is a user gesture associated with the network request that initiated the download (e.g. the user clicked a link to the download).
  2. Familiar Initiator: There is a recorded prior visit to the referring origin prior than the most recent midnight (i.e. yesterday or earlier). Such a visit implies that the user has at least some history of visiting the site that kicked off the download.

The download will also proceed automatically if the user explicitly initiated a download by using the Save link as context menu command, entered directly into the browser’s address bar the download’s URL, or if Microsoft Defender SmartScreen (in Edge, or Google Safe Browsing in Chrome) indicates that the file is known safe.

Update: Starting in version 91, Microsoft Edge will join Google Chrome in interrupting downloads that lack the required gesture.

User Experience for Downloads Lacking Gestures

Within Google Chrome, a download lacking a required gesture shows explicit buttons to allow the user to decide whether to proceed with the download or abandon it:

In contrast, Microsoft Edge states that the download “was blocked”, although the same options, titled Keep and Delete, are available from the … menu on the download item.

If you visit edge://downloads, you’ll see the same options:

Enterprise Controls

While users are somewhat unlikely to encounter download interruptions for sites they use every day, they might encounter them for legitimate downloads on sites that they use rarely or in sites that hit “Corner Cases” described in a section below.

To help streamline the user-experience for Enterprises, a Group Policy is available.

Enterprises can use ExemptDomainFileTypePairsFromFileTypeDownloadWarnings to specify the filetypes that are allowed to download from specific sites without interruption.

[{"file_extension":"xml","domains":["contoso.com", "woodgrovebank.com"]},
{"file_extension":"msg", "domains": ["*"]}]

If the SmartScreenForTrustedDownloadsEnabled (or equivalent policy for Chrome) is set to 0 (disabled), and the file download’s URL is Trusted (on Windows, in the Local Machine, Intranet, or Trusted zone) then the download will proceed without interruption (even without a gesture), regardless of danger_level. (Aside: This seems a bit strange, but feels more logical if you pretend that the file type warnings are a part of SmartScreen).

File Types Requiring a Gesture

The latest file types policies are published in the Chromium source code. As of May 2021, file types with a danger_level of ALLOW_ON_USER_GESTURE on at least one OS platform include:
accda, accdb, accde, accdr, action, ad, ade, adp, apk, app, application, appref-ms, as, asp, asx, bas, bash, bat, caction, cdr, cer, chi, chm, cmd, com, command, configprofile, cpgz, cpi, cpl, crt, crx, csh, dart, dc42, deb, definition, der, desktop, dex, diskcopy42, dmg, dmgpart, dvdr, dylib, efi, eml, exe, fon, fxp, hlp, htt, img, imgpart, inf, ins, internetconnect, inx, isp, isu, job, js, jse, ksh, lnk, mad, maf, mag, mam, maq, mar, mas, mat, mau, mav, maw, mda, mdb, mde, mdt, mdw, mdz, mht, mhtml, mmc, mobileconfig, mpkg, msc, msg, msh, msh1, msh1xml, msh2, msh2xml, mshxml, msi, msp, mst, ndif, networkconnect, ocx, ops, out, oxt, paf, partial, pax, pcd, pet, pif, pkg, pl, plg, prf, prg, ps1, ps1xml, ps2, ps2xml, psc1, psc2, pst, pup, py, pyc, pyo, pyw, rb, reg, rels, rgs, rpm, run, scr, sct, search-ms, service, settingcontent-ms, sh, shar, shb, shs, slk, slp, smi, sparsebundle, sparseimage, svg, tcsh, toast, u3p, udif, vb, vbe, vbs, vbscript, vdx, vsd, vsdm, vsdx, vsmacros, vss, vssm, vssx, vst, vstm, vstx, vsw, vsx, vtx, wflow, workflow, ws, wsc, wsf, wsh, xip, xml, xnk, xrm-ms, xsd, xsl

Other Fields in the File Type Policies

  • You’ll also note that some file types have an auto_open_hint which controls whether the user may configure that type of file to open automatically when the download completes.
  • File type settings sometimes vary depending on the client OS platform (an .exe is not dangerous on a Mac, while an .applescript is harmless on Windows). The platform attribute of an entry specifies on which OS the danger_level applies.
  • The max_file_size_to_analyze field controls how big of a file (.zip, .rar, etc) the browser will be willing to unpack to scan it for dangerous content.

Corner Cases

  • If you put referrerpolicy=”no-referrer” on your download link (or otherwise suppress referrers), the Familiar Initiator check fails.
  • If you initiate the download by dynamically creating an A element with a download attribute, then click it from JavaScript, User Gesture check fails.

As of July 2021, Microsoft Outlook Web Access’ email attachment file downloads encounter both of these issues.

Test cases for these conditions can be found here. (Note that you’ll have to have visited webdbg.com yesterday or earlier for the familiarity check to pass).


1 ALLOW_ON_USER_GESTURE_AND_FAMILIAR_INITIATOR would be the accurate name for the setting

Per-Site Permissions in Edge

Last year, I wrote about how the new Microsoft Edge browser mostly ignores Security Zones (except in very rare circumstances) to configure security and permissions decisions. Instead, in Chromium per-site permissions are controlled by settings and policies expressed using a simple syntax with limited wildcarding support.

Settings Page’s Site Permissions and Group Policy

Internet Explorer offered around 88 URLAction permissions, but the majority (62) of these settings have no equivalent; for instance, there are a dozen that control various features of ActiveX controls, a technology that does not exist in the new Edge.

Unfortunately, there’s no document mapping the old URLActions to the new equivalents (if any) available within the new Edge. 

When users open chrome://settings/content/siteDetails?site=https://example.com, they’ll find a long list of configuration switches and lists for various permissions. Users rarely use the Settings Page directly, instead making choices using various widgets and toggles in the Page Info dropdown (which appears when you click the lock) or via various prompts or buttons at the right-edge of the address bar/omnibox.

Enterprises can use Group Policy to provision site lists for individual policies that control the browser’s behavior. To find these policies, simply open the Edge Group Policy documentation and search for ForUrls to find the policies that allow and block behavior based on the loaded site’s URL. I recently wrote a post about Chromium’s URL Filter syntax, which doesn’t always work like one might expect. Most of the relevant settings are listed within the Group Policy for Content Settings.

There are also a number of policies whose names contain Default that control the default behavior for a given setting.

Here’s a list of Site Settings with information about their policies and behavior:

As you can see, some of these settings are very obscure (WebSerial, WebMIDI) while others will almost never be changed away from their defaults (Images).

-Eric

Specifying Per-Site Policy with Chromium’s URL Filter Format

Chromium-based browsers like Microsoft Edge make very limited use of Windows Security Zones. Instead, most permissions and features that offer administrators per-site configuration via policy rely on lists of rules in the URL Filter Format.

Filters are expressed in a syntax that is similar to other matching rules, but different enough to cause confusion. For instance, consider a URLBlocklist rule expressed as follows:

These filters don’t work as expected. The HTTPS rule should not have a trailing * in the path component (it won’t match anything), while the data: rule requires a trailing * to function.

The syntax has a few other oddities as well:

  • A leading dot before the host means exactly match; the filter .example.com matches just https://example.com but not https://www.example.com
  • You may specify a path prefix (example.com/foo) but you must not include a wildcard * anywhere in the path
  • You may specify wildcards in a query (example.com?bar=*); you may omit the path to have the querystring checked on all pages, or include a path to only check the querystring on pages within the path.
  • A rule of blob:* doesn’t seem to match blob URLs, while a rule of data:* does seem to match all data URLs.

Unfortunately, there’s not a great debugger for figuring out the proper syntax. You can use the chrome://policy page to see whether Chrome finds any glaring error in the policy:

…but short of testing your policy there’s not a great way to verify it does what you hope.

Q: The problem of special-URLs

There are a variety of special URLs (particularly blob and data) that do not directly express a hostname– instead, the URL exists within a security context that is not included in the URL itself. This can cause problems for Policies if the code implementing the policy does not check the URL of the security context and looks only at the blob/data URL directly. A system administrator might set a policy for downloads from https://example.com/download, but if download page on that site uses a script-generated file download (e.g. a blob), the policy check might overlook the rule for example.com because it checks just the blob: URL.

An example bug can be found here.

Q: Can filters match on a site’s IP?

The Permissions system’s “Site Lists” feature does not support specifying an IP-range for allow and block lists. Wildcards are not supported.

It does support specification of individual IP literals, but such rules are only respected if the user navigates to the site using said literal (e.g. http://127.0.0.1/). If a hostname is used (http://localhost), the IP Literal rule will not be respected even though the resolved IP of the host matches the filter-listed IP.

Q: Can filters match just dotless hostnames?

Not today, no. You must individually list each desired hostname, e.g. (https://payroll, https://stock, https://who, etc).

Chromium’s URL Filter Format is convenient if your intranet is structured under one private domain (e.g. *.intranet.example.com) but is much less convenient if your Intranet uses dotless hostnames (http://example) or many disjoint private domains.

The ability to match only hostnames not containing dots would be convenient to accommodate the old IE behavior whereby Windows would map dotless hostnames to the Local Intranet Zone by default. (To my surprise, there’s been no significant demand for this capability in the first year of Edge’s existence, so perhaps corporate intranets are no longer using dotless hostnames very much?)

References

Web Proxy Authentication

Last year, I wrote about how the new Microsoft Edge’s adoption of the Chromium stack changed proxy determination away from the Windows Service (WinHTTP Proxy Service) to similar but not identical code in Chromium. This change mostly goes unnoticed, but it can have performance and functionality implications.

In today’s post, I want to explore another side-effect of changing away from the Windows stack– the impact on Proxy Authentication.

Background

When the client is configured to route its traffic through an authenticating proxy server, the proxy responds to any request that does not contain a Proxy-Authorization request header with a HTTP/407 response that demands credentials, specifying the desired authentication scheme using a Proxy-Authenticate header:

The client is expected to resend the request with the requested credentials added in the Proxy-Authorization header.

Browsers tend to support four methods for authentication: the same four used for plain HTTP authentication: Basic, Digest, NTLM, and Negotiate (NTLM+Kerberos). The NTLM and Negotiate schemes are generally much preferred, as they can be configured to authenticate silently without showing a prompt to the user. However, depending on configuration (e.g. not in the right zone, or when the user’s Windows Login Credentials are not the ones the proxy requires), the user might see a proxy-authentication prompt in the browser.

Browser Credential UX

The prompt differs depending on the client; Google Chrome, and until recently Microsoft Edge, show an auth “bubble” anchored to the bottom of the browser address bar UI. In contrast, the latest versions of Edge and Internet Explorer show the battleship gray Windows Hello modal dialog box:

While these experiences are all conceptually similar, they differ in very important ways. First, note that Internet Explorer’s option shows a “Remember my credentials” checkbox while Chrome/Edge do not. Proxy credentials saved by Internet Explorer will be reused without prompting, across browsing sessions and across other applications that use the WinHTTP Proxy Service.

In contrast, in Chrome and older Edge, the proxy credentials prompt is integrated with the browser’s Password Manager. If the user accepts the followup prompt to save the proxy credentials, those credentials will autofill the dialog in future browser sessions. Notably, however, the proxy authentication is not automatic– the user must manually click the Sign in button once per browser session.

Storing a credential in your Edge profile’s password manager only applies to that one browser profile– other profiles and apps do not have access to the credential.

In Edge 88+, this password manager integration was lost in the switch to use the Windows Hello prompt for NTLM/Negotiate. A new policy (WindowsHelloForHTTPAuthEnabled) added to Edge 90 changes Edge’s proxy credentials dialog back to the older dialog.

Other App Credential UX

Support for manually-authenticating proxies outside of browsers has always been terrible, and many Win8 applications didn’t even bother to handle it. Sadly, that shortcoming remains in Windows 10, where you can’t even use the Windows Store when a proxy needs credentials that are not supplied automatically. An generic error page is shown:

However, as mentioned earlier, around IE11 a change was made to push proxy authentication creds entered in Internet Explorer’s proxy authentication prompt into the Windows Credential Manager (CredMan):

As seen in the Control Panel view (Persistence: Enterprise):

Even if you untick the “Remember my credentials” box on the prompt, the entered creds are still remembered for the Windows session (Persistence: Login Session):

As a consequence, users can “unbreak” various Windows scenarios by just using the browser to populate the Credential Manager with the proxy credentials.

We’ve lost that workaround in the new world, because Chromium’s password manager is not integrated with CredMan (for a number of sane1 reasons).

Surface Hub

The Microsoft Surface Hub is a “whiteboard computing” device designed for corporate conference rooms and the like. By default, the Windows logon session on the device uses an account named ComputerName\SurfaceHub, and that account is very unlikely to be supported by an authenticating proxy. To support it, Windows’ Settings applet on Surface Hub has a custom option:

However, this option doesn’t presently do anything for Chromium’s network stack. When Edge gets a Proxy-Authorization: Negotiate challenge from the proxy, it will respond with the credentials for the SurfaceHub account rather than the machine credentials, and the login will likely fail. At present, we don’t have a great workaround for this: it’s probably possible to write a browser extension (catching the onAuthRequired event and responding with non-default credentials) but this won’t pick up the Windows settings.

Other Brokenness

  • We recently fixed a bug in Edge Canary where the browser would crash on startup if the user was behind a manually-authenticated proxy server. A network request triggered very early in browser startup wasn’t prepared for an auth challenge.
  • If you have the Microsoft Office Chrome extension installed, it can (for reasons unknown) automatically cancel the manual proxy authentication dialog before the user has a chance to submit credentials.

In conclusion: Two decades in, support for proxy authentication remains a mess. The feature is required by the networks of some corporate titans (and thus support continues to limp along), but many corner cases remain broken. Consider carefully whether proxy authentication needs to be a part of your network strategy.

-Eric

1 There are several reasons, but one obvious one is that Chromium uses a “profiles” system whereby one Windows logon account may have multiple browser profiles, each of which is independent and does not share data with the others. This precludes direct use of system-level features that do not offer such partitioning.

window.close() Restrictions

Sometimes, Web Developers are surprised to find that the window.close() API doesn’t always close the browser window.

When looking at the Developer Tools console, they’ll see a message like:

Scripts may close only the windows that were opened by them.

Why Do Browsers Limit close()?

Before we dive into what factors govern what happens when close() is called, it’s important to understand why there’s a restriction at all.

While sometimes explained away as nebulous “Security reasons!“, the core reason is more aligned with simple user-experience— users could lose important state in their browser window or “back/forward” stack (in IE, we called this the TravelLog) if a tab or window went away unexpectedly. The user might’ve been using a tab to explore a set of search results, and if one of those results could blow away both itself and the backstack with the search results page, this would be pretty annoying.

There’s a minor anti-abuse/security argument as well– if a browser tab could blow itself away freely, this might be useful as a part of scareware or other user-experience abuses.

What Does the Standard Say?

Here’s what the dom-window-close section of the HTML Standard has to say:

A browsing context is script-closable if it is an auxiliary browsing context that was created by a script (as opposed to by an action of the user), or if it is a top-level browsing context whose session history contains only one Document.

This seems simple enough, although the parts I’ve bolded hide a lot of complexity and nuance. (Most obviously, “what should we do if the script was run in response to an action of the user?“)

What Do Browsers Do?

Unfortunately for us, each browser has a different set of behaviors (explore with this test page), partly because most were implemented before the standard was written.

Internet Explorer

In Internet Explorer, a tab/window will be closed silently if it was created as a result of a JavaScript window.open() call. There’s no attempt to check whether the back/forward stack contains only one document: a tab with a large TravelLog will still close silently if it was opened by script. (IE also allows HTA documents to close themselves without restriction.)

In all other circumstances, the tab/window will not silently close: instead, the user is presented with a one of two modal dialogs, depending on whether the page represents the only tab in the browser window:

Chromium (Microsoft Edge / Google Chrome / etc)

As of Chromium 88, window.close() succeeds if the new window/tab has an opener or if the back/forward stack contains fewer than two entries.

As you can see, there are subtle differences here between what the spec requires and what the browser implements.

First, notice that I said “has an opener” rather than “was created by a script.” Recall that the opener property allows a popup window to refer to the tab that created it.

  • If the user creates a new tab by clicking the frame button, hitting Ctrl+T, Shift+Clicking a link, or launching a URL from the Shell, the resulting new tab doesn’t have an opener set.
  • In contrast, if the tab is opened via open() or hyperlink with a named target (not _blank), then by default, the opener property is set.
  • Any hyperlink may include rel=opener or rel=noopener to specify whether the new tab has an opener set.
  • A open() JavaScript call can specify noopener in its windowFeatures string to set the new tab’s opener to null.

As you can see from the above list, both normal link clicks and JavaScript open() invocations can result in tabs with or without the opener set. This can be very confusing: Shift+click on a link might result in a tab that cannot self-close, while Left-click on that same link results in a tab that can always close itself.

Secondly, notice I said “entries” rather than “Documents.” In most cases, these are equivalent, but they’re not exactly the same. Consider the case where the new tab navigates to a HTML document with a Table of Contents at the top. The user clicks on a ToC link to #Section3 and the browser dutifully scrolls down to that section. The back/forward stack now contains two entries, both pointing to the same document. Chromium blocks window.close(), but it shouldn’t. This longstanding shortcoming was made more visible in Chromium 88, which now gives links targeting _blank the noopener behavior by default.

crbug.com/1170131 tracks fixing this issue by counting the number of Documents in the back/forward stack, but it will be tricky to fix because presently the renderer process running JavaScript has access only to the count of entries in the back/forward stack, not their URLs.

Chromium: User-Experience

When Chrome blocks close(), it emits a notice to the console:

Scripts may close only the windows that were opened by them.

…but there’s no notice to the user, who may be left confused if they clicked on a “Close” button or link inside the page. Recently filed crbug.com/1170034 suggests introducing a dialog box like Internet Explorer’s. (As an aside, it also sets a new standard for bug filing by including comic-book style art showing unhappy users converting to happy users if the proposed feature is landed. :)

Chromium: Exotic Bug Trivia

This is a super-obscure corner case. However, I’ve seen it independently reported against both Chrome and Edge in the span of five years, and it makes me laugh.

If you set Chromium’s On Startup option to “Continue where you left off”, navigate to a page that tries to close itself, and then close the window, the browser will subsequently kill itself on every startup.

It’s hard to get into this state, but it’s still possible in Chrome/Edge 90.

Repro steps: Visit https://webdbg.com/test/opener/. Click the ‘Page that tries to close itself‘ link. Hit Ctrl+Shift+Delete and Clear Browsing History (to clear the back/forward stack). Close the browser with the ‘X’ icon. Now, try to launch the browser from your start menu. Giggle wildly as your browser launches and then blows itself away.

Safari/WebKit

WebKit’s code is similar to Chromium’s (unsurprising, considering their shared lineage) except it does not equate navigations triggered with noopener with those created by browser UI. Thus, in Safari, the user may click between many different same-origin pages and close() is still permitted.

If close() is blocked, Safari’s (well-hidden) JavaScript console shows:

Can't close the window since it was not opened by JavaScript

Firefox

Unlike Chromium, Firefox correctly implements the “only one Document” part of the spec. Firefox calls IsOnlyTopLevelDocumentInSHistory which calls IsEmptyOrHasEntriesForSingleTopLevelPage() which enumerates the history list to check. If there is more than one entry, whether all of the entries are for the same Document. If they are all for the same document, the close() succeeds.

Firefox offers an about:config override setting named dom.allow_scripts_to_close_windows that bypasses the default restrictions.

When Firefox blocks close(), it emits a notice to the console:

Scripts may not close windows that were not opened by script.

There’s an 18 year-old feature request for Firefox to show a confirmation dialog instead of failing silently.

In Conclusion

umm…. browsers are wicked complicated?

-Eric

Sandboxing vs. Elevated Browsing (As Administrator)

The Web Browser is the most security-critical application on most users’ systems– it accepts untrusted input from servers anywhere in the world, parses that input using dozens to hundreds of parsers, and renders the result locally as fast as it can. For performance reasons, almost all code in almost all browsers is written in memory-unsafe languages.

To mitigate this insanity, web browsers perform this chainsaw juggling act inside nested layers of sandboxing. When an attacker inevitably manages to find an exploit, the resulting damage might be contained by the sandbox. Chromium calls this “The Rule of 2”:

alt text

As a consequence, most real-world browser attacks now must include exploits for two vulnerabilities: the original RCE, and a Sandbox Escape.

A web browser uses multiple mechanisms for sandboxing, from “Design sandboxing” (Same-Origin-Policy, limited JavaScript capabilities) to Process Sandboxing (AppContainers, UAC Integrity Levels, Restricted Tokens, etc) to optional VM sandboxing (WDAG).

Process sandboxing arose in the mid-200Xs as a defense-in-depth against Design Sandboxing mistakes – any error in implementation meant that an attacker could achieve arbitrary code execution (RCE) and take over the victim user’s computer.

Windows Vista introduced the notion of UAC Integrity Levels, whereby each system object (including processes) had an integrity level that limited the other objects it could read and write. Internet Explorer 7 built atop the UAC Integrity Levels system and introduced the notion of “Protected Mode” whereby Internet Zone content ran inside a tighter sandbox with decreased write-access to the system. Google Chrome and later versions of IE went much further, with AppContainers and broader process restrictions to make exploitation more challenging and reduce its impact.

Tools like SysInternals’ Process Explorer allow inspecting some of the sandboxing limitations applied to browsers’ processes. For instance, you can see the Windows Job restrictions applied to this Edge Renderer Process:

You’ll note that browsers typically use multiple processes with different sandboxing restrictions attached to each. For instance, in a default launch of Chrome, you’ll see that the main browser process launches at the default (“Medium”) Integrity Level, while the GPU process runs at Low Integrity, and the Renderer processes run at Untrusted Integrity (the lowest):

Looking at Internet Explorer, you can see that its main process runs at Medium, an Intranet Zone tab process runs at Medium, and an Internet Zone tab process runs at Low:

If you enable Enhanced Protected Mode, that Internet Zone tab process is further restricted to run in an AppContainer:

Run As Administrator

One interesting difference between Legacy IE and Microsoft Edge/Chrome is the behavior when you “Run As Administrator”: In the old IE, this resulted in the tab processes running unsandboxed at High IL:

For reasons that are hopefully obvious, running web content outside of a sandbox at High IL is an extremely dangerous configuration.

In contrast to IE behavior, in Edge/Chrome, the browser process, crash handler, and some utility processes run at High IL, but the sandboxed Renderers stay in Untrusted IL processes.

While this is clearly a safer configuration that what Internet Explorer had, the transitive nature of UAC Elevation means that many executables launched by this browser are launched at High Integrity, including Native Messaging Hosts, AppProtocol Handlers, downloaded executables, etc.

Functional Bugs

Beyond the increased security attack surface, there are some functional problems with running a Chromium-based browser at High IL. For instance, if you have AppLocker enabled, you may find that all of your sandboxed renderer processes crash (crbug.com/740132), or that link launches from non-admin processes are ignored (crbug.com/801539#c9). 

As a result of problems like these, Microsoft Edge will attempt to silently deelevate and relaunch itself when launched as an Administrator, running at Medium IL instead of High. If this feature is disabled (--do-not-de-elevate), Edge will show a warning to help users recognize the unexpected situation:

Administrator Mode Detected balloon in Edge

Generally speaking, there’s no legitimate reason to run your browser at High IL, and you shouldn’t ever do it.

But…

Impact on Windows Integrated Authentication

Recently, I was disappointed to learn that there are some scenarios where running at HighIL is still necessary. These scenarios are based on Windows Integrated Authentication, whereby the web server uses passthrough authentication to pass the user’s Elevated Authentication Token through to some backend service that requires membership in the Administrators group. If the web server receives and passes along the admin user’s non-elevated token (because the browser was run at Medium), the backend services will refuse to undertake the privileged operation.

The fact that such an application architecture ever worked is interesting but a bit bananas.

I’d be much happier with a design whereby this Elevated token passing scenario didn’t have to happen in the browser— either configuration should happen in some non-browser application, or the Service Account could invoke some native UI to collect the admin’s elevated credentials during the specific configuration operation where such credentials are needed. From a layering POV, I’m sure this would be complicated, because the ability of a background service to pop stuff on the admin user’s desktop is intentionally limited.

If you’re depending upon an architecture like this, please stop. Pretty please! It’s fragile: it behaves differently based on the user’s default browser, behaves differently depending on what browser windows the user already has open (launch msedge.exe as Admin doesn’t result in an Elevated browser if Edge was already open), and exposes the user to a greater attack surface if they use the Elevated browser for anything else after finishing with your scenario.

Stay safe (and de-elevated) out there!

-Eric

Objectively, the best cat

On March 15, 2009 we put my cat Jill (Jillian, Jilly, Jilly Bean, Jillkin) to sleep at an emergency vet. We didn’t know for sure why her kidneys failed, but it was sudden and unexpected. At times Jill could be grumpy and stubborn, but she was usually friendly, curious, and she was always very loved. I lamented at the time that it was “never going to be time to feed the monster again,” and we would always miss her. I didn’t expect to have another pet for a long time.

But then life intervened.

A Friday night two weeks later, my fiancĂ©e and I were waiting for a movie to start at the Regal theater next to the Crossroads mall in Bellevue. We saw that the Petco store adjacent was hosting some cats for adoption. We had no plans to get a new pet, especially not so soon, but we took a look anyway. I liked the look of a big cat (“Yoda”) snoring lazily in his cage, but Jane preferred a tiny black cat with a shaved belly and a curious face. After looking for a few minutes, we left for our movie.

In our seats, waiting for the previews to end, Jane remarked “She’s black and white. We could call her Domino.” and I spent the rest of subsequent movie impatiently waiting for it to end: the name was so perfect I couldn’t imagine living in a world where it wasn’t hers and she ours. Unfortunately, the pet store closed before our movie ended, but we were back bright and early the next day to adopt her.

A street kitten, under-weight, under-furred and under-age (she was allegedly a year old but plainly was only a few months) she took to her new home immediately; her fur grew in and she eventually stopped chasing her own tail.

Domino was playful and alert, and always pushing boundaries. We let her out a few times, but she kept getting hurt, one time returning with teeth marks on both sides of her little ribs suggesting that a dog had gotten its jaws around her entire body. After that, she was confined to our new deck– as soon as she stepped off, we’d pick her up and put her back on it. She clued in and began keeping a single paw arched back to the bottom step of the deck, her other three triumphantly planted in the grass. It was soon a common sight, one so comical that it was ingrained in my memory to the point that I never bothered to take a picture. I’d’ve never believed if I hadn’t seen it.

Unlike many cats who are standoffish or aloof, Domino was empathic– if we ever cried or even sounded sad, she’d immediately appear and start rubbing up against us, a furry friend without advice, just love. We bought her a fancy bed, but she never wanted to sleep in it– she always snuggled up to us instead.

When we moved from Seattle to Austin in 2012, we decided to drive rather than fly to make the trip easier on her– drugging her for a flight seemed cruel and unnecessary. A day into the trip, we realized what an error we’d made… she spent the first two days of the trip hiding in the back of the car, climbing up to the dashboard to peer out, then, terrified at America whizzing by at 75mph, jumping into the back or a lap to lay back down. This process repeated every few minutes until she’d fall asleep for short naps.

By the third day, she was plainly miserable; when we finally rolled into our new driveway, the first order of business was to let her out to explore her new back yard. Her backyard explorations were rare after that– we’d seen coyotes and snakes in the neighborhood, and between those threats and the blazing Texas summers, she was happy enough as an indoor cat. An occasional foray into the backyard to nibble some grass or roll around on the patio seemed to keep her happy.

We grew up together.

In 2013, we nervously brought our first son home, worried about how Domino might react. She kept her distance. In fact, if anything, she seemed terrified, avoiding both his crib and her food bowl. Her odd behavior went on for a few days until I finally figured out what had happened.

Tied to newborn Noah’s car seat floated a congratulatory Mylar balloon, and having never seen one, Domino was terrified of it. I foolishly made the mistake of trying to carry her over to it to show her that it was nothing to be afraid of. She clawed her way out of my arms, over my shoulder, and down my back, peeing the whole way. Later that afternoon, Noah peed all over me as I changed his diaper, and I joked that it was “pee on Dad day.”

After we got rid of the balloon, Domino was the perfect big sister, tolerating a toddler who would occasionally pull her tail or chase her around, never lifting a paw against him no matter how rough he wanted to play. Her patience continued with our second son, and it paid off– as they grew up, the boys were always happy to snuggle, pet her, and build increasingly elaborate cat forts out of whatever cardboard boxes they could get their hands on. She humored them and snuggled them every chance she got.

But she snuggled with all of us, including sleepy mom and dad.

When my wife asked me to move out a year ago, there wasn’t much debate — our cat was coming with me. Domino kept me company through the roughest pandemic year alone in a new home. Two or three other cats in my new neighborhood roamed free and Domino noisily defended her turf through the windows. As the weather cooled down, she started keeping me company on the porch, and then demanding to go out between dinner and bedtime. A month ago, she went out late and didn’t come back until first light, banging on my bedroom window and demanding breakfast.

So I didn’t think too much about it as I acceded to her demands and let her out the back door at around 11pm the night of November 9th, 2020. “Be good. Don’t get in trouble!” I admonished as always. At 12:30am, I checked for her, but she wasn’t around, so I reluctantly went to bed.

I woke up a few times in the night and checked for her, but she wasn’t around. When I woke up at 8am, I was officially worried. By the afternoon, I was papering the neighborhood with posters, updating the contact info for her microchip, and learning about all of the web resources for reuniting lost pets (some of which are pretty cool).

Going to bed alone that night with her still at large was hard. And it hasn’t gotten any easier over the last week, no matter how many posters I’ve put up, or web postings I’ve scoured.

o

Sounds like a great cat, huh? Well, you haven’t heard the real story yet.

Back when Noah was four, his right knee started swelling. We assumed it was bruising from a fall, but he didn’t complain about pain and the swelling didn’t go down in weeks. A series of doctors’ visits, scans, and a fully-anesthetized MRI led to a diagnosis of PVNS, a non-cancerous tumor that nonetheless would require major surgery on both the front and back of Noah’s knee. The expert doctor at Dell Children’s Hospital assured us that it was as clear a case as he’d ever seen, and we needed to do the surgery soon to ensure that Noah’s leg would grow normally. I pushed for doing the operation right away, but Jane suggested we do it later in the fall so that Noah would be able to recover without the temptations of the swimming pool and other summer fun.

Around this time we noticed Domino had developed a large growth (about the size of a large marble) in her neck, a recurrence of a problem (keratin buildup?) she’d had previously. We hoped she’d get better on her own again this time, as we had bigger worries now.

Throughout August and September, Noah would watch cartoons on the couch, and Domino would lay her swollen chin on his swollen knee. When it came time to go back to the hospital for Noah’s operation, the doctor was gobsmacked– the mass was completely gone. He had no explanation– he’d never seen anything like it. The doctor asked us to do an MRI in an attempt to figure out what had happened, but after confirming that Noah wouldn’t need surgery and an MRI wouldn’t change anything for his diagnosis, we declined. We were overwhelmed with relief. Domino’s growth disappeared too.

I’m not saying my cat cured my son (and vice-versa), but our best doctors don’t seem to be in a position to say that’s not what happened either.

I miss my friend.

Simply Making Simple Fixes Simple for Chromium

Google recently introduced a cool web-based editing tool for Chromium source code, a very stripped down version of the Willy Wonka tooling Googlers get to use for non-Chromium projects.

I’ve used this tool to submit two trivial change lists (CLs, aka PRs) to Chromium, but I was curious about whether this new feature would work for something not completely trivial, and while remaining simpler than the full Checklist for contributing to Chromium.

Let’s try it.

First, find an Available GoodFirstBug. Say, this one to remove an expired flag. Let’s search for the flag. Not too many hits:

Let’s click into the first hit to open it. Click Edit Code at the top right:

A web-based code editor opens:

Remove the &kOverlayNewLayout line and the function that references it later in the file. Use the navigation panel on the left to open the .h file corresponding and remove the declaration of the feature.

Open about_flags.cc and remove the lines referencing the flag:

Note that this will create orphan variables for the flag_descriptions that we need to go delete too. Go delete those from the flag_descriptions.cc and flag_descriptions.h files.

At this point, your Pending Changes pane contains all of the files with all of the hits that came up in the search.

If we think we’re done, we can go hit the Create Change button to actually create the change for review.

However, suspiciously, we haven’t actually found anything that respects the Feature we removed. If we instead search for OverlayNewLayout we see an additional hit appears:

Having worked on stuff like this before, I know that there’s some magical marshalling of feature values from C++ into Java-land (Android’s UI layer). So we need to go eliminate the use of the Java-side feature. Search for OVERLAY_NEW_LAYOUT.

Oof. There are 8 more files: 7 are Java and one is a similarly-named variable in a CC file. (It’s a variable in C++ code that set based on the feature’s state from the Java-side based on a resource ID. Sneaky sneaky).

Worse, one of the early hits is this one:

… So we also need to remove this function entirely and update all of its callers to not check it. Fortunately, only two more files have references, but clearing up some of those implies that we might have resources that will become unused:

When we’re done pulling all the threads, the change has grown pretty big. Click Create change to put all of our changes together into a change list:

In the dialog that appears, enter a description of the changelist and click Create:

After a minute or so, a notice will appear:

Sadly, it’s not a hyperlink, but if you look down at the bottom of the editor, you’ll find a direct link to the Chromium Code Review site, where you can find the created change. After the CL is prepared, you can add reviewers for a code review, ask anyone with Try Permissions to run your change through the bots, and otherwise follow the process for contributing to Chromium.

If you need to modify a change after you’ve created a CL, you can do so using the Edit link on the Code Review Site. Push the button:

… then use the new commands down near the file list to modify (or add) files in the CL:

A few observations from this process:

1. A lot of Chromium GoodFirstBug‘s are much more complicated than they appear. This one required a followup CL to remove 400 more lines!

2. This new web-editing workflow is great for trivial fixes, but very limited. Most importantly, if you need to use any advanced tools (e.g. source formatting, manual code generators, presubmit checks, etc), you’ll need to download your CL from the code review site to use the tools on it.

3. Multi-file changes are challenging. It would be awesome to be able to just click “Edit Code” from the Chrome Source Viewer on multiple files and have them all open in the same “in-progress” change set, but it doesn’t seem to work like that (yet?). You must manually find all files after the first one in the editor’s sidebar.

I’m excited to see how Chromium’s Edit Code feature evolves.

-Eric