Edge

As we rebuild Microsoft Edge atop the Chromium open-source platform, we are working through various scenarios that behave differently in the new browser. In most cases, such scenarios also worked differently between 2018’s Edge (aka “Spartan”) and Chrome, but users either weren’t aware of the difference (because they used Trident-derived browsers inside their enterprise) or were aware and simply switched to a Microsoft-browser for certain tasks.

One example of a behavioral gap is related to running ClickOnce apps. ClickOnce is a Microsoft application deployment framework that aims to allow installation of native-code applications from the web in (around) one click.

Chrome and Firefox can successfully install and launch ClickOnce’s .application files if the .application file specifies a deploymentProvider element with a codebase attribute (example):

InstallPrompt

Installation prompt when opening an .application file.

However, it’s also possible to author and deploy an .application that doesn’t specify a deploymentProvider element (example). Such files launch correctly from Internet Explorer and pre-Chromium Edge, but fail in Firefox and Chrome with an error message:

ApplicationCannotBeStarted

ClickOnce fails for a downloaded .application file.

So, what gives? Why does this scenario magically work in Edge Spartan but not Firefox or Chrome?

The secret can be found in the EditFlags for the Application.Manifest ProgId (to which the .application filename extension and application/x-ms-application MIME type are mapped):

ApplicationManifestRegistry

Registry settings for the Application.Manifest ProgId.

The EditFlags contain the FTA_AlwaysUseDirectInvoke flag, which is documented on MSDN as 

FTA_AlwaysUseDirectInvoke 0x00400000
Introduced in Windows 8. Ensures that the verbs for the file type are invoked with a URL instead of a downloaded version of the file. Use this flag only if you’ve registered the file type’s verb to support DirectInvoke through the SupportedProtocols or UseUrl registration.

If you peek in the Application.Manifest’s Shell\Open\Command value, you’ll find that it calls for running the ShOpenVerbApplication function inside dfshim.dll, passing along the .application file’s path or URL in a parameter (%1):

“C:\Windows\System32\rundll32.exe” “C:\Windows\System32\dfshim.dll”,ShOpenVerbApplication %1

And therein lies the source of the behavioral difference.

When you download and open an Application.Manifest file from Edge Spartan, it passes the source URL for the .application to the handler. When you download the file in Firefox or Chrome, it passes the local file path of the downloaded .application file. With only the local file path, the ShOpenVerbApplication function doesn’t know how to resolve the relative references in the Application Manifest’s XML and the function bails out with the Cannot Start Application error message.

Setting FTA_AlwaysUseDirectInvoke also has the side-effect of removing the “Save” button from Edge’s download manager:

NoSave

…helping prevent the user from accidentally downloading an .application file that won’t work if opened outside of the browser from the Downloads folder (since the file’s original URL isn’t readily available to Windows Explorer).

Advice to Publishers

If you’re planning to distribute your ClickOnce application from a website, specify the URL in Visual Studio’s ClickOnce Publish Wizard:

Manifest

Specify “From a Web site” in the ClickOnce Publish Wizard.

This will ensure that even if DirectInvoke isn’t used, ShOpenVerbApplication can still find the files needed to install your application.

Workarounds

A company called Meta4 offers a Chrome browser extension that aims to add fuller support for ClickOnce to Chrome. The extension comes in two pieces– a traditional JavaScript extension and a trivial “native” executable (written in C#) that simply invokes the ShOpenVerbApplication call with the URL. The JavaScript extension launches and communicates with the native executable running outside of the Chrome sandbox using Native Messaging.

Unfortunately, the extension is a bit hacky– it installs a blocking onBeforeRequest handler which watches all requests (not just downloads), and if the target URL’s path component ends in .application, it invokes the native executable. Alas, it’s not really safe to make any assumptions about extensions in URLs (the web is based on MIME types, rather than filenames).

Next Steps

For the Edge team– TBD.

Do you use ClickOnce to deploy your applications? If so, are you specifying the deployment URL in the manifest file?

-Eric

PS: Notably, Internet Explorer doesn’t rely upon the DirectInvoke mechanism; removing the EditFlags value entirely causes IE to show an additional prompt but the install still succeeds. That’s because IE activates the file using a MIME handler (see the CLSID subkey of Application.Manifest) much like it does for .ZIP files. The DirectInvoke mechanism was invented, in part, to replace the legacy MIME handler mechanism.

This issue report complains that Edge doesn’t stream AAC files and instead tries to download them. It notes that, in contrast, URLs that point to MP3s result in a simple audio player loading inside the browser.

Edge has always supported AAC so what’s going on?

The issue here isn’t about AAC, per-se; it’s instead about whether or not the browser, upon direct navigation to an audio stream, will accommodate that by generating a wrapper HTML page with an <audio> element pointed at that audio stream URL.

PlaceholderPage

A site that wants to play streaming AAC in Edge (or, frankly, any media type, for any browser) should consider creating a HTML page with an appropriate Audio or Video element pointed at the stream.

The list of audio types for which Edge will automatically generate a wrapper page does not include AAC:

audio/mp4, audio/x-m4a, audio/mp3, audio/x-mp3, audio/mpeg,
audio/mpeg3, audio/x-mpeg, audio/wav, audio/wave, audio/x-wav,
audio/vnd.wave, audio/3gpp, audio/3gpp2

In contrast, Chrome creates the MediaDocument page for a broader set of known audio types:

static const char* const kStandardAudioTypes[] = {
 "audio/aac",  "audio/aiff", "audio/amr",  "audio/basic",  "audio/flac",
 "audio/midi",  "audio/mp3",  "audio/mp4",  "audio/mpeg",  "audio/mpeg3", 
 "audio/ogg", "audio/vorbis",  "audio/wav",  "audio/webm",  "audio/x-m4a",
 "audio/x-ms-wma",  "audio/vnd.rn-realaudio",  "audio/vnd.wave"};

If the the response sends Content-Type: application/octet-stream, includes a Content-Dispostion: attachment, or puts a download attribute on the anchor <a> element that leads to the media, Edge will download the media file instead of playing it in the browser.

Note: In Windows 10 RS5, the extension model is capable enough that it’s possible to write a browser extension that intercepts navigation directly to audio/video Media types and renavigates to a wrapper page. [Sample code]

-Eric

PS: Edge has similar special handling for video types:

"application/mp4","video/mp4","video/x-m4v","video/3gpp",
"video/3gpp2","video/quicktime"

 

Update: The October 2018 Cumulative Security Update (KB4462919) brings the RS5 Cookie Control changes described below to Windows 10 RS2, RS3, and RS4.

Cookies are one of the most crucial features in the web platform, and large swaths of the web don’t work properly without them. Unfortunately, cookies are also one of the primary mechanisms that trackers and ad networks utilize to follow users around the web, potentially impacting users’ privacy. To that end, browsers have offered cookie controls for over twenty years.

Back in 2010, I wrote a summary of Internet Explorer’s Cookie Controls. IE’s cookie controls were very granular and quite powerful. The basic settings were augmented with P3P, a once-promising feature that allowed sites to advertise their privacy practices and browsers to automatically enforce users’ preferences against cookies. Unfortunately, major sites created fraudulent P3P statements, regulators failed to act, and the entire (complicated) system collapsed. P3P was removed from IE11 on Windows 10 and never implemented in Microsoft Edge.

Instead, Edge offers a very simple cookie control in the Privacy and Security section of the settings. Under the Cookies option, you have three choices: Don’t block cookies (the default), Block all cookies, and Block only third party cookies:

CookieSetting

This simple setting hides a bunch of subtlety that this post will explore.

Cookie => Cookie-Like

For the October 2018 update (aka “Redstone Five” aka “RS5”) we’ve made some important changes to Edge’s Cookie control.

The biggest of the changes is that Edge now matches other browsers, and uses the cookie controls to restrict cookie-like storage mechanisms, including localStoragesessionStorageindexedDB, Cache API, and ServiceWorkers. Each of these features can behave much like a cookie, with a similar potential impact on users’ privacy.

While we didn’t change the UI, it would be accurate to change it to:

CookieLike

This change improves privacy and can even improve site compatibility. During our testing, we were surprised to discover that some website flows fail if the browser blocks only 3rd party cookies without also blocking 3rd-party localStorage. This change brings Edge in line with other browsers with minor exceptions. For example, in Firefox 62, when 3rd-party site data is blocked, sessionStorage is still permitted in a 3rd-party context. In Edge RS5 and Chrome, 3rd party sessionStorage is blocked if the user blocks 3rd-party cookies.

Block Setting and Sending

Another subtlety exists because of the ambiguous terminology “third-party cookie.” A cookie is just a cookie– it belongs to a site (eTLD+1). Where the “party” comes into play is the context where the cookie was set and when it is sent.

In the web platform, unless a browser implements restrictions:

  • A cookie set in a first-party context will be sent to a first-party context
  • A cookie set in a first-party context will be sent to a third-party context
  • A cookie set in a third-party context will be sent to a first party context
  • A cookie set in a third-party context will be sent to a third-party context

For instance, in this sample page, if the IFRAME and IMG both set a cookie, these cookies are set in a third-party context:Contexts

  • If the user subsequently visits domain2.com, the cookie set by that 3rd-Party IFRAME will now be sent to the domain2.com server in a 1st-Party context.
  • If the user subsequently visits domain3.com, the cookie set by that 3rd-Party IMG will now be sent to the domain3.com server in a 1st-Party context.

Historically, Edge and IE’s “Block 3rd party cookies” options controlled only whether a cookie could be set from a 3rd party context, but did not impact whether a cookie initially set in a 1st party context would be sent to a 3rd party context.

As of Edge RS5, setting “Block only 3rd party cookies” will now also block cookies that were set in a 1st party context from being sent in a 3rd-party context. This change is in line with the behavior of other browsers.

Edge Controls Impacted By Zones

With the move from Internet Explorer to Edge, the Windows Security Zones architecture was largely left by the wayside.

Zones

However, cookie controls are one of a small number of exceptions to this; Edge applies the cookie restrictions only in the Internet Zone, the zone almost all sites fall into (outside of users on corporate networks).

Perhaps surprisingly, cookie-like features and the document.cookie getter are restricted, even in the Intranet and Trusted zones.

Chrome and Firefox do not take Windows Security Zones into account when applying cookie policies.

Test Cases

I’ve updated my old “Cookies” test page with new storage test cases. You can set your browser’s privacy controls:

Block3rdPartyChrome

Block3rdPartyFF

…then visit the test page to see how the browser limits features from 3rd-party contexts. You can use the Swap button on the page to swap 1st-party and 3rd-party contexts to see how restrictions have been applied. You should see that the latest versions of Chrome, Firefox, and Edge all behave pretty much the same way.

One interesting exception is that when configured to Block 3rd-party Cookies, Edge still allows 3rd-party contexts to delete their own cookies. (This is used by federated logout pages, for instance). Chrome does not allow deletion in this scenario– the attempt to delete cookies is ignored.

 

-Eric


Appendix: Chromium Audit

In the course of our site-compatibility investigations, I had a look at Chromium’s behavior with regard to their cookie controls. In Chromium, Blink asks the host application for permission to use various storages, and these chokepoints check:

cookie_settings_->IsCookieAccessAllowed(origin_url, top_origin_url);

…which is sensitive to the various “Block Cookies” settings.

Mojo messages come up through renderer_host/chrome_render_message_filter.cc, gating access to

Additionally, ChromeContentBrowserClient gates

Elsewhere, IsCookieAccessAllowed is used to limit:

  • Flash Storage (PP_FLASHLSORESTRICTIONS_BLOCK)
  • Client Hints

Of these, Edge does not support WebSQL, FileSystem, SharedWorker, or Client Hints.

In yesterday’s episode, I shared the root cause of a bug that can cause document.cookie to incorrectly return an empty string if the cookie is over 1kb and the cookie grows in the middle of a DOM document.cookie getter operation.

Unfortunately, that simple bug wasn’t the root cause of the compatibility problem that I was investigating when my code-review uncovered it. The observed compatibility bug was slightly different– in the repro case, only one of the document’s cookies goes missing, and it goes missing even when only one page is setting the cookie.

After the brain-melting exercise of annotating the site’s minified framework libraries (console.log(‘…’) ftw!) via Fiddler’s AutoResponder, I found that the site uses the document.cookie API to save the same cookie (named “ld“) three times in a row, adding some information to the cookie each time. However, the ld cookie mysteriously disappears between 0.4 and 6 milliseconds after it gets set the third time. I painstakingly verified that the cookie wasn’t getting manipulated from any other context when it disappeared.

Hmm…

As I wrote up the investigation notes, I idly noted that due to a trivial typo in the website’s source code, the ld cookie was set first as a Persistent cookie, then (accidentally) as a Session cookie, then as a Persistent cookie.

In re-reading the notes an hour later, again my memory got tickled. Hadn’t I seen something like this before?

Indeed, I had. Just about five years ago, a user reported a similar bug where a HTTP response contained two Set-Cookie calls for the same cookie name and Internet Explorer didn’t store either cookie. I built a reduced test case and reported it to the engineering team.

Pushing Cookies

The root cause of the cookie disappearance relates to the Internet Explorer and Edge “loosely-coupled architecture.”

In IE and Edge, each browser tab process runs its own networking stack, in-process1. For persistent cookies, this poses no problem, because every browser process hits the same WinINET cookie storage area and gets back the latest value of the persistent cookie. In contrast, for session cookies, there’s a challenge. Session cookies are stored in local (per-process) variables in the networking code, but a browser session may include multiple tab processes. A Session cookie set in a tab process needs to be available in all other tab processes in that browser session.

As a consequence, when a tab writes a Session cookie, Edge must send an interprocess communication (IPC) message to every other process in the browser session, telling each to update its internal variables with the new value of the Session cookie. This Cookie Pushing IPC is asynchronous, and if the named cookie were later modified in a process before the IPC announcing the earlier update to the cookie is received, that later update is obliterated.

The Duplicate Set-Cookie header version of this bug got fixed in the Fall 2017 Update (RS3) to Windows 10 and thus my old Set-Cookie test case case no longer reproduces the problem.

Unfortunately, it turns out that the RS3 fix only corrected the behavior of the network stack when it encounters this pattern– if the cookie-setting calls are made via document.cookie, the problem reappears, as in this document.cookie test case.

BadBehavior

Playing with the repro page, you’ll notice that manually pushing “Set HOT as a Session cookie” or “Set as a Persistent cookie” works fine, because your puny human reflexes aren’t faster than the cookie-pushing IPC. But when you push the “Set twice” button that sets the cookie twice in fast succession, the HOT cookie disappears in Edge (and in IE11, if you have more than one tab open).

Until this bug is fixed, avoid using document.cookie to change a persistent cookie to a session cookie.

-Eric

In contrast, in Chrome, all networking occurs in the browser process (or a networking-only process), and if a tab process wants to get the current document.cookie, it must perform an IPC to ask the browser process for the cookie value. We call this “cookie pulling.”

As we finish up the next release of Windows 10 (Fall 2018), my team is hard at work triaging incoming bugs.

Many such bugs take the form “Edge does the wrong thing for this page. ${Other_Browser} works okay.

This post is designed to be an (ever-growing) index of some of the behavioral deltas that are the root cause of such issues:


Edge doesn’t allow navigation to DATA urls, even when they’d otherwise be converted to file downloads.

Using pushState or replaceState with |undefined| as the URL argument shows “undefined” in the Address box in Edge/IE but not Chrome or Firefox.

IE/Edge strip the Content-Encoding header from a compressed response; Firefox and Chrome leave the header in. For XmlHttpRequest’s getAllResponseHeaders, IE and Firefox maintain the case of HTTP Response header names while Chrome/Edge/Safari do not.

Chrome recognizes that a file with a .JSON extension has the type application/json (and vice versa) while IE/Edge only recognize that when the registry is configured with that mapping.

Chrome includes a hack that works around certificates that do not exactly match the domain on which they are served. Firefox, Edge, and IE do not include this hack, leading to a Certificate Name Mismatch Error when loading:

WWWAddition

Edge does not fully support the URL standard, meaning that URLs of the form http:/example.com (note the missing slash) do not work as expected.

Edge and IE do not allow navigation to HTTP URLs containing a UserInfo component. Other browsers currently (reluctantly) allow this syntax.

Edge RS5 introduces support for Web Authentication specification (in order to support FIDO2 tokens). That specification extends the Credential Management API with new methods, so the navigator.credentials object now exists. However, Edge does not implement the navigator.credentials.preventSilentAccess() method and attempting to call it will cause an exception due to the missing method. (Edge always prevents silent access, so a future implementation of this method will simply fulfill the promise immediately).

When a server returns a HTTP/[301|302|303|307|308] response, Edge/IE are unable to read the response body if the server didn’t include a Content-Length header or Transfer-Encoding: chunked (HTTP/1.1). This turns out to break login to YouTube TV, where Google returns a response body over HTTP/2 (which does not require explicit content lengths thanks to its inherent message framing).

Edge supports most of CSP2 but currently does not support nonces on sourced script elements (only inline script and styles) [Test page]. This limitation significantly complicates deployment of CSP for sites that cannot easily enumerate their source locations in the Content-Security-Policy header (Edge does not support CSP3’s strict-dynamic directive yet either). A broad rule (e.g. script-src https: ) can be used as a workaround but this does increase attack surface.

IE and Edge begin immediately downloading the content of a SCRIPT SRC, not waiting until the SCRIPT element is added to this DOM. This means, for instance, that adding a |crossorigin| attribute to that element after setting its source does not result in an |Origin| header being sent on the request.


(…to be continued…)

-Eric

As we finish up the next release of Windows 10, my team is hard at work triaging incoming bugs. Here’s a pattern that has come up a few times this month:

Bug: I click download in Edge:

DownloadButtonbut I end up on an error page:

WompWompDataURI

Womp womp.

If you watch the network traffic, you’ll see that no request even hits the network in the failing case. But, if you carefully scroll that ugly error URL to see the middle, the source of the problem appears:

ms-appx-web://microsoft.microsoftedge/assets/errorpages/dnserror.html?ErrorStatus=0x80704006&NetworkStatusSupported=1#data:text/csv;charset=UTF-8, ID,Datetime,Type,Status,Note,From,To,Amount%20(total),Amount%20(fee),Funding%20Source,Destination%0D%0A

The error shows that Edge failed to navigate to a URL with the Data URI scheme.

Ever since we introduced support for DATA URLs a decade ago in Internet Explorer 8, they’ve been throttled with one major limitation: You cannot navigate to these URIs at the top level of the browser. Edge loosened things up so that Data URLs under 4096 characters can be used as the source of IFRAMEs, but the browser will not navigate to a data URL at the top level.

(Yes, this error page could use some love.)

Now, you might remember that last winter, Chrome took a change to forbid top-level navigation to data URIs (due to spoofing concerns), but that restriction contains one important exception: navigations that get turned into downloads (due to their MIME type being one other than something expected to render in the browser) are exempted. So this scenario sorta works in Chrome. (I say “sorta” because the authors of this site failed to specify a meaningful filename on the link, so the file downloads without the all-important .csv extension).

ChromeWorksSorta2

So, does IE/Edge’s restriction on Data URIs mean that webdevs cannot generate downloadable files dynamically in JavaScript in a way that works in all browsers?

No, of course not.

There are many alternative approaches, but one simple approach is to just use a blob URL, like so:

  var text2 = new Blob(["a,b,c,d"], { type: 'text/csv'});
  var down2 = document.createElement("a");
  down2.download = "simple.csv";
  down2.href = window.URL.createObjectURL(text2);
  document.body.appendChild(down2);
  down2.innerText="I have a download attribute. Click me";

When the link is clicked, the CSV file is downloaded with a proper filename.

 

-Eric

The Microsoft Edge (nee Internet Explorer) team held one of their “#AskMSEdge chats” on Twitter yesterday.

image

After watching the stream, @MarkXA neatly summarized the chat:

image

The folks over on WindowsCentral built out a larger summary of the tidbits of news that did get answered on the chat, some of which were just pointers to their Status and UserVoice sites.

After the chat ended, I complained that none of my questions had been answered:

image

In response, an IE Engineer retorted:

image

I don’t think that’s fair. Here are my questions, and a few remarks on each:

image

As far as I know, I’ve never asked the IE/Edge team about Brotli before, as I hadn’t spent any time looking at it until very recently. I’m interested in the team’s plans for Content-Encoding: Brotli because it can significantly improve browser performance, and if the team implements WOFF2, they must integrate Brotli decoding logic anyway.

image

I don’t think I’ve ever asked the IE/Edge team about their plans here before. More efficient HTTPS algorithms are important for both performance and battery life on mobile devices in particular, and thus I think they’re a great investment.

image

I have asked this before. IE has had non-standard network export for four years and I was really excited that Edge moved from HTTP Archive XML to the standard HTTP Archive JSON format. Unfortunately, this bug makes their code non-interoperable. The fix will be one or two lines of code. I feel justified in asking for status since weeks or months have passed without update.

image

I have asked this before. Edge regressed a significant piece of functionality and created a denial-of-service condition in their browser. I feel justified in asking for status since weeks or months have passed without update.

image

I have asked this one over and over again. I find it galling that Microsoft products are less secure together, and especially when Microsoft’s new President promised to close these sort of gaps nearly two years ago. It’s clear that the team agrees that the behavior is bad, because Edge uses Bing securely and doesn’t even allow users to add non-HTTPS search providers.

image

This one is probably the least “fair” of the questions, insofar as I already know the answer and I’m effectively just calling the team out on the specious nature of the promise to “watch demand” they made when the original concerns about the absence of Windows 7 support were raised.

However, I’ll note that the team answered several repeats of the question “When will it run on Mac? When will it run on iOS and Android.” Given the Windows 7 marketshare dominance, I think this question remains fair.

Other Unanswered

Other folks asked several great questions that didn’t get answered:

image

I really want this feature.

image

I think the Edge team is making a huge mistake if they’re not piloting their new extension model with critical extension developers like uBlock, NoScript, etc.

image

Because of the nature of the legacy Win32 Address Bar’s context menu, Paste-and-Go was always prohibitively expensive. The UI replacement for Metro IE and now Edge makes this a trivially added feature that was requested by several questions.

Unwanted Answers

Some questions got answers that I’m just not happy with, but I’m tired of complaining about:

image

The Edge team replied “No” and suggested they consider this a scenario for the new extension model. I think this is a mistake and a case where “different” isn’t likely to be “better.”

 

Several folks asked when the new extension model would be released. “Stay tuned” was the answer.

Subtext

I think the subtext of Adrian’s complaint is that “You’ve worked here, you know we don’t announce things on IEDevChats.” There’s some truth to that frustration – I know that announcements are carefully vetted and published on the blog and I understand why live chats aren’t a source of new information. However…  

Rabble-Rousing and Information Asymmetry

I know some folks think my questions are just rabble-rousing and that, as an ex-teammate and current MVP I should be asking these questions in private, directly to the IE team. A few points on that:

  • Microsoft has basically requested we provide feedback in this manner, with the “we’re watching feedback to influence our decisions” position on everything from features to bugfixes. Feedback that isn’t getting public traction is largely ignored.
  • A significant number of my friends and colleagues are no longer on the IE team. In the photo of the team answering questions, I recognize seven of the seventeen engineers.
  • My emails to the IE MVP discussion list generally do not receive replies.
  • Direct emails to individual engineers on the IE team often do not get replies.
  • Status on bugs I’ve filed with MSRC is similarly hard to get– issues have languished for months without so much as a “working on it” status update.

Having been on both sides of the fence now, it’s plain to me that one serious problem Microsoft has is that they don’t realize how incredibly opaque things are from outside the company. As an engineer racing from one issue to the next, it’s easy to deprioritize status updates and justify doing so when there are so many higher-priority things to fix. From outside the company, however, “working on it and coming as soon as we can”  is often indistinguishable from “ignoring—really hope this goes away.” That problem is exacerbated by Microsoft’s tendency not to deliver hard messages like “Silverlight is dead dead dead, get off it now!” in a timely manner to allow customers and partners to plan appropriately.