text/plain

DotNet Makes Me Sad, In Pictures

.NET Framework KB 3088956:

Ouch, that sounds pretty severe.

I guess I’d better go manually install a hotfix?

Seriously? An email address and a CAPTCHA? Fine.

Oh, an email delivered HTTP URL pointed at an executable file? That seems totes legit.

Yup, definitely legit, it says “Microsoft” right there at the top!

Sure, let’s put those files in the root of the C:\ drive. I’m sure that’ll work.

Oh, guess it did. Magic!

Oh, wait. No files in C:\. No magic, I guess.

Let’s use Process Monitor to watch the file writes… that’s what all the cool kids have to do to install patches, right?

Ah, there you are! UAC virtualization like it’s 2006!

Of course I must, Microsoft, of course I must.

-Eric

The Budget

“Don’t tell me what you value, show me your budget, and I’ll tell you what you value.” – Joe Biden

Across the political spectrum, Americans have thoughts on how the government should spend the money we send its way. That’s great (and it’d be better if more of us voted), but many arguments about spending are based on very misguided notions of how we’re currently spending our money. So, today, some charts.

Source: OMB.

See that blue piece? That’s what has a lot of folks worried. That’s the interest on the $18.4T national debt, which grew by another $426 billion dollars this year. So how much is $426 billion? Just over a third of what the US government got to spend on discretionary spending this year:

That military chunk is huge (54%), but it’s worth keeping in mind that this money isn’t just disappearing; we’re spending it on American-built weapons, salaries for Americans, etc. It’s fair to question whether we should be bringing those resources to bear on other challenges.

So, where does the money go when you include everything?

Social Security and Medicare are a huge chunk of where our money goes, but a large percentage of Americans seem very confused about how these programs work; politicians on both sides of the aisle are guilty of deliberately confusing the public about them.

If you ask Americans how much we spend on Foreign Aid, the average guess is 28%; in actuality, it’s about 1%.

Interestingly the government also publishes how much money it forgoes in the form of explicit tax breaks:

To whom do we owe that money?

Today, we owe most of the money to ourselves.

Other Stuff

Not all debt is bad—if you’re investing what you’re borrowing and getting a greater return on that investment, you’re doing it right.
Comparisons of national income and spending to household income and spending are often trite and very misleading. But the counterarguments against such comparisons tend to obscure their own weaknesses too.
XKCD did a comic on money.
The White House Calligrapher’s Office spends $277k per year on salaries for its three employees.

What’s New in Fiddler 4.6.0.7

TLDR? – Get the newest Fiddler here. We’re performing a staged rollout of this build; it won’t be on autoupdate until next week.

Under the Hood

As mentioned in our notes about the Fiddler 4.6 release, we’ve started taking a very close look at Fiddler’s performance. Fiddler’s use of the CPU, system memory, and the network have gone under the microscope and this new release includes several major changes to how Fiddler uses threads and memory. If you frequently run Fiddler with a large amount of traffic in parallel, or run Fiddler on a slower or heavily-loaded PC, this new version should provide significantly improved performance. We’ve also improved overall performance by using better algorithms in scenarios like the Find Sessions (CTRL+F) experience.

The !threads and !memory QuickExec commands have been enhanced to provide insights into fine-grained performance details about Fiddler’s operation.

The Performance Tab

The new Performance tab in the Fiddler Options dialog offers choices that can significantly change Fiddler’s runtime performance and memory usage.

The Show Memory panel in status bar controls whether Fiddler’s status bar shows a panel that indicates the current memory usage tracked by the garbage collector. For example, when Fiddler has 64mb of managed memory allocated, the panel looks like this:

Left-click the memory panel to instruct the .NET Framework to perform an immediate garbage collection. Right-click the panel to launch the Fiddler Options dialog box with the Performance tab activated.

The Parse WebSocket Messages checkbox controls whether Fiddler will parse WebSocket streams into individual messages, allowing display in the WebSocket tab and manipulation using the OnWebSocketMessage event handler. Disabling WebSocket Message parsing will reduce CPU usage and may save a significant amount of memory if high-traffic WebSockets are in use. Even if you disable WebSocketMessage parsing globally using this checkbox, it can be reenabled on a per-Session basis by setting the x-Parse-WebSocketMessages flag on the Session object.

The Stream and forget bodies over box controls the maximum size of a message body that Fiddler will retain. By default, the limit is just under 2 gigabytes for 64bit Fiddler and 16 megabytes for 32bit Fiddler; the much smaller default for 32bit helps avoid problems with “Out of Memory” errors when running Fiddler in a small address space. If, while reading a message body, Fiddler finds that it is larger than the threshold, it will configure the body to stream and will “drop” the bytes of the body to conserve memory. If you attempt to inspect a Session which has been dropped, you will see the following notification bar:

Clicking the bar will open the Fiddler Options dialog to allow you to reconfigure the limit for subsequent Sessions.

The If client aborts while streaming dropdown controls Fiddler’s behavior if a response body is streaming to the client but the client closes the connection. Depending on your choice here, Fiddler can continue to read the body from the server (useful if you’re collecting traffic) or abort the Session (useful to save memory and CPU cycles).

The Run Fiddler at AboveNormal Priority alters Fiddler’s default scheduling priority. If you enable this option, Windows will prioritize activation of Fiddler’s threads when they have work to do (e.g. reading a new request or response from the network). You can easily experiment with this option to see whether it improves the overall throughput of your client (browser) and Fiddler.

New QuickFilters

The Session list’s Filter Now context menu has been enhanced with two new filters:

The Hide /1stpath/ filter hides any traffic whose Url path component starts with the specified string.

The Hide Url… option uses the current Session’s Url as the default of a Url filter; you can edit the string to apply more broadly by removing text from the start or end of the string:

High DPI Improvements

Today, only a small number of Fiddler users (< 4%) run Fiddler on Windows systems with a non-default screen DPI, but we want Fiddler to work great for those users too. The latest build of Fiddler includes a number of DPI-related fixes. Fiddler is not yet marked DPI aware in its manifest; if you’d like to see Fiddler in its DPI-aware mode, run Fiddler with the -dpiAware command line argument:

fiddler.exe -dpiAware

We will continue to make improvements as problems are discovered or reported and expect to eventually set the dpiAware flag by default.

New HTTPS Cipher Option

Fiddler 4 on Windows 7 and later supports modern TLS versions (TLS 1.1 and TLS 1.2) and the HTTPS tab on the Fiddler Options dialog enables you to easily enable these protocols. However, TLS 1.1 and 1.2 remain off-by-default for compatibility reasons.

The Enabled Protocols link on the HTTPS dialog now supports a new token <client>; if present, this token adds to the list of versions offered to the server the latest protocol that has been offered by the client. For instance, with these settings:

… a request from Internet Explorer offering TLS 1.2 will be presented to the server with TLS 1.2 and TLS 1.0 enabled. The advantage of using the <client> token is that if the request fails, many browser clients are configured to “fall back” and attempt negotiation with an earlier protocol version. In this example, if the TLS 1.2 connection fails, the browser will retry with TLS 1.0 and the connection may succeed.

You must include at least one specific TLS version in the HTTPS Protocols list to handle cases where the request was generated by Fiddler itself (e.g. the Composer tab).

Brotli Compression Support

Researchers at Google have developed a new compression algorithm named Brotli that offers significantly better compression than the DEFLATE algorithm used by Gzip. This new compression algorithm is already in use by many browsers today (inside the WOFF2 font format) and it is expected to appear as a HTTP Content-Encoding in early 2016.

Fiddler now supports Brotli as a Content-Encoding everywhere compression is supported; simply download the Authenticode-signed Windows Brotli.exe and place it in the Fiddler2\Tools\ subfolder in your Program Files folder. After you restart Fiddler, you will find a new Brotli option on the Transformer tab and Fiddler APIs like utilDecodeResponse() will be able to decompress Brotli-encoded content:

The new version of Fiddler also has better handling of unsupported compression schemes like SDCH—if a response with Content-Encoding: sdch,gzip is encountered, for instance, the various decoding APIs will decompress with GZIP and then stop without removing the SDCH token.

Hidden Tabs

FiddlerScript’s BindUITab attribute now supports a <hidden> token:

If this token is present, the tab is not shown until the user manually activates it via the View > Tabs menu:

This feature helps you “unclutter” Fiddler by keeping uncommonly-used script tabs hidden.

More Powerful ImageView Extensions

Fiddler’s ImageView Extensions feature allows you to add new commands to the Tools context menu on the ImageView Inspector:

Now you can use a new Options parameter to specify that Fiddler should show the <stdout> or <stderr> results of running the target tool, and a new {out:extension} token enables you to specify that the target tool writes a file that Fiddler should load as a new Session in the Web Sessions list.

For instance, here’s the logic to add a new ToWebP Lossless command to the list:

To use it, add the registry entries and place CWebP.exe in Fiddler’s Tools subfolder. When you invoke the command, Fiddler will run the tool, passing an input temporary file containing the JPEG image to the tool in the {in} parameter and specifying an autogenerated filename with a .webp file extension in the {out:webp} parameter. The cwebp.exe tool will be run, any text from Standard Error will be collected and displayed to the user, and the file named by the {out} token will be reloaded as a new Web Session:

Updated – Show Image Bloat

I’ve also updated the Show Image Bloat add-on (described here) with some additional tweaks and features; the add-on has improved bloat detection for JPEG and GIF files and has other minor improvements. Install the latest build of Show Image Bloat (v2.6) and activate it from the Fiddler Rules menu.

I hope you enjoy these new improvements to Fiddler – Keep sending in your feedback to ensure we’re evolving the tool to best meet your needs.

-Eric Lawrence

Viewing HTTPS Handshakes in Fiddler

You can easily use Fiddler to evaluate what algorithms a client is using to connect to a HTTPS server in Fiddler.

First, adjust Fiddler’s configuration using Tools > Fiddler Options to enable capture of CONNECT tunnels but disable decryption:

HTTPS Options

Disabling decryption is necessary because Fiddler decrypts traffic using a HTTPS man-in-the-middle technique, which means that when it’s enabled you’ll see what the client and server are using to talk to Fiddler, which could be different than what they’d use if Fiddler were not in the middle.

After making this change, simply load a HTTPS site inside your browser, and double-click on any of the CONNECT tunnel entries shown in Fiddler:

CONNECT Tunnel

Fiddler’s TextView Request Inspector will show you a parsed view of the Client’s HTTPS handshake:

TextView Request Inspector

… and the TextView Response Inspector below will show the parameters chosen by the server for the connection:

TextView Response Inspector

In this case, you can see that the Server agreed to a TLS/1.2 connection using RSA for key exchange and 128-bit AES as the symmetric encryption algorithm. The server ignored the ALPN extension indicating that the connection will not be using HTTP2 or SPDY for data transfers.

Fiddler will show you exactly what your client and server agreed upon. If you’re interested in exploring what other clients with other options might negotiate, the SSLLabs Server Test is a great tool. Similarly, if you’re curious about the capabilities of your browser, check out the SSLLabs Client Test.

-Eric

Fiddler and Brotli

Regular readers of my blog know how excited I am about Google’s new open compression engine, Brotli. Support is in Firefox 44 Nightly already and is expected for other browsers. Because Brotli is used inside the WOFF2 font format, many clients already have the compression code and just need to expose it in a new way.

Yesterday’s Fiddler release supports Brotli. To expose Brotli inside Fiddler 4.6.0.5+, download and place brotli.exe inside Fiddler’s %ProgramFiles(x86)%\Fiddler2\Tools\ subfolder and restart to see it appear on the Transformer tab:

After installation, Content-Encoding: br is also supported by Fiddler’s Decode toolbar option, as well as the encoding Infobar:

Test servers are starting to come online for Brotli, particularly now that Google has released a new Brotli module for the nginx web server. For now, you can try downloading this image:

…that is served with brotli encoding. If your browser supports brotli, it will render a star; if not, the image will appear broken unless you set the Decode option in Fiddler’s toolbar so Fiddler will decode the image before the browser sees it.

-Eric Lawrence

Duct Tape and Baling Wire–Cookie Prefixes

Update: Cookie Prefixes are supported by Chrome 49, Opera 36, and Firefox 50. Test page; no status from the Edge team.

A new cookie feature called SameSite Cookies has been shipped by Chrome, Firefox and Edge; it addresses slightly different threats.

When I worked on Internet Explorer, we were severely constrained on development resources. While the team made a few major investments for each release (Protected Mode, Loosely-coupled IE, new layout engines, etc), there was a pretty high bar to get any additional feature work in. As a consequence, I very quickly learned to scope down any work I needed done to the bare minimum required to accomplish the job. In many cases, I wouldn’t even propose work if I wasn’t confident that I (a PM) could code it myself.

In many cases, that worked out pretty well; for instance, IE led the way in developing the X-Frame-Options clickjacking protection, not only because we found other approaches to be unworkable (bypassable, compat-breaking, or computationally infeasible) but also because a simple header (internally nicknamed “Don’t Frame Me, Bro”) was the only thing we could afford to build¹.

In other cases, aiming for the bare minimum didn’t work out as well. The XDomainRequest object was a tiny bit too simple—for security reasons, we didn’t allow the caller to set the request’s Content-Type header. This proved to be a fatal limitation because it meant that many existing server frameworks (ASP, ASPNET, etc) would need to change in order to be able to properly parse a URLEncoded request body string.

One of the “little features” that lingered on my whiteboard for several years was a proposal called “Magic-Named Cookies.” The feature aimed to resolve one significant security shortcoming of cookies—namely, that a server has no way to know where a given cookie came from. This limitation relates to the fact that the attributes of a cookie (who set it, for what path, with what expiration time, etc) are sent to the client in the Set-Cookie header but these attributes are omitted when the Cookie header is sent back to the server. Coupled with cookies’ loose-scoping rules (where a cookie can be sent to both “parent” and “sub” domains, and cookies sent from a HTTP origin are sent to the HTTPS origin of the same hostname) this leads to a significant security bug, whereby an attacker can perform a “Cookie Fixation” attack by setting a cookie that will later be sent to (and potentially trusted by) a secure origin. These attacks still exist today, although various approaches (e.g. HSTS with includeSubdomains set) are proposed to mitigate it.

RFC2965 had attempted to resolve this but it never got any real adoption because it required a major change in the syntax of the Cookie header sent back to the server, and changing all of the clients and servers proved too high a bar.

My Magic-Named Cookies proposal aimed to address this using the “The simplest thing that could possibly work” approach. We’d reserve a cookie name prefix (I proposed $SEC-) that, if present, would indicate that a cookie had been set (or updated) over a HTTPS connection. The code change to the browser would be extremely simple: When setting or updating a cookie, if the name started with $SEC-, the operation would be aborted if the context wasn’t HTTPS. As a consequence, a server or page could have confidence that any cookie so named had been set by a page sent on a HTTPS connection.

While magic naming is “ugly” (no one likes magic strings), the proposal’s beauty is in its simplicity—it’d be a two line code change for the browser, and wouldn’t add even a single bitfield to the cookie database format. More importantly, web server platforms (ASP, ASPNET, etc) wouldn’t have to change a single line of code. Web Developers and frameworks could opt-in simply by naming their cookies with the prefix—no other code would need to be written. Crucially, the approach degrades gracefully (albeit unsecurely)—legacy clients without support for the restriction would simply ignore it and not enforce the restriction, leaving them no more (or less) safe than they were before.

Unfortunately, this idea never made it off my whiteboard while I was at Microsoft. Over the last few years, I’ve tweeted it at the Chrome team’s Mike West a few times when he mentions some of the other work he’s been doing on cookies, and on Wednesday I was delighted to see that he had whipped up an Internet Draft proposal named Cookie Prefixes. The draft elaborates on the original idea somewhat:

changing $SEC- to __SECURE-
requiring a __SECURE- cookie to have the secure attribute set
adding an __HOST- prefix to allow cookies to inform the server that they are host-locked

In Twitter discussion, some obvious questions arose (“how do I name a cookie to indicate both HTTPS-set and Origin locked?” and “is there a prefix I can use for first-party-only cookies”?) which lead to questions about whether the design is too simple. We could easily accommodate the additional functionality by making the proposal uglier—for instance, by adding a flags field after a prefix:

Set-Cookie: $RESTRICT_ofh_MyName= I+am+origin-locked+first+party+only+and+httponly; secure; httponly

Set-Cookie: $RESTRICT_s_MyName2= I+am+only+settable+by+HTTPS+without+other+restrictions

… but some reasonably wonder whether this is too ugly to even consider.

Cookies are an interesting beast—one of the messiest hacks of the early web, they’re too important to nuke from orbit, but too dangerous to leave alone. As such, they’re a wonderful engineering challenge, and I’m very excited to see the Chrome team probing to find improvements without breaking the world.

-Eric Lawrence

¹ See Dan Kaminsky’s proposal to understand, given infinite resources, the sort of ClickJacking protection we might have tried building.

WebP–What Isn’t Google Telling Us?

Beyond their awesome work on Zopfli and Brotli, Google has brought their expertise in compression to bear on video and image formats. One of the most interesting of these efforts is WebP, an image format designed to replace the aging JPEG (lossy) and PNG (lossless) image formats.

WebP offers more efficient compression mechanisms than both PNG and JPEG, as you can see in this comparison of a few PNG files on Google’s top sites vs. WebP-Lossless versions that are pixel-for-pixel identical:

size table

You can see these savings everywhere, from Google’s homepage logo, which is 3918 bytes (29%) smaller, to Google applications’ image sprites (59% smaller!) to advertisements served by Google’s ad network (18% smaller). These compression savings are much greater than those provided by Zopfli, which is constrained by compatibility with the legacy PNG format.

As an additional benefit, WebP files don’t contain the sort of metadata bloat found in PNG, JPEG, and GIF.

So, the bandwidth and cache-size savings are obvious.

While the format is currently only supported in Chrome and Opera, web servers can easily serve WebP to only clients that request it via the Accept header:

Fiddler screenshot showing WebP in use

This approach to WebP adoption is in use today by major sites like the Washington Post.

Google invented the format, so it’s not a case of “not-invented-here.”

The non-adoption of their own format leads to a troubling question—is there something about WebP that Google isn’t telling us? Surely there must be a good reason that Google’s own properties aren’t reaping the benefits of the format they’ve invented?

Update: Alex Russell retorts “uh, we use webp in TONS of places.”

-Eric Lawrence

PS: WebP Status Tracking links for Firefox and IE/Edge

Help–The Whole Web Thinks I’m Using IE7!!!

Every few weeks for the last six or so years, I see someone complain on Twitter or in forums that the entire Internet seems to think they’re running an old version of IE. For instance, an IE11 user on Windows 8.1 might see the following warning on Facebook:

These warnings typically occur when the browser is using Compatibility View mode for a site and the site demands a browser that supports modern standards. Many customers used to find themselves accidentally in this state because they were overzealously clicking the “Compatibility View” button (back when IE had one) or clicking the “Display all sites in Compatibility View” checkbox (back when IE had it).

Since IE11 has cleaned that mess up (by hiding Compatibility View), you might wonder how a user could end up in such a broken state.

The answer is both complicated and interesting, deeply intertwined with nearly 15 years of subtle Internet Explorer behaviors.

When I ask the affected IE11 user to visit my User-Agent string test page, they see IE7’s Compatibility View user-agent string:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.3; Win64; x64; Trident/7.0; .NET4.0E; .NET4.0C; Media Center PC 6.0; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729)

But why?

Since IE no longer shows the Zone in the status bar, you must right-click the page and choose Properties to get your next clue:

Wait, what?!? Why is some random site on the Internet in the privileged Local Intranet security zone?

Next the user does the same test on Facebook.com and finds that it too is in the Intranet Zone. In fact, the whole web is getting zoned as Intranet!

This represents a significant security hole, and the user has only discovered it because, by default, Tools > Compatibility View Settings has Display Intranet sites in Compatibility View set, and the unwanted CompatView causes sites like Facebook to complain.

So what’s going on here!?!

Click Tools > Internet Options > Connections > LAN Settings, and observe that the settings are the defaults:

Wait… what exactly does that Automatically detect settings option do?

Why, it allows a computer on your network to decide what proxy server your client should use through a process called WPAD. The server in question gets to supply a proxy configuration script that implements a function FindProxyForUrl(). That function returns either a proxy (e.g. “PROXY myproxy:8080” or “DIRECT” to indicate that the request should be sent directly to the origin server and bypass the proxy.

And now we’re getting somewhere. Take a look at the checkboxes inside Tools > Internet Options > Security > Local Intranet > Custom Level, specifically the second checkbox:

Yup, that’s right—if a proxy script returns DIRECT for a given site, IE defaults to treating that site as a part of the Local Intranet Zone, giving it additional privileges and also defaulting it to CompatView. Oops.

You might think: “well, surely a network proxy administrator would never make that mistake!”

Back in 2011, the IE team started getting email from all over the company complaining that “IE is broken. It doesn’t support HTML5!” Guess why not? Oops.

Recommendations

Unless you’re running IE on a Corporate Network that requires support for things like Negotiate Authentication and the like, you should untick the Automatically detect intranet network checkbox and all of the checkboxes beneath it. This improves security and enhances IE’s sandbox.

Unless you’re running a laptop that moves to corporate networks, you should also disable the Automatically detect settings checkbox to prevent IE from asking your network what proxy to use.

-Eric Lawrence

Ad Publishers–A TODO List

Where’s Google’s* blog on how they’re doing everything they can to make ads they serve as fast and small as possible?

Where’s Google’s blog on how many ads they’ve nuked as “deceptive” and trumpeting how policy forbids ads for “adware-wrapped” installers?

Where’s Google’s blog about how many billions of ad-generated dollars they’ve supplied to content sites and open-source products that people care about like Firefox?

Where’s Google’s blog on how much smaller they’ve made ads served using WebP instead of older formats? And Zopflinated PNGs for legacy browsers?

Ad publishers can’t expect a company like EmbarcaderoTech to know how to make fast ads. Publishers need to help.

Where’s the argument that the worst privacy impact of ads comes from trying to recover revenue lost through blocks and fraud?

Without good answers to these questions, ad publishers are going to have a very hard time regaining any control of the narrative. The entire industry is demonized for performance, security, and privacy problems, even though each publisher has different practices.

-Eric

*Note: Google does plenty of things right from an advertising point-of-view; I only mention them by name because they’re probably the biggest and I expect the best technology from them. They’ve invented much of the interesting technology in this space, including Zopfli and WebP.

Ad-Block: Current Status

Microsoft used to joke about cutting off a competitor’s air supply. Apple instead handed shears to devs, pointed at a hose, and walked out.

As iOS9 launches, here’s the list of top paid apps in the iTunes AppStore:

AppStore - Top Paid Screenshot

It’s only a question of when, not if, the top slot of the Free Apps category is an open-source alternative that will save users $3-$4 and offer the same functionality.

I’m pretty sure the Web is at exactly this moment:

The next year or so is going to be very very messy.

Update (Sept-18-2015): After 36 hours, the maker of the top-grossing blocker bowed out of the market. This is unexpected windmill tilting. I love it.