If I were a better writer, I’d write as well as Maciej does.
A very nice presentation on privacy, advertising, malvertising, click fraud, and more: http://idlewords.com/talks/what_happens_next_will_amaze_you.htm
Help–The Whole Web Thinks I’m Using IE7!!!
Every few weeks for the last six or so years, I see someone complain on Twitter or in forums that the entire Internet seems to think they’re running an old version of IE. For instance, an IE11 user on Windows 8.1 might see the following warning on Facebook:
These warnings typically occur when the browser is using Compatibility View mode for a site and the site demands a browser that supports modern standards. Many customers used to find themselves accidentally in this state because they were overzealously clicking the “Compatibility View” button (back when IE had one) or clicking the “Display all sites in Compatibility View” checkbox (back when IE had it).
Since IE11 has cleaned that mess up (by hiding Compatibility View), you might wonder how a user could end up in such a broken state.
The answer is both complicated and interesting, deeply intertwined with nearly 15 years of subtle Internet Explorer behaviors.
When I ask the affected IE11 user to visit my User-Agent string test page, they see IE7’s Compatibility View user-agent string:
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.3; Win64; x64; Trident/7.0; .NET4.0E; .NET4.0C; Media Center PC 6.0; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729)
But why?
Since IE no longer shows the Zone in the status bar, you must right-click the page and choose Properties to get your next clue:
Wait, what?!? Why is some random site on the Internet in the privileged Local Intranet security zone?
Next the user does the same test on Facebook.com and finds that it too is in the Intranet Zone. In fact, the whole web is getting zoned as Intranet!
This represents a significant security hole, and the user has only discovered it because, by default, Tools > Compatibility View Settings has Display Intranet sites in Compatibility View set, and the unwanted CompatView causes sites like Facebook to complain.
So what’s going on here!?!
Click Tools > Internet Options > Connections > LAN Settings, and observe that the settings are the defaults:
Wait… what exactly does that Automatically detect settings option do?
Why, it allows a computer on your network to decide what proxy server your client should use through a process called WPAD. The server in question gets to supply a proxy configuration script that implements a function FindProxyForUrl(). That function returns either a proxy (e.g. “PROXY myproxy:8080” or “DIRECT” to indicate that the request should be sent directly to the origin server and bypass the proxy.
And now we’re getting somewhere. Take a look at the checkboxes inside Tools > Internet Options > Security > Local Intranet > Custom Level, specifically the second checkbox:
Yup, that’s right—if a proxy script returns DIRECT for a given site, IE defaults to treating that site as a part of the Local Intranet Zone, giving it additional privileges and also defaulting it to CompatView. Oops.
You might think: “well, surely a network proxy administrator would never make that mistake!”
Back in 2011, the IE team started getting email from all over the company complaining that “IE is broken. It doesn’t support HTML5!” Guess why not? Oops.
Recommendations
Unless you’re running IE on a Corporate Network that requires support for things like Negotiate Authentication and the like, you should untick the Automatically detect intranet network checkbox and all of the checkboxes beneath it. This improves security and enhances IE’s sandbox.
Unless you’re running a laptop that moves to corporate networks, you should also disable the Automatically detect settings checkbox to prevent IE from asking your network what proxy to use.
-Eric Lawrence
Ad Publishers–A TODO List
Where’s Google’s* blog on how they’re doing everything they can to make ads they serve as fast and small as possible?
Where’s Google’s blog on how many ads they’ve nuked as “deceptive” and trumpeting how policy forbids ads for “adware-wrapped” installers?
Where’s Google’s blog about how many billions of ad-generated dollars they’ve supplied to content sites and open-source products that people care about like Firefox?
Where’s Google’s blog on how much smaller they’ve made ads served using WebP instead of older formats? And Zopflinated PNGs for legacy browsers?
Ad publishers can’t expect a company like EmbarcaderoTech to know how to make fast ads. Publishers need to help.
Where’s the argument that the worst privacy impact of ads comes from trying to recover revenue lost through blocks and fraud?
Without good answers to these questions, ad publishers are going to have a very hard time regaining any control of the narrative. The entire industry is demonized for performance, security, and privacy problems, even though each publisher has different practices.
-Eric
*Note: Google does plenty of things right from an advertising point-of-view; I only mention them by name because they’re probably the biggest and I expect the best technology from them. They’ve invented much of the interesting technology in this space, including Zopfli and WebP.
Ad-Block: Current Status
Microsoft used to joke about cutting off a competitor’s air supply. Apple instead handed shears to devs, pointed at a hose, and walked out.
As iOS9 launches, here’s the list of top paid apps in the iTunes AppStore:
It’s only a question of when, not if, the top slot of the Free Apps category is an open-source alternative that will save users $3-$4 and offer the same functionality.
I’m pretty sure the Web is at exactly this moment:
The next year or so is going to be very very messy.
Update (Sept-18-2015): After 36 hours, the maker of the top-grossing blocker bowed out of the market. This is unexpected windmill tilting. I love it.
Developer Advocacy
The Microsoft Edge (nee Internet Explorer) team held one of their “#AskMSEdge chats” on Twitter yesterday.
After watching the stream, @MarkXA neatly summarized the chat:
The folks over on WindowsCentral built out a larger summary of the tidbits of news that did get answered on the chat, some of which were just pointers to their Status and UserVoice sites.
After the chat ended, I complained that none of my questions had been answered:
In response, an IE Engineer retorted:
I don’t think that’s fair. Here are my questions, and a few remarks on each:
As far as I know, I’ve never asked the IE/Edge team about Brotli before, as I hadn’t spent any time looking at it until very recently. I’m interested in the team’s plans for Content-Encoding: Brotli because it can significantly improve browser performance, and if the team implements WOFF2, they must integrate Brotli decoding logic anyway.
I don’t think I’ve ever asked the IE/Edge team about their plans here before. More efficient HTTPS algorithms are important for both performance and battery life on mobile devices in particular, and thus I think they’re a great investment.
I have asked this before. IE has had non-standard network export for four years and I was really excited that Edge moved from HTTP Archive XML to the standard HTTP Archive JSON format. Unfortunately, this bug makes their code non-interoperable. The fix will be one or two lines of code. I feel justified in asking for status since weeks or months have passed without update.
I have asked this before. Edge regressed a significant piece of functionality and created a denial-of-service condition in their browser. I feel justified in asking for status since weeks or months have passed without update.
I have asked this one over and over again. I find it galling that Microsoft products are less secure together, and especially when Microsoft’s new President promised to close these sort of gaps nearly two years ago. It’s clear that the team agrees that the behavior is bad, because Edge uses Bing securely and doesn’t even allow users to add non-HTTPS search providers.
This one is probably the least “fair” of the questions, insofar as I already know the answer and I’m effectively just calling the team out on the specious nature of the promise to “watch demand” they made when the original concerns about the absence of Windows 7 support were raised.
However, I’ll note that the team answered several repeats of the question “When will it run on Mac? When will it run on iOS and Android.” Given the Windows 7 marketshare dominance, I think this question remains fair.
Other Unanswered
Other folks asked several great questions that didn’t get answered:
I really want this feature.
I think the Edge team is making a huge mistake if they’re not piloting their new extension model with critical extension developers like uBlock, NoScript, etc.
Because of the nature of the legacy Win32 Address Bar’s context menu, Paste-and-Go was always prohibitively expensive. The UI replacement for Metro IE and now Edge makes this a trivially added feature that was requested by several questions.
Unwanted Answers
Some questions got answers that I’m just not happy with, but I’m tired of complaining about:
The Edge team replied “No” and suggested they consider this a scenario for the new extension model. I think this is a mistake and a case where “different” isn’t likely to be “better.”
Several folks asked when the new extension model would be released. “Stay tuned” was the answer.
Subtext
I think the subtext of Adrian’s complaint is that “You’ve worked here, you know we don’t announce things on IEDevChats.” There’s some truth to that frustration – I know that announcements are carefully vetted and published on the blog and I understand why live chats aren’t a source of new information. However…
Rabble-Rousing and Information Asymmetry
I know some folks think my questions are just rabble-rousing and that, as an ex-teammate and current MVP I should be asking these questions in private, directly to the IE team. A few points on that:
- Microsoft has basically requested we provide feedback in this manner, with the “we’re watching feedback to influence our decisions” position on everything from features to bugfixes. Feedback that isn’t getting public traction is largely ignored.
- A significant number of my friends and colleagues are no longer on the IE team. In the photo of the team answering questions, I recognize seven of the seventeen engineers.
- My emails to the IE MVP discussion list generally do not receive replies.
- Direct emails to individual engineers on the IE team often do not get replies.
- Status on bugs I’ve filed with MSRC is similarly hard to get– issues have languished for months without so much as a “working on it” status update.
Having been on both sides of the fence now, it’s plain to me that one serious problem Microsoft has is that they don’t realize how incredibly opaque things are from outside the company. As an engineer racing from one issue to the next, it’s easy to deprioritize status updates and justify doing so when there are so many higher-priority things to fix. From outside the company, however, “working on it and coming as soon as we can” is often indistinguishable from “ignoring—really hope this goes away.” That problem is exacerbated by Microsoft’s tendency not to deliver hard messages like “Silverlight is dead dead dead, get off it now!” in a timely manner to allow customers and partners to plan appropriately.
text/plain 
