OpenSearch

History

The new UI of Internet Explorer 7 included a dedicated search box adjacent to the address bar, like the then-new Firefox. As IE7 was built between 2004 and 2006, Microsoft didn’t have a very credible entry into the search engine market—Bing wouldn’t appear until 2009. The IE team made a wise decision in support of the open web—we embraced the nascent OpenSearch specification developed by Amazon for search provider specification, allowing the browser to easily discover search providers offered by the site and enabling users to easily add those providers to IE’s search box.

This was a huge win for openness– it ensured that IE users had their choice of the best search engines the web had to offer. There was no lock-in.

Aside: The Narrative

Part of the Internet Explorer team’s internal narrative1 for years was that only two browsers were properly aligned with user’s interest—the only browsers where the customer was also the user were Safari and Internet Explorer. Safari and IE were the browsers bought by the customers who purchased their vendor’s hardware and software respectively. In contrast, the story went, Firefox had one customer, Google, who paid hundreds of millions of dollars for the right to be the default search engine. Later, Chrome had many thousands of customers, the AdSense advertisers who were buying access to the real product (millions of users’ eyeballs). As a consequence, the narrative went, the IE team were champions of the user and thus we’d make every decision with only our customers’ experience in mind.

What Happened Next

Fortunately, OpenSearch was quickly successful, and both Chrome and Firefox adopted it and the window.external.AddSearchProvider API that allowed a site (upon a user-initiated action) to offer to add a new Search Provider to the browser. This enabled customers to easily access search engines both large (Google, Yahoo, Bing, etc) and niche (Amazon, MSDN, etc) within their browser of choice. Some browsers even used OpenSearch to allow users to access search providers without installing them.

Openness won…

Today

… until it didn’t. The Internet Explorer team has indicated that they don’t plan to support the de facto standard AddSearchProvider API they invented in their next browser, currently codenamed Project Spartan. They’ve offered a variety of defenses of the decision (e.g. “Safari doesn’t support it so we don’t have to!”) that they’ve previously ridiculed in other contacts (e.g. Pointer Events).

Currently, in fact, Spartan is hardcoded to use just one search engine—Bing. I have no doubt that the Spartan team will add additional search engines to their browser before they ship, but only an open API provides the freedom for sites and users to interact without any confounding politics and economic decisions. If I want to switch to a privacy-focused engine like DuckDuckGo, that should be trivial. If I want the ability to quickly run MSDN searches, this shouldn’t require petitioning the IE development team.

Security And Privacy

Making matters worse, Spartan users’ searches are sent to Bing over the network in plaintext, despite the fact that Bing supports HTTPS and the latest versions of both Chrome and Firefox use that HTTPS provider.

Some have argued that AddSearchProvider is “too powerful” and browsers shouldn’t offer APIs that enable changes to their configuration, even with user consent. This argument is compelling until you notice what actually happens—when you take away the sandboxed, restricted API, sites don’t just throw up their hands and say “Ah well, guess we’ll go home.” Instead, they send the user executable downloads that can take any action they like on the system, changing the search provider and, while they’re at it, reconfiguring the user’s other browsers, changing the search page, throwing in a toolbar or two, or whatever. Once the user’s suckered into running your code, why not maximize the value? And the Windows ecosystem continues its swirl toward the drain…

Other users have argued that a “gallery” of search providers, like IEAddons.com is the right way to go. There are many problems with this approach. First, it requires that each site go to Microsoft, hat in hand, and register a provider. It requires that users go out of their way to go to the Gallery. Worst of it, it doesn’t provide any user workaround when the Gallery gets things wrong: for example, both Bing and Google offer HTTPS-based searches, and have for years. But if you install their official providers from the IE Gallery, you get insecure search and leak of your keystrokes as you type in the address bar. Microsoft Security Response Center (MSRC) has indicated that they do not consider this a security vulnerability.

In contrast, when AddSearchProvider is supported, the search engine can itself offer the proper, secure, search URLs. Or a user can build their own provider.

Please join me in begging the Internet Explorer team to reconsider: Support freedom. #SupportOpenSearch.

Vote here to fix Spartan: Bug Tracker link

Update: Hours after this post, the April security update for IE broke the AddSearchProvider API in existing IE versions. :-(

-Eric Lawrence

1 The validity of this narrative is itself worthy of its own post, so please don’t bother flaming the comments below.

On Appreciation

Over on Twitter today, there’s a trending hashtag #HonoringWebFolk, started by the very honorable Molly E. Holzschlag. My feed is full of friends, colleagues, acquaintances, and folks I’ve yet to meet expressing their gratitude to “unsung leaders” for their contributions to the web. Of course, some of the folks being honored aren’t exactly unsung, but even Internet famous tends to rate below a C-list Hollywood celebrity:

image

The list of people I should thank is long—from the folks who took a chance on me back before I’d really written anything of note, to the colleagues who challenged me and helped me grow. One of the recurring themes developing as I slowly crank out my Microsoft-years memoir is how unbelievably lucky I am to have had the chance to work with so many fantastic people over the years.

However, when I think about the unsung heroes I’ve worked with, the first name that comes to mind is Ed Praitis. Ed was a developer on the IE Fundamentals team who made many contributions over the years; one of his most important tasks in the IE9 timeframe was the grueling work of improving add-on compatibility as we continually ratcheted up the security and reliability of the browser. As we made fundamental architecture changes along the way, add-ons written in the era of single-process, non-Protected Mode browsers blew up, often in spectacular fashion. Many of these add-ons were broadly used and essential to everyday users, especially in countries like South Korea. While we evangelized the HTML5 future as loudly as we could, we know we’d be unable to get these users to even upgrade to browsers with better standards support if their banks and other critical sites didn’t work on Day 1. As if debugging sites and add-ons written in Korean wasn’t hard enough, most of these add-ons were designed with obfuscation and anti-debugging code in an attempt to foil malware. And yet Ed soldiered on and helped ensure IE9 had a smooth launch, unblocking deployments and helping drive toward a HTML5 future.

More personally, Ed provided unexpected and fruitful encouragement at exactly the right time. As we built IE9, I lamented that the dev team couldn’t muster the resources to fix around a dozen performance bugs in the network cache code. As I explained the changes needed and how important they were, he listened thoughtfully and then quietly noted: “It seems like you understand this stuff pretty well. Why don’t you just fix it yourself?”

I chuckled until I saw he was serious. “But I’m a PM!” I protested, “we don’t check-in code. At least, nothing like this.”

“I’ll review it for you if you want,” he offered.

And this was just the push I needed. Within a few weeks, I checked in my fixes, and this was probably the work I’m proudest of in over a decade at the company… helping save hundreds of millions of users untold billions of seconds in downloading pages.

Unfortunately, I never properly thanked Ed for his encouragement.

Two other Microsoft stories come to mind.

Early one morning, the IE PM team was having some sort of internal team-building event in a nearby building. One of the exercises called for us to form two long lines facing one another, and we did so in the entry hallway, one line to each side of the hall. Before the exercise started, a few of the employees who worked in that building arrived for work. They looked confused at what must have looked like a receiving line of some sort, until one of the PMs (I cannot recall who) started clapping and the rest of us joined in, cheering on the arrivals as if they were rockstars who’d arrived at a venue. They were beaming as they walked off to their offices, doubtless still confused, but certainly happier than when they’d come in. It was random and beautiful, and later I often mused that teams ought to consider doing “Rockstar Hellos” as a morale event.

When I left Microsoft, I sent out the typical farewell mail to my immediate coworkers. This mail was circulated more broadly to various folks and I received many extremely kind farewells from all over the company. There’s nothing quite like getting a personal mail thanking you for specific contributions you’ve made from someone you admire, especially when you didn’t expect that person to even know who you are.

Express your appreciation today, and as often as you can. You never know who you might inspire.

-Eric Lawrence

Authenticode and ClickOnce

On my old IEInternals blog, I posted a fair bit about using Authenticode to sign your programs so that their origins could be identified and to avoid triggering warnings from SmartScreen. My last post on that blog before Microsoft took it away was about using a hardware token to improve security of your certificate’s private key.

Unfortunately, one topic I didn’t cover was the use of Authenticode with ClickOnce applications. Fortunately @RobinDotNet did a deep dive on this topic nearly two years ago and published her results in this two-part series:

A simple summary of these thorough posts: You should use Authenticode to sign both your app’s executable and the setup.exe bootstrapper to help ensure that your ClickOnce invocation isn’t blocked by SmartScreen. This signing is slightly trickier than it sounds because you must sign the files in the proper sequence or the ClickOnce manifest file will contain the wrong hash value. The most reliable place to invoke signcode.exe is in the AfterCompile or BeforePublish steps.

Note: Signing ClickOnce apps is especially confusing because there are so many different things to sign: the manifest, the assemblies, setup.exe, and the executable. When you specify a signing certificate in the project properties, that certificate is used to sign setup.exe (used to install dependencies if the required frameworks aren’t yet installed) and the ClickOnce manifest, but it isn’t used to sign the executable. If you tick the Sign Assembly box, you might expect this to code-sign the executable too, but this doesn’t perform Authenticode signing– it instead applies a strongname to the assemblies. Strongnames aren’t examined by SmartScreen (or much of anything else for that matter).

-Eric Lawrence

How Hard Could It Be

I have an idea for a new reality show; it’ll be called “How Hard Could It Be?” The show will feature everyday people approaching straightforward (but new to them) tasks, for instance, minor home repairs.

For instance, in the kickoff show, a software developer will be faced with filling a gap introduced in his front porch when a professional replaced an exterior door.

gap

To make it more interesting, we’ll have the work done on the highly-visible front porch, and we’ll have the man’s wife be pregnant to ensure a steady flow of family and friends visiting over the next few months.

The show will start with the man rubbing his chin and looking at the gap. “I know”, he’ll say, “I can fill this with concrete. I mean, ‘How hard could it be?'”

We’ll follow our character as he researches by watching assorted videos on the Internet, wherein a wide variety of workaday joes install concrete successfully in a wide variety of projects.

Confirming the ease of his project, he’ll drive over to the local home depot, grab an 80lb bag of concrete, a few stirs, a 5 gallon bucket, and some red concrete dye. That’s right, dye. The dye serves two purposes: One– to attempt to match the final result to the nearby brick and other décor. Two– so that the viewers at home (and any passers-by) can easily see all of the unintended places that the concrete ends up. To up the ante a bit, this project will begin late on Sunday morning (ensuring that only one day can be spared for the work) and we’ll film on a May day in Austin with afternoon temperatures in the mid-90s.

Reading the bag of concrete mix, he’ll scoff as he reads “Do not let concrete dry in your hair” and feel confident that this project is going to be trivial: “How hard could it be, if the package needs to warn the typical user of such things.”

As the afternoon proceeds, we’ll watch our hapless hero increasingly drenched in sweat, as he learns valuable lessons like: ‘Why you don’t want to pour concrete mix without a facemask’, and ‘Why do professionals mix concrete in wheelbarrows instead of buckets.’ A counter on the screen will track the skyrocketing obscenity count as the afternoon proceeds toward dusk.

The show will end with our red-tinted grimy hero and his very-pregnant wife on the porch looking forlornly at the final lumpy, off-color result.

She’ll look on the bright side. “It won’t look so bad after we cover it with the welcome mat.”