Demystifying Browsers

Last update: June 18, 2020

I started building browser extensions more than 22 years ago, and I started building browsers directly just over 16 years ago. At this point, I think it’s fair to say that I’m entering the grizzled veteran phase of my career.

With the Edge team continuing to grow with bright young minds from college and industry, I’m increasingly often asked “Where do I learn about browsers?” and I haven’t had a ready answer for that question.

This post aims to answer it.

First, a few prerequisites for developing expertise in browsers:

  1. Curiosity. While browsers are more complicated than ever, there are also better resources than ever to learn how they work. All major browsers are now based on open-source code, and if you’re curious, you no longer need to join a secret priesthood to discover how they operate under the hood.
  2. Willingness to Experiment. Considering how complex browsers are (and because they’re so diverse, across platforms, maker, and version), it’s often easiest to definitively answer questions about how browsers work by trying things, rather than reading an explainer (possibly outdated or a map that doesn’t match the terrain) or reading the code (often complex and potentially misleading). Build test cases and try them in each browser to see what happens. When you encounter surprising behavior, let your curiosity guide you into figuring it out. Browsers contain no magic, but plenty of butterfly effects.
  3. Doggedness. I’ve been doing this for half of my life, and I’m still learning daily. While historical knowledge will serve you well, things are changing in this space every day, and keeping up is an endless challenge. And it’s often fun.

Now, how do you apply these prerequisites and grow to become a master of browsers? Read on.

Fundamental Understanding

Over the years, a variety of broad resources have been developed that will give you a good foundation in the fundamentals of how browsers work. Taking advantage of these will help you more effectively explore and learn on your own.

  • First, I recommend reading the Chrome Comic Book. This short, 38 page comic book from legend Scott McCloud was published alongside the first version of Google Chrome back in 2008. It clearly and simply explains many of the core concepts behind modern browsers as application platforms.
  • HTML5Rocks has a great introduction into How Browsers Work. This is a lengthy and detailed introduction into how browsers turn HTML and CSS into what you see on the screen. Read this article and you’ll understand more about this topic than 90% of web developers.
  • The folks at Google have created a fantastic four-part illustrated series about how modern browsers work: Inside look at modern web browsers. Navigation, the Rendering Engine and Input and Compositing as a part of their Web Fundamentals site.
  • Mozilla wrote a fantastic cartoon introduction to WebAssembly, explaining the basics behind this new technology; there’s tons of other invaluable content on Mozilla Hacks.
  • The Chromium Chronicle is a monthly series geared specifically to the Chromium developers who build the browser.
  • Web Developers should check out Web.Dev, a great source of articles on building fast and secure websites.

Books

If you prefer to learn from books, I can only recommend a few. Sadly, there are few on browsers themselves (largely because they tend to evolve too quickly), but there are good books on web technologies.

Tools

One of the best ways to examine what’s going on with browsers is to just use tools to watch what’s going on as you use your favorite websites.

  • Built-in DevTools (just hit F12!) – They’re amazingly powerful. I don’t know of a great tutorial, but there are likely some on YouTube.
  • Telerik Fiddler – See what requests hit the network and what they contain.
  • The VisBug Chrome Extension – Easily manipulate any page layout, directly in your browser.

Use the Source, Leia

The fact that all of the major browsers are built atop open-source projects is a wonderful thing. No longer do you need to be a reverse-engineering ninja with a low-level debugger to figure out how things are meant to work (although sometimes such approaches can still be super-valuable).

Source code locations:

Navigating the Code

While simply perusing a browser’s source code might give you a good feel for the project, browsers tend to be enormous. Chromium is over 10 million lines of code, for example.

If you need to find something in particular, one often effective way to find it easily is to search for a string shown in the browser UI near the feature of interest. (Or, if you’re searching for a DOM function name or HTML attribute name, try searching for that.) We might call this method string chasing.

By way of example, today I encountered an unexpected behavior in the handling of the “Go to <url>” command on Chromium’s context menu:

So, to find the code that implements this feature, I first try searching for that string:

…but there are a gazillion hits, which makes it hard to find what I need. So I instead search for a string that’s elsewhere in the context menu, and find only one hit in the Chromium “grd” (resources) file:

When I go look at that grd file, I quickly find the identifier I’m really looking for just below my search result:

So, we now know that we’re looking for usages of IDS_CONTENT_CONTEXT_GOTOURL, probably in a .CC file, and we find that almost immediately:

From here, we see that the menu item has the command identifier IDC_CONTENT_CONTEXT_GOTOURL, which we can then continue to chase down through the source until we find the code that handles the command. That command makes use of a variable selection_navigation_url_, which is filled elsewhere by some pretty complicated logic.

After you gain experience in the Chromium code, you might learn “Oh, yeah, all of the context menu stuff is easy to find, it’s in the renderer_context_menu directory” and limit your searches to that area, but after four years of working on Chrome, I still usually start my searches broadly.

Optional: Compile the code

If you’d actually like to compile the code of a major browser, things are a bit more involved, but if you follow the configuration instructions to the letter— your first build will succeed. Back in 2015, Monica Dinculescu created an amazing illustrated guide to contributing to Chromium.

You can compile Chromium or Firefox on a mid-range machine from 2016, but it will take quite a long time. A beefy PC will speed things up a bunch, but until we have cloud compilers available to the public, it’s always going to be pretty slow.

Look at their Bugs

All browsers except Microsoft Edge have a public bug tracker where you can search for known issues and file new bugs if you encounter them.

  • FirefoxFirefox Bugzilla
  • WebkitWebKit Bugzilla
  • ChromiumCRBug
  • Microsoft Edge‘s – Platform bugs that are inherited from Chromium are tracked using CRBug. Sadly, at present there is no public tracker for bugs that reproduce only in Edge. Bugs reported by the “Feedback” button are tracked internally by Microsoft.
  • Braveon GitHub
  • HTML5 Specification – on GitHub

Here are instructions for filing a great browser bug.

Blogs to Read

  • This One – I write mostly about browsers.
  • My (archived) IEInternals – I started writing this blog because it was the only reliable way for me to find my notes from investigations and troubleshooting years later.
  • Cloudflare’s – Cloudflare is a $5B company whose primary product is their amazing blog. I understand they also run a CDN on the side to generate interesting topics for their blog to talk about.
  • Nasko Oskov’s – Nasko is an engineer on the Chrome Security team and writes mostly about security topics.
  • Chris Palmer’s – Chris is an engineer on the Chrome Security team and writes about secure design.
  • Adam Langley’s – Google’s expert cryptographer
  • Bruce Dawson’s – Bruce is a Chrome Engineer who posts lots of interesting information about debugging and performance troubleshooting, especially on Windows.
  • Anne van Kesteren’s – Anne works on the HTML5 spec.
  • Mark Nottingham’s – Mark co-chairs the HTTP and QUIC working groups

Specific Posts of Interest

People to Follow

I’ve doubtless forgotten some, see who I follow.

The Business of Browsers

Public data reveals each point of marketshare in the browser market is worth at least $100,000,000 USD annually (most directly in the form of payments from the browser’s configured search engine).

Remembering this fact will help you understand many other things, from how browsers pay their large teams of expensive software engineers, to how they manage to give browsers away for free, to why certain features behave the way that they do.

Extra Resources

Browsers are hugely complicated beasts, and tons of fun. If the resources above leave you feeling both overwhelmed and excited, maybe you should become a browser builder.

Want to change the world? Come join the new Microsoft Edge team today!

-Eric

Published by ericlaw

Impatient optimist. Dad. Author/speaker. Created Fiddler & SlickRun. PM @ MSFT '01-'12, and '18-, presently working on Microsoft Edge. My words are my own.

One thought on “Demystifying Browsers

  1. This is an awesome list of resources (I wasn’t surprised that many of the Twitter handles were ones I have been following for years).

    As someone that wants to be able to dig in and know exactly what is happening for every page load I love things like the window.performance[.timing] API https://www.w3.org/TR/navigation-timing/#processing-model to inspect where time is being spent from the initial browser request for a page through the fully rendered response. Much of this I could determine through Fiddler and the browser dev tools network tab, but I’ve now opened up Pandora’s box with a tool I wrote sitting on top of the window.performance API. I’m getting great data to find and fix the “hot spots” for slowness but I also noticed there are chunks of “stall time” that vary and I’m wondering if (A) browsers intend to provide more granularity into this timeline and/or (B) if there are details about what happens in these gaps.

    e.g. between the TCP “connectEnd” and the Request “requestStart”… this seems to vary on sites I inspect from 0ms to well over 500ms. Ditto for any delays between DNS “domainLookupEnd” and TCP “connectStart”.

    I’d love to *demystify* what happens in these spots (*and more importantly*) is there anything I can do to reduce these gaps?

    PS I understand that the performance.timing is technically deprecated: https://developer.mozilla.org/en-US/docs/Web/API/Performance/timing I’m hoping to be able to use other options to get fine-grained info wherever possible… the performance.timing object just happens to neatly wrap up all of these properties for me. ;-)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s