I’ve written about File Downloads quite a bit, and early this year, I delivered a full tech talk on the topic.
From my very first days online (a local BBS via 14.4 modem, circa 1994), I spent decades longing for faster downloads. Nowadays, I have gigabit fiber at the house, so it’s basically never my connection that is the weak link. However, when I travel, I sometimes find myself on slow WiFi connections (my current hotel is 3mbps) and am reminded of the performance pains of yesteryear.
Over the years, various schemes have been concocted for faster downloads; the most effective concern sending fewer bytes (typically by using compression). Newer protocols (e.g. H3’s QUIC) are able to offer some improvements even without changing the number of bytes transferred.
For the last twenty years or so, one performance idea has hung around at the periphery: “What if we used multiple connections to perform a single download?” You can see this Parallel downloading option today inside Chrome and Edge’s about:flags page:
This is an “Enthusiast” feature, but does it really improve download performance?
Mechanics: How Parallel Downloads Work
Parallel downloads work by using the Range Request mechanism in HTTP that allows a client to request a particular range (portion) of a file. This mechanism is most commonly used to resume incomplete downloads.
For example, say you went offline after downloading the first 3.6mb of a file. When you get back online, the browser can request that the server send only the remainder of the file by sending a Range header that specifies that it wants bytes #3,600,000 and later, like so:
The If-Range request header supplies the ETag received on the first part of the response; if the server can send the requested range of that original response, it will return a HTTP/206 Partial Content response. If the server cannot return the requested portion of the original resource, it will either return a HTTP/200 with the entire file, or a HTTP/416 Requested Range Not Satisfiable error.
The Range Request mechanism is also used for retrieving specific portions of media in other scenarios where that makes sense: imagine the user skips ten minutes ahead in a long web video, or jumps down 50 pages in a PDF file. There’s no point in wasting data transfer or time downloading the part that the user skipped over.
The Parallel Downloads feature uses the same mechanism. Instead of downloading just one range at a time, the browser establishes, say, 4 simultaneous connections to the server and downloads 25% of the file on each connection.
Limitations
Notably, not every server supports Range requests. A server is meant to indicate its support via the Accept-Ranges response header, but the client only gets this header after it gets its first response.
Even if a server does support Range requests, not every response supports ranges. For example, the server may not support returning a specific range for a dynamically-generated download because the bytes of that download are no longer available after the first connection. A client that wishes to support parallel downloads must carefully probe for support and correctly handle all of the possible error cases without breaking the download process.
Finally, standards dictate that browsers should limit themselves to 6 connections per HTTP/1.1 server, so using multiple connections for a single download could cause problems with interacting with a server if multiple connections are being used to download a single file.
Are Parallel Downloads Faster?
From a theoretical point-of-view, parallel downloads should never be faster. The performance overhead of establishing additional connections:
- TCP/IP connection establishment requires extra roundtrips
- The TCP/IP congestion-controlling behavior called “Slow start” that throttles newly-established connections
- HTTPS connections require relatively expensive TLS handshakes
… means that performing downloads across multiple parallel connections should never be faster than using a single connection, and it should usually be slower.
Now, having said that, there are two cases where performing downloads in parallel can be faster:
First, there exist file download servers that deliberately throttle download speeds for “free” customers. For example, many file download services (often used for downloading pirated content, etc) throttle download speeds to, say, 100kbps to entice users to upgrade to a “pay” option. By establishing multiple parallel connections, a user may be able to circumvent the throttle and download at, say, 400kbps total across 4 parallel connections. This trick only works if the service is oblivious to this hack; smarter servers will either not support Range requests, reject additional connections, or divide the cap across all of a user’s connections.
Next, HTTP/1 and HTTP2 are based on TCP/IP, and TCP/IP has a “head-of-line” blocking problem that can cause slowness for spotty network connections wherein a dropped packet can cause the stream to stall. If a download is conducted across multiple parallel connections, the impact is muted because each TCP/IP connection stalls independently. A parallel download could conceivably still make progress on the other connections’ chunks until the stalled connection recovers. HTTP3 does not have this problem because it is based on QUIC, and UDP does not suffer from the head-of-line blocking problem.
In some locales, support for Parallel Downloading may be an important competitive feature. In India and China specifically, connection quality is somewhat lower, making it somewhat more likely users will encounter head-of-line blocking in TCP/IP. However, I believe that the main reason that Chromium offers this feature is that niche browsers in those markets trumpet their support for Parallel Downloads in their marketing materials, so natively offering this feature in Chromium is important from a competitive marketing point-of-view.
Update: Edge 148
In Edge 148, parallel downloads are enabled by default. For example, when downloading a giant 24 gigabyte file, Edge now makes three parallel HTTP requests for the content: the initial stream, and then two Range requests starting at 8GB and 16GB into the file:
Neat, right? Well…
Over on Reddit, there are some complaints that huge downloads are now behaving weirdly in Edge. Specifically, after a download is initiated, it takes a very long time for the UI to show any progress. What’s going on?
A look at the Process Monitor log shows what’s happening. When a download starts, Edge creates a .crdownload file to download the content into. In a normal download, this destination file starts at 0 bytes long and grows as downloaded data is read from the network and written into the destination file on disk.
In a multi-segment download, however, there’s a tricky issue: we now have three streams writing content into the target file at different offsets (the beginning, middle, and end). For those parallel writes to work, the destination file has to be long enough that all of the offsets being written to are within the size of the file. So, in this case, we first have to write 16 gigabytes to the disk in order for the 3rd download stream (which is starting 16gb into the download) to write the content it’s reading off the network onto disk:
Aside: What we’re describing here, a large file in which only some bytes are filled, is formally known as a sparse file. Several Windows filesystems, including NTFS, support SPARSE files, but it’s not supported for FAT and exFAT, and support for sparse files over SMB is not universal.
Depending on the location of that destination file, growing the file up to that 16 gigabytes might take quite a long time. That’s especially true if the download destination is on a spinning rust HDD, a flash drive, or a remote disk.
And thus arises the UX problem: the browser UI wasn’t written with a “Hold on a sec, reserving a ginormous file” state. So, instead, it just kinda … sits there. If you’ve configured Edge to ask where to save each download, after you pick a location, the UI doesn’t switch over to show the progress, instead just leaving the buttons there while it silently chugs away in the background, working its little heart out to get the destination file sized:
If you instead click “Save”, you might see the UI showing the download, but without the progress update moving at all:
What’s even wackier is how messed up the system gets into while it’s working as fast as it can to grow the destination file. For instance, you might try closing the browser that doesn’t seem to be working, but the browser process keeps running even after its window is gone. Trying to kill the hidden process with the outstanding I/O requests doesn’t work. You can see my flash drive maxxed out while getting written to by a process that the system claims “isn’t running”:
While ultimately the overall time for the download should be about the same (the browser is doing work, mind you, just not telling you about it), if this glitchy UX bothers you, disable parallel downloads explicitly by visiting edge://flags/#enable-parallel-downloading and setting the toggle to DISABLED.
It looks like this problem was reported to upstream Chromium back in 2021 but the reproduction scenario was not understood at the time.
text/plain 