Problems in accessing websites can often be found and fixed if the network traffic between the browser and the website is captured as the problem occurs. This short post explains how to capture such logs.
Capturing Network Traffic Logs
If someone asked you to read this post, chances are good that you were asked to capture a web traffic log to track down a bug in a website or your web browser.
Fortunately, in Google Chrome or the new Microsoft Edge (version 76+), capturing traffic is simple:
- Optional but helpful: Close all browser tabs but one.
- Navigate the tab to about://net-export
- In the UI that appears, press the Start Logging to Disk button.
- Choose a filename to save the traffic to. Tip: Pick a location you can easily find later, like your Desktop.
- Reproduce the networking problem in a new tab. If you close or navigate the //net-export tab, the logging will stop automatically.
- After reproducing the problem, press the Stop Logging button.
- Share the Net-Export-Log.json file with whomever will be looking at it. Optional: If the resulting file is very large, you can compress it to a ZIP file.
In some cases, especially when you dealing with a problem in logging into a website, you may need to set either the
Include cookies and credentials or
Include raw bytes options before you click the Start Logging button.
Note that there are important security & privacy implications to selecting these options– if you do so, your capture file will almost certainly contain private data that would allow a bad actor to steal your accounts or perform other malicious actions. Share the capture only with a person you trust and do not post it on the Internet in a public forum.
If you’re more of a visual learner, here’s a short video demonstrating the traffic capture process.
If you use Edge’s “Recreate my problem” button on the Feedback Wizard, the feedback tool will capture and include a network trace (in “include cookies and credentials” mode) as a part of your feedback report.
This method is the easiest way to get a network trace to Microsoft: the JSON is transmitted and stored securely without you having to find a way to encrypt and transfer the data. However, this method is inflexible: it does not allow you to send your traffic log to a friend or attach it to a bug in the Chromium bug database, and it does not expose the option to “include raw bytes.”
Analyzing Traffic Logs
In a followup post, I explore how developers can analyze captured traffic.
Thanks for your help in capturing network logs to diagnose and fix problems!
Appendix A: Capture on Startup
In rare cases, you may need to capture network data early (e.g. to capture proxy script downloads and the like. To do that, close Edge, then run:
If you want to capture unsanitized cookies and authentication headers, but not the response bodies, use
--net-log-capture-mode=IncludeSensitive instead. Omit the final parameter entirely if you do not want to include the raw data and want just the “Strip Private Information” mode of capture.
Appendix A.1: Capturing Electron and WebView2
Note: The command line argument approach also works for Electron JS applications like Microsoft Teams:
Note: This will only capture the network traffic from the Chromium layer of Electron apps (e.g. web requests from the nodeJS side will not be captured) but it still may be very useful.
WebView2-based applications can either pass the
--log-net-log command line into the WebView2 to initiate the capture, or they can add a second WebView control to their application (in the same context) and navigate it to
about:net-export to allow the debugging user to manually trigger logging.
Appendix B: Mobile
The Net-Export feature works great on Android, but take care to ensure that you switch tabs to perform your repro after starting your capture– tab switching is less obvious on Android.
On mobile, when the capture completes, the resulting file is offered to be sent via email (because the mobile file system is not very accessible).
Unfortuately, on iOS, the Network Export feature is somewhat unlikely to contain the data you need because the capture contains only the data sent by Chromium’s network stack, not the web content traffic (HTML, JS, CSS, images, etc) used inside the WkWebView control (embedded Safari). To capture data from the entire browser on iOS, you’ll need to use another approach, e.g. Telerik Fiddler.
Appendix C: Limitations
No POST Data
One important shortcoming in the current NetLog file format is that it does not contain any request body data, even if you select the “Include Raw Bytes” option. If you need the request body data, you may need to collect a HTTP Archive (HAR) file instead.
- Hit F12 to open the Developer Tools.
- Activate the Network tab.
- Ensure the recording button at the top of the tab is red
- Tick the Preserve log checkbox.
- Reproduce the problem
- Right-click entries in the the grid and choose Save all as HAR with content
- Share the HAR file only with a person you trust and do not post it on the Internet in a public forum.
Alternatively, you might just capture the traffic using Fiddler.
Can’t Capture Requests that Don’t Reach the Network Stack
Perhaps surprisingly, the browser’s ServiceWorker feature lives above the network stack, so if the browser issues requests that are satisfied locally by the ServiceWorker, that “traffic” is not seen in the NetLog. (
fetch requests that are sent from the ServiceWorker to the Network will appear in the log, however.) To see requests that are satisfied by the ServiceWorker, use the F12 Developer Tools.
Similar to the ServiceWorker case, the Blink engine has a memory cache for content that can be reused within a page. Certain requests (e.g. if there are ten image tags all pointed at the same URL) may be satisfied by this cache without sending the request down to the network stack.
Can’t Capture IE Mode
Pages running in Edge’s IE Mode tabs are loaded using URLMon and WinINET, the Windows Network Stacks used by Internet Explorer. Because this traffic does not go through the Chromium Network Stack, it is not recorded in NetLogs.
To work around this problem, you’re probably best off just capturing the traffic using Fiddler.