fiddler

Firefox and Fiddler – Easier than Ever

In a world where software and systems seem to march inexorably toward complexity, I love it when things get simpler.

Years ago, Firefox required non-obvious configuration changes to even send traffic to Fiddler. Eventually, Mozilla changed their default behavior on Windows to adopt the system’s proxy, meaning that Firefox would automatically use Fiddler when it was attached, just like Chrome, IE, Edge and other major browsers.

Unlike other browsers, Firefox also has its own Trusted Root Certificates store, which means that if you attempt to load a HTTPS page through Firefox with Fiddler’s HTTPS Decryption Mode enabled, you’ll get an error page:

FirefoxMITMDetected
MOZILLA_PKIX_ERROR_MITM_DETECTED error page

To configure Firefox to trust Fiddler’s root certificate, you used to have to manually install it by opening the FiddlerRoot.cer file, ticking the “Trust this CA to identify websites” box, and clicking OK:

FirefoxCA
The old way: Manually trusting Fiddler’s certificate

Making matters more annoying, any time you reset Fiddler’s root certificates (e.g. using the Actions button inside Tools > Fiddler Options > HTTPS), you had to do the whole dance over again. If you wanted to remove the obsolete root certificates, you had to visit a buried configuration UI:

ManualTrustFF
The old way: Administering the Firefox Certificate Store

Yesterday, I was delighted to learn that Firefox added a better option way back in Firefox 49. Simply visit about:config in Firefox and toggle the security.enterprise_roots.enabled setting to true.

FirefoxEnterprise
Enable the new hotness in about:config

After you make this change, Firefox will automatically trust certificates chained to roots in the Windows Trusted Root Certificate store, just like Chrome, Edge, IE and other browsers. Unfortunately, Mozilla has declined to turn this on by default, but it’s still a great option.

 

-Eric

FiddlerCore and Brotli compression

Recently, a developer asked me how to enable Brotli content-compression support in FiddlerCore applications, so that APIs like oSession.GetResponseBodyAsString() work properly when the entity body has been compressed using brotli.

Right now, support requires two steps:

  1. Put brotli.exe (installed by Fiddler or off Github) into a Tools subfolder of the folder containing your application’s executable.
  2. Ensure that the Environment.SpecialFolder.MyDocuments folder exists and contains a FiddlerCore subfolder (e.g. C:\users\username\documents\FiddlerCore).

Step #1 allows FiddlerCore to find brotli.exe. Alternatively, you can set the fiddler.config.path.Tools preference to override the folder.

Step #2 allows FiddlerCore to create necessary temporary files. Sadly, this folder cannot presently be overridden [Bug].

One day, Fiddler might not need brotli.exe any longer, as Brotli compression is making its way into the framework.

 

 

Fiddler And LINQ

Since moving to Google at the beginning of 2016, I’ve gained some perspective about my work on Fiddler over the prior 12+ years. Mostly, I’m happy about what I accomplished, although I’m a bit awed about how much work I put into it, and how big my “little side project” turned out to be.

It’s been interesting to see where the team at Telerik has taken the tool since then. Some things I’m not so psyched about (running the code through an obfuscator has been a source of bugs and annoyance), but the one feature I think is super-cool is support for writing FiddlerScript in C#. That’s a feature I informally supported via an extension, but foolishly (in hindsight) never invested in baking into the tool itself. That’s despite the fact that JScript.NET is a bit of an abomination which is uncomfortable for both proper JavaScript developers and .NET developers. But I digress… C# FiddlerScript is really neat, and even though it may take a bit of effort to port the many existing example FiddlerScript snippets, I think many .NET developers will find it worthwhile.

I’ve long been hesitant about adopting the more fancy features of the modern .NET framework, LINQ key among them. For a while, I justified this as needing Fiddler to work on the bare .NET 2.0 framework, but that excuse is long gone. And I’ll confess, after using LINQ in FiddlerScript, it feels awkward and cumbersome not to.

To use LINQ in FiddlerScript, you must be using the C# scripting engine and you must add System.core.dll inside Tools > Fiddler Options > Scripting. Then, add using System.Linq; to the top of your C# script file.

After you make these changes, you can do things like:

    var arrSess = FiddlerApplication.UI.GetAllSessions();
    bool b = arrSess.Any(s=>s.HostnameIs("Example.com"));
    FiddlerApplication.UI.SetStatusText((b) ? "Found it!":"Didn't find it.");

-Eric Lawrence

Using Fiddler With iOS 10 and Android 7

If you’ve tried to use Fiddler with iOS10 beta or Android 7 Nougat, you have probably found that HTTPS decryption isn’t working, even if you use the latest Fiddler and the Fiddler Certificate Maker add-on. Unfortunately, at the moment both platforms are broken, but for different reasons. In both cases, the client will fail to receive responses for HTTPS requests, and Fiddler will only show a CONNECT tunnel.

iOS 10 Change

After installing the FiddlerRoot certificate, one also needs to go to Settings -> General -> About -> Certificate Trust Settings and manually enable full trust for the FiddlerRoot root certificate, including accepting a dialog that says that this will allow a third-party to eavesdrop on all your communications.

iOS 10 Beta Bug (Fixed for final version)

The beta of iOS 10 had a bug whereby, if the response to a HTTP CONNECT tunnel request contains a Connection: close response header, the client will close the connection instead of doing as it should and waiting until the TCP/IP connection closes. A few minor platforms have had the same bug over the years, but iOS is definitely the first important platform with this issue. At least two bugs have been filed with the Apple “Radar” bug reporter.

Working around this limitation is simple. In Fiddler, click Rules > Customize Rules. Scroll to the OnBeforeResponse function. Just inside that function, add the following lines:

  if (oSession.HTTPMethodIs("CONNECT")) {
oSession["ui-backcolor"] = "red";
oSession.ResponseHeaders.Remove("Connection");
}

Save the file and try connecting again.

Android 7 Feature

In contrast to the iOS regression, the change in Android 7 was intentional. The Android team has decided that, by default, HTTPS certificate validation for apps targeting API Level 24 and later will ignore all user-installed root certificates, meaning that your efforts to manually trust Fiddler’s root certificate will be fruitless. Individually application developers can temporarily override this change while debugging by updating the application’s configuration:

<network-security-config>  
      <debug-overrides>  
           <trust-anchors>  
                <!-- Trust user added CAs while debuggable only -->
                <certificates src="user" />  
           </trust-anchors>  
      </debug-config>  
 </network-security-config>

…or at all times…

<network-security-config>  
      <base-config>  
            <trust-anchors>  
                <!-- Trust preinstalled CAs -->  
                <certificates src="system" />  
                <!-- Additionally trust user added CAs -->  
                <certificates src="user" />  
           </trust-anchors>  
      </base-config>  
 </network-security-config>

…Unfortunately, these changes can only be undertaken by application developers and not end-users. End-users will probably need to jailbreak their devices, akin to what is required to circumvent certificate pinning.

Certificate Validity Length

By default, Fiddler-generated certificates are valid for five years (and backdated one year). However, this can cause an ERR_CERT_VALIDITY_TOO_LONG error in Chrome on Android.  To fix this for Fiddler’s default (CertEnroll) certificate generator,  run about:config in QuickExec to edit preferences. Set fiddler.certmaker.ValidDays to 820 and, if needed, reset your certificates in Fiddler using the Tools > Fiddler Options > HTTPS > Actions button.

-Eric

Silliness – Fiddler Blocks Malware

Enough malware researchers now depend upon Fiddler that some bad guys won’t even try to infect your system if you have Fiddler installed.

The Malware Bytes blog post has the details, but the gist of it is that the attackers use JavaScript to probe the would-be victim’s PC for a variety of software. Beyond Kaspersky, TrendMicro, and MBAM security software, the fingerprinting script also checks for VirtualBox, Parallels, VMWare, and Fiddler. If any of these programs are thought to be installed, the exploit attempt is abandoned and the would-be victim is skipped.

This isn’t the only malware we’ve seen hiding from Fiddler—earlier attempts use tricks to see whether Fiddler is actively running and intercepting traffic and only abandon the exploit if it is.

This behavior is, of course, pretty silly. But it makes me happy anyway.

Preventing Detection of Fiddler

Malware researchers who want to help ensure Fiddler cannot be detected by bad guys should take the following steps:

  1. Do not put Fiddler directly on the “victim” machine.
    – If you must, at least install it to a non-default path.
  2. Instead, run Fiddler as a proxy server external to the victim machine (either use a different physical machine or a VM).
    – Tick the Tools > Fiddler Options > Connections > Allow remote computers to connect checkbox. Restart Fiddler and ensure the machine’s firewall allows inbound traffic to port 8888.
    – Point the victim’s proxy settings at the remote Fiddler instance.
    – Visit http://fiddlerserverIP:8888/ from the victim and install the Fiddler root certificate
  3. Click Rules > Customize Rules and update the FiddlerScript so that the OnReturningError function wipes the response headers and body and replaces them with non-descript strings. Some fingerprinting JavaScript will generate bogus AJAX requests and then scan the response to see whether there are signs of Fiddler.

-Eric

Automatically Evaluating Compressibility

Fiddler’s Transformer tab has long been a simple way to examine the use of HTTP compression of web assets, especially as new compression engines (like Zopfli) and compression formats (like Brotli) arose. However, the one-Session-at-a-time design of the Transformer tab means it is cumbersome to use to evaluate the compressibility of an entire page or series of pages.

Introducing Compressibility

Compressibility is a new Fiddler 4 add-on1 which allows you to easily find opportunities for compression savings across your entire site. Each resource dropped on the compressibility tab is recompressed using several compression algorithms and formats, and the resulting file sizes are recorded:

Compressibility tab

You can select multiple resources to see the aggregate savings:

Total savings text

WebP savings are only computed for PNG and JPEG images; Zopfli savings for PNG files are computed by using the PNGDistill tool rather than just using Zopfli directly. Zopfli is usable by all browsers (as it is only a high-efficiency encoder for Deflate) while WebP is supported only by Chrome and Opera. Brotli is available in Chrome and Firefox, but limited to use from HTTPS origins.

Download the Addon…

To show the Compressibility tab, simply install the add-on, restart Fiddler, and choose Compressibility from the View > Tabs menu2.

View > Tabs > Compressibility menu screenshot

The extension also adds ToWebP Lossless and ToWebP Lossy commands to the ImageView Inspector’s context menu:

ImagesMenuExt

I hope you find this new addon useful; please send me your feedback so I can enhance it in future updates!

-Eric

1 Note: Compressibility requires Fiddler 4, because there’s really no good reason to use Fiddler 2 any longer, and Fiddler 4 resolves a number of problems and offers extension developers the ability to utilize newer framework classes.

2 If you love Compressibility so much that you want it to be shown in the list of tabs by default, type prefs set extensions.Compressibility.AlwaysOn true in Fiddler’s QuickExec box and hit enter.