browsers, fiddler

Firefox and Fiddler – Easier than Ever

In a world where software and systems seem to march inexorably toward complexity, I love it when things get simpler.

Years ago, Firefox required non-obvious configuration changes to even send traffic to Fiddler. Eventually, Mozilla changed their default behavior on Windows to adopt the system’s proxy, meaning that Firefox would automatically use Fiddler when it was attached, just like Chrome, IE, Edge and other major browsers.

Unlike other browsers, Firefox also has its own Trusted Root Certificates store, which means that if you attempt to load a HTTPS page through Firefox with Fiddler’s HTTPS Decryption Mode enabled, you’ll get an error page:

FirefoxMITMDetected
MOZILLA_PKIX_ERROR_MITM_DETECTED error page

To configure Firefox to trust Fiddler’s root certificate, you used to have to manually install it by opening the FiddlerRoot.cer file, ticking the “Trust this CA to identify websites” box, and clicking OK:

FirefoxCA
The old way: Manually trusting Fiddler’s certificate

Making matters more annoying, any time you reset Fiddler’s root certificates (e.g. using the Actions button inside Tools > Fiddler Options > HTTPS), you had to do the whole dance over again. If you wanted to remove the obsolete root certificates, you had to visit a buried configuration UI:

ManualTrustFF
The old way: Administering the Firefox Certificate Store

Yesterday, I was delighted to learn that Firefox added a better option way back in Firefox 49. Simply visit about:config in Firefox and toggle the security.enterprise_roots.enabled setting to true.

FirefoxEnterprise
Enable the new hotness in about:config

After you make this change, Firefox will automatically trust certificates chained to roots in the Windows Trusted Root Certificate store, just like Chrome, Edge, IE and other browsers. Unfortunately, Mozilla has declined to turn this on by default, but it’s still a great option.

 

-Eric

Standard
dev, fiddler

Fiddler And LINQ

Since moving to Google at the beginning of 2016, I’ve gained some perspective about my work on Fiddler over the prior 12+ years. Mostly, I’m happy about what I accomplished, although I’m a bit awed about how much work I put into it, and how big my “little side project” turned out to be.

It’s been interesting to see where the team at Telerik has taken the tool since then. Some things I’m not so psyched about (running the code through an obfuscator has been a source of bugs and annoyance), but the one feature I think is super-cool is support for writing FiddlerScript in C#. That’s a feature I informally supported via an extension, but foolishly (in hindsight) never invested in baking into the tool itself. That’s despite the fact that JScript.NET is a bit of an abomination which is uncomfortable for both proper JavaScript developers and .NET developers. But I digress… C# FiddlerScript is really neat, and even though it may take a bit of effort to port the many existing example FiddlerScript snippets, I think many .NET developers will find it worthwhile.

I’ve long been hesitant about adopting the more fancy features of the modern .NET framework, LINQ key among them. For a while, I justified this as needing Fiddler to work on the bare .NET 2.0 framework, but that excuse is long gone. And I’ll confess, after using LINQ in FiddlerScript, it feels awkward and cumbersome not to.

To use LINQ in FiddlerScript, you must be using the C# scripting engine and you must add System.core.dll inside Tools > Fiddler Options > Scripting. Then, add using System.Linq; to the top of your C# script file.

After you make these changes, you can do things like:

    var arrSess = FiddlerApplication.UI.GetAllSessions();
    bool b = arrSess.Any(s=>s.HostnameIs("Example.com"));
    FiddlerApplication.UI.SetStatusText((b) ? "Found it!":"Didn't find it.");

-Eric Lawrence

Standard
dev, security

Chrome Deprecates Subject CN Matching

If you’re using a Self-Signed certificate for your HTTPS server, a deprecation coming to Chrome may affect your workflow.

Chrome 58 will require that certificates specify the hostname(s) to which they apply in the SubjectAltName field; values in the Subject field will be ignored. This follows a similar change in Firefox 48. If impacted, you’ll see something like this blocking page as you load your HTTPS site:

NET::ERR_CERT_COMMON_NAME_INVALID blocking page in Chrome

NET::ERR_CERT_COMMON_NAME_INVALID is an unfortunate error code, insofar as all common names are now ignored. Chrome is working to improve the debuggability of this experience, via:

Update: Both of these have landed. Chrome now shows [missing_subjectAltName] in the details on the Certificate Error page, and a Subject Alternative Name Missing warning in the Security panel of the Developer tools.

Notably, Windows’ ancient makecert.exe utility cannot set the SubjectAltName field in certificates, which means that if you’re using it to generate your self-signed certificates, you need to stop. Instead, users of modern Windows can use the New-SelfSignedCertificate command in PowerShell.

New-SelfSignedCertificate -DnsName "www.example.com", "example.com" -CertStoreLocation "cert:\CurrentUser\My"

Using openssl for self-signed certificate generation? See https://stackoverflow.com/a/43860138.

This new restriction may also impact users of very old versions of Fiddler (or FiddlerCore), or users who have configured Fiddler to use MakeCert for whatever reason. Fortunately, Fiddler offers a number of different certificate generators, so you just need to make a small configuration change. To switch away from MakeCert, click Tools > Fiddler Options > HTTPS and click the “Certificates generated by MakeCert engine” link. Change the dropdown to CertEnroll and click OK. Click Actions > Reset All Certificates and restart Fiddler.

image

If you’re building an application atop FiddlerCore, you’ll need to make sure you’re not using makecert; see the end of this post for help.

-Eric Lawrence

PS: There’s also a EnableCommonNameFallbackForLocalAnchors policy. You shouldn’t use it and you should just fix your certificates, or they’ll break when it’s removed in Chrome 65 or earlier.

PS: There’s now official documentation on this topic.

Standard
fiddler

Using Fiddler With iOS 10 and Android 7

If you’ve tried to use Fiddler with iOS10 beta or Android 7 Nougat, you have probably found that HTTPS decryption isn’t working, even if you use the latest Fiddler and the Fiddler Certificate Maker add-on. Unfortunately, at the moment both platforms are broken, but for different reasons. In both cases, the client will fail to receive responses for HTTPS requests, and Fiddler will only show a CONNECT tunnel.

iOS 10 Change

After installing the FiddlerRoot certificate, one also needs to go to Settings -> General -> About -> Certificate Trust Settings and manually enable full trust for the FiddlerRoot root certificate, including accepting a dialog that says that this will allow a third-party to eavesdrop on all your communications.

iOS 10 Beta Bug (Fixed for final version)

The beta of iOS 10 had a bug whereby, if the response to a HTTP CONNECT tunnel request contains a Connection: close response header, the client will close the connection instead of doing as it should and waiting until the TCP/IP connection closes. A few minor platforms have had the same bug over the years, but iOS is definitely the first important platform with this issue. At least two bugs have been filed with the Apple “Radar” bug reporter.

Working around this limitation is simple. In Fiddler, click Rules > Customize Rules. Scroll to the OnBeforeResponse function. Just inside that function, add the following lines:

  if (oSession.HTTPMethodIs("CONNECT")) {
oSession["ui-backcolor"] = "red";
oSession.ResponseHeaders.Remove("Connection");
}

Save the file and try connecting again.

Android 7 Feature

In contrast to the iOS regression, the change in Android 7 was intentional. The Android team has decided that, by default, HTTPS certificate validation for apps targeting API Level 24 and later will ignore all user-installed root certificates, meaning that your efforts to manually trust Fiddler’s root certificate will be fruitless. Individually application developers can temporarily override this change while debugging by updating the application’s configuration:

<network-security-config>  
      <debug-overrides>  
           <trust-anchors>  
                <!-- Trust user added CAs while debuggable only -->
                <certificates src="user" />  
           </trust-anchors>  
      </debug-config>  
 </network-security-config>

…or at all times…

<network-security-config>  
      <base-config>  
            <trust-anchors>  
                <!-- Trust preinstalled CAs -->  
                <certificates src="system" />  
                <!-- Additionally trust user added CAs -->  
                <certificates src="user" />  
           </trust-anchors>  
      </base-config>  
 </network-security-config>

…Unfortunately, these changes can only be undertaken by application developers and not end-users. End-users will probably need to jailbreak their devices, akin to what is required to circumvent certificate pinning.

Certificate Validity Length

By default, Fiddler-generated certificates are valid for five years (and backdated one year). However, this can cause an ERR_CERT_VALIDITY_TOO_LONG error in Chrome on Android.  To fix this for Fiddler’s default (CertEnroll) certificate generator,  run about:config in QuickExec to edit preferences. Set fiddler.certmaker.ValidDays to 820 and, if needed, reset your certificates in Fiddler using the Tools > Fiddler Options > HTTPS > Actions button.

-Eric

Standard
fiddler, security

Silliness – Fiddler Blocks Malware

Enough malware researchers now depend upon Fiddler that some bad guys won’t even try to infect your system if you have Fiddler installed.

The Malware Bytes blog post has the details, but the gist of it is that the attackers use JavaScript to probe the would-be victim’s PC for a variety of software. Beyond Kaspersky, TrendMicro, and MBAM security software, the fingerprinting script also checks for VirtualBox, Parallels, VMWare, and Fiddler. If any of these programs are thought to be installed, the exploit attempt is abandoned and the would-be victim is skipped.

This isn’t the only malware we’ve seen hiding from Fiddler—earlier attempts use tricks to see whether Fiddler is actively running and intercepting traffic and only abandon the exploit if it is.

This behavior is, of course, pretty silly. But it makes me happy anyway.

Preventing Detection of Fiddler

Malware researchers who want to help ensure Fiddler cannot be detected by bad guys should take the following steps:

  1. Do not put Fiddler directly on the “victim” machine.
    – If you must, at least install it to a non-default path.
  2. Instead, run Fiddler as a proxy server external to the victim machine (either use a different physical machine or a VM).
    – Tick the Tools > Fiddler Options > Connections > Allow remote computers to connect checkbox. Restart Fiddler and ensure the machine’s firewall allows inbound traffic to port 8888.
    – Point the victim’s proxy settings at the remote Fiddler instance.
    – Visit http://fiddlerserverIP:8888/ from the victim and install the Fiddler root certificate
  3. Click Rules > Customize Rules and update the FiddlerScript so that the OnReturningError function wipes the response headers and body and replaces them with non-descript strings. Some fingerprinting JavaScript will generate bogus AJAX requests and then scan the response to see whether there are signs of Fiddler.

-Eric

Standard
browsers

On Daylight Savings Time

In Fiddler, the Caching tab will attempt to calculate the cache freshness lifetime for responses that lack an explicit Expires or Cache-Control: max-age directive. The standard suggests clients use (0.1 * (DateTime.Now – Last-Modified)) as a heuristic freshness lifetime.

An alert Fiddler user noticed that the values he was seeing were slightly off what he expected: sometimes the values were 6 minutes shorter than he thought they should be.

Consider the following scenarios:

Last-Modified: February 28, 2016 01:00:00
Date: February 29, 2016 01:00:00
These are 24 hours apart (1440 minutes); 10% of that is 144 minutes.
Last-Modified: March 13, 2016 01:00:00
Date: March 14, 2016 01:00:00
Due to the “spring forward” adjustment of Daylight Savings Time, these values are just 23 hours apart (1380 minutes); 10% of that is 138 minutes.
Last-Modified: November 6, 01:00:00
Date: November 7, 01:00:00

Due to the “fall back” adjustment of Daylight Savings Time, these values are 25 hours apart (1500 minutes); 10% of that is 150 minutes.

So when a timespan encompasses an even number of those DST transitions, the effect cancels out. When a timespan encompasses an odd number of these DST transitions, the span is either an hour longer or an hour shorter than it would be if Daylight Savings Time did not exist.

-Eric

Standard
browsers, fiddler, perf

Automatically Evaluating Compressibility

Fiddler’s Transformer tab has long been a simple way to examine the use of HTTP compression of web assets, especially as new compression engines (like Zopfli) and compression formats (like Brotli) arose. However, the one-Session-at-a-time design of the Transformer tab means it is cumbersome to use to evaluate the compressibility of an entire page or series of pages.

Introducing Compressibility

Compressibility is a new Fiddler 4 add-on1 which allows you to easily find opportunities for compression savings across your entire site. Each resource dropped on the compressibility tab is recompressed using several compression algorithms and formats, and the resulting file sizes are recorded:

Compressibility tab

You can select multiple resources to see the aggregate savings:

Total savings text

WebP savings are only computed for PNG and JPEG images; Zopfli savings for PNG files are computed by using the PNGDistill tool rather than just using Zopfli directly. Zopfli is usable by all browsers (as it is only a high-efficiency encoder for Deflate) while WebP is supported only by Chrome and Opera. Brotli is available in Chrome and Firefox, but limited to use from HTTPS origins.

Download the Addon…

Update: Telerik updated Fiddler to install per-user. To use this extension, copy the files from C:\program files (x86)\Fiddler to %userprofile%\appdata\local\programs\Fiddler.

To show the Compressibility tab, simply install the add-on, restart Fiddler, and choose Compressibility from the View > Tabs menu2.

View > Tabs > Compressibility menu screenshot

The extension also adds ToWebP Lossless and ToWebP Lossy commands to the ImageView Inspector’s context menu:

ImagesMenuExt

I hope you find this new addon useful; please send me your feedback so I can enhance it in future updates!

-Eric

1 Note: Compressibility requires Fiddler 4, because there’s really no good reason to use Fiddler 2 any longer, and Fiddler 4 resolves a number of problems and offers extension developers the ability to utilize newer framework classes.

2 If you love Compressibility so much that you want it to be shown in the list of tabs by default, type prefs set extensions.Compressibility.AlwaysOn true in Fiddler’s QuickExec box and hit enter.

Standard