Chrome Deprecates Subject CN Matching

If you’re using a Self-Signed certificate for your HTTPS server, a deprecation coming to Chrome may affect your workflow.

Chrome 58 will require that certificates specify the hostname(s) to which they apply in the SubjectAltName field; values in the Subject field will be ignored. This follows a similar change in Firefox 48. If impacted, you’ll see something like this blocking page as you load your HTTPS site:

NET::ERR_CERT_COMMON_NAME_INVALID blocking page in Chrome

NET::ERR_CERT_COMMON_NAME_INVALID is an unfortunate error code, insofar as all common names are now ignored. Chrome is working to improve the debuggability of this experience, via:

Notably, Windows’ ancient makecert.exe utility cannot set the SubjectAltName field in certificates, which means that if you’re using it to generate your self-signed certificates, you need to stop. Instead, users of modern Windows can use the New-SelfSignedCertificate command in PowerShell.

New-SelfSignedCertificate -DnsName "www.example.com", "example.com" -CertStoreLocation "cert:\CurrentUser\My"

Using openssl for self-signed certificate generation? See https://stackoverflow.com/a/27931596.

This new restriction may also impact users of very old versions of Fiddler (or FiddlerCore), or users who have configured Fiddler to use MakeCert for whatever reason. Fortunately, Fiddler offers a number of different certificate generators, so you just need to make a small configuration change. To switch away from MakeCert, click Tools > Fiddler Options > HTTPS and click the “Certificates generated by MakeCert engine” link. Change the dropdown to CertEnroll and click OK. Click Actions > Reset All Certificates and restart Fiddler.

image

If you’re building an application atop FiddlerCore, you’ll need to make sure you’re not using makecert; see the end of this post for help.

-Eric Lawrence

PS: There’s also a EnableCommonNameFallbackForLocalAnchors policy. You shouldn’t use it and you should just fix your certificates, or they’ll break when it’s removed in Chrome 65 or earlier.

Chrome Deprecates Subject CN Matching

Using Fiddler With iOS 10 and Android 7

If you’ve tried to use Fiddler with iOS10 beta or Android 7 Nougat, you have probably found that HTTPS decryption isn’t working, even if you use the latest Fiddler and the Fiddler Certificate Maker add-on. Unfortunately, at the moment both platforms are broken, but for different reasons. In both cases, the client will fail to receive responses for HTTPS requests, and Fiddler will only show a CONNECT tunnel.

iOS 10 Change

After installing the FiddlerRoot certificate, one also needs to go to Settings -> General -> About -> Certificate Trust Settings and manually enable full trust for the FiddlerRoot root certificate, including accepting a dialog that says that this will allow a third-party to eavesdrop on all your communications.

iOS 10 Beta Bug (Fixed for final version)

The beta of iOS 10 had a bug whereby, if the response to a HTTP CONNECT tunnel request contains a Connection: close response header, the client will close the connection instead of doing as it should and waiting until the TCP/IP connection closes. A few minor platforms have had the same bug over the years, but iOS is definitely the first important platform with this issue. At least two bugs have been filed with the Apple “Radar” bug reporter.

Working around this limitation is simple. In Fiddler, click Rules > Customize Rules. Scroll to the OnBeforeResponse function. Just inside that function, add the following lines:

  if (oSession.HTTPMethodIs("CONNECT")) {
oSession["ui-backcolor"] = "red";
oSession.ResponseHeaders.Remove("Connection");
}

Save the file and try connecting again.

Android 7 Feature

In contrast to the iOS regression, the change in Android 7 was intentional. The Android team has decided that, by default, HTTPS certificate validation for apps targeting API Level 24 and later will ignore all user-installed root certificates, meaning that your efforts to manually trust Fiddler’s root certificate will be fruitless. Individually application developers can temporarily override this change while debugging by updating the application’s configuration:

…or at all times…

 

…Unfortunately, these changes can only be undertaken by application developers and not end-users. End-users will probably need to jailbreak their devices, akin to what is required to circumvent certificate pinning.

-Eric

Using Fiddler With iOS 10 and Android 7

Silliness – Fiddler Blocks Malware

Enough malware researchers now depend upon Fiddler that some bad guys won’t even try to infect your system if you have Fiddler installed.

The Malware Bytes blog post has the details, but the gist of it is that the attackers use JavaScript to probe the would-be victim’s PC for a variety of software. Beyond Kaspersky, TrendMicro, and MBAM security software, the fingerprinting script also checks for VirtualBox, Parallels, VMWare, and Fiddler. If any of these programs are thought to be installed, the exploit attempt is abandoned and the would-be victim is skipped.

This isn’t the only malware we’ve seen hiding from Fiddler—earlier attempts use tricks to see whether Fiddler is actively running and intercepting traffic and only abandon the exploit if it is.

This behavior is, of course, pretty silly. But it makes me happy anyway.

Preventing Detection of Fiddler

Malware researchers who want to help ensure Fiddler cannot be detected by bad guys should take the following steps:

  1. Do not put Fiddler directly on the “victim” machine.
    – If you must, at least install it to a non-default path.
  2. Instead, run Fiddler as a proxy server external to the victim machine (either use a different physical machine or a VM).
    – Tick the Tools > Fiddler Options > Connections > Allow remote computers to connect checkbox. Restart Fiddler and ensure the machine’s firewall allows inbound traffic to port 8888.
    – Point the victim’s proxy settings at the remote Fiddler instance.
    – Visit http://fiddlerserverIP:8888/ from the victim and install the Fiddler root certificate
  3. Click Rules > Customize Rules and update the FiddlerScript so that the OnReturningError function wipes the response headers and body and replaces them with non-descript strings. Some fingerprinting JavaScript will generate bogus AJAX requests and then scan the response to see whether there are signs of Fiddler.

-Eric

Silliness – Fiddler Blocks Malware

On Daylight Savings Time

In Fiddler, the Caching tab will attempt to calculate the cache freshness lifetime for responses that lack an explicit Expires or Cache-Control: max-age directive. The standard suggests clients use (0.1 * (DateTime.Now – Last-Modified)) as a heuristic freshness lifetime.

An alert Fiddler user noticed that the values he was seeing were slightly off what he expected: sometimes the values were 6 minutes shorter than he thought they should be.

Consider the following scenarios:

Last-Modified: February 28, 2016 01:00:00
Date: February 29, 2016 01:00:00
These are 24 hours apart (1440 minutes); 10% of that is 144 minutes.
Last-Modified: March 13, 2016 01:00:00
Date: March 14, 2016 01:00:00
Due to the “spring forward” adjustment of Daylight Savings Time, these values are just 23 hours apart (1380 minutes); 10% of that is 138 minutes.
Last-Modified: November 6, 01:00:00
Date: November 7, 01:00:00

Due to the “fall back” adjustment of Daylight Savings Time, these values are 25 hours apart (1500 minutes); 10% of that is 150 minutes.

So when a timespan encompasses an even number of those DST transitions, the effect cancels out. When a timespan encompasses an odd number of these DST transitions, the span is either an hour longer or an hour shorter than it would be if Daylight Savings Time did not exist.

-Eric

On Daylight Savings Time

Automatically Evaluating Compressibility

Fiddler’s Transformer tab has long been a simple way to examine the use of HTTP compression of web assets, especially as new compression engines (like Zopfli) and compression formats (like Brotli) arose. However, the one-Session-at-a-time design of the Transformer tab means it is cumbersome to use to evaluate the compressibility of an entire page or series of pages.

Introducing Compressibility

Compressibility is a new Fiddler 4 add-on1 which allows you to easily find opportunities for compression savings across your entire site. Each resource dropped on the compressibility tab is recompressed using several compression algorithms and formats, and the resulting file sizes are recorded:

Compressibility tab

You can select multiple resources to see the aggregate savings:

Total savings text

WebP savings are only computed for PNG and JPEG images; Zopfli savings for PNG files are computed by using the PNGDistill tool rather than just using Zopfli directly. Zopfli is usable by all browsers (as it is only a high-efficiency encoder for Deflate) while WebP is supported only by Chrome and Opera. Brotli is available in Chrome and Firefox, but limited to use from HTTPS origins.

Download the Addon…

To show the Compressibility tab, simply install the add-on, restart Fiddler, and choose Compressibility from the View > Tabs menu2.

View > Tabs > Compressibility menu screenshot

The extension also adds ToWebP Lossless and ToWebP Lossy commands to the ImageView Inspector’s context menu:

ImagesMenuExt

I hope you find this new addon useful; please send me your feedback so I can enhance it in future updates!

-Eric

1 Note: Compressibility requires Fiddler 4, because there’s really no good reason to use Fiddler 2 any longer, and Fiddler 4 resolves a number of problems and offers extension developers the ability to utilize newer framework classes.

2 If you love Compressibility so much that you want it to be shown in the list of tabs by default, type prefs set extensions.Compressibility.AlwaysOn true in Fiddler’s QuickExec box and hit enter.

Automatically Evaluating Compressibility

What’s New in Fiddler 4.6.2

TLDR? – Get the newest Fiddler here.

It’s been just over two months since the last significant release, and Fiddler 4.6.2.0 (and v2.6.2.0) are now available.

As always, the latest build includes a slew of bugfixes and minor tweaks, as well as a number of features described in this post.

Default Certificate Generator Changed

Changes coming to certificate validation in browsers and other clients mean that certificates generated by makecert.exe (previously Fiddler’s default generator) will soon be rejected. To address this problem, the default certificate generator on Windows 7 and later has been changed to CertEnroll. (Windows XP and Vista users should consider installing the similar CertMaker Addon).

Unfortunately, if you’re upgrading from an earlier version of Fiddler which used a different certificate generator, you may need to explicitly reset Fiddler’s certificates. Doing so is simple:

  1. Click Tools > Fiddler Options.
  2. Click the HTTPS tab.
  3. Ensure that the text says Certificates generated by CertEnroll engine.
  4. Click Actions > Reset Certificates. This may take a minute.
  5. Accept all prompts
  6. If you are using Fiddler to capture secure traffic from a mobile device or Firefox, you will need to remove the old Fiddler root certificate from that device (or Firefox) and install the newly-generated Fiddler certificate.

If necessary, you can read more about resetting Fiddler’s Certificates or read more about Fiddler’s Certificate Generators.

SAZ Repair

From time to time, users have asked for help with Fiddler Session Archive files (.SAZ or .RAZ files) that are corrupt, either because they are incomplete (e.g. power failed) or they were mangled by an incomplete download or a failing disk drive.

Fiddler 4.6.2 includes a new feature that can recover data from corrupt Session Archive files. If the Session Archive fails to load due to corruption, you’ll be prompted to attempt a repair of the file. Data recovered from the SAZ file will be stored in a new archive and loaded for display.

Notably, this feature may also be useful to recover corrupt .zip, .docx, .xlsx, .pptx, etc files that have nothing to do with Fiddler; give it a try!

FiddlerHook Removed

This release removes the FiddlerHook extension for Firefox. Mozilla is changing their add-on model for Firefox extensions. Short-term, Firefox requires that extensions be signed (and Mozilla has declined to sign FiddlerHook) and over the next year, Mozilla will be removing the XUL Overlay extension model upon which FiddlerHook was based.

Fortunately, you don’t really need FiddlerHook to use Fiddler with Firefox. For HTTP traffic, it will often “just work” and for HTTPS traffic, only minor configuration updates are needed. You can read this post for tips on using Fiddler with Firefox.

Decryption Control

Previously, Fiddler UI only allowed you to exempt certain hosts from HTTPS decryption; if you wanted to only decrypt from a small number of hosts, you were forced to use the script engine. That limitation has been removed via a new option on the Tools > Fiddler Options > HTTPS tab. Simply click the link to toggle between exclusion and inclusion:

imageimage

Extensibility Improvements

This release adds a number of improvements to Fiddler’s extensibility, from both FiddlerScript and .NET extensions.

BindUIButton

You can now add buttons to Fiddler’s toolbar in a supported way. Simply add a new BindUIButton attribute to a static method in your FiddlerScript file; the string argument is the caption with which to label the button.

For instance:

    BindUIButton("Copy HAR")

yields:

toolbar button

Toolbar buttons are added to the left of the toolbar in the order opposite of their listing within the FiddlerScript. Adding images is not supported from FiddlerScript but you can use Unicode Emoji symbols if you’d like.

Fiddler extensions may add to the toolbar using the static method FiddlerToolbar.AddToolstripItem() and may remove entries using .RemoveToolstripItem().

Export to String

In many cases, you may wish to generate a string representing one or more Sessions in either HTTPArchive (HAR) or cURL format. While you can manually use File > Export to generate files of either format, you can now skip the middle-man and export to these types in memory. To do so, simply set the ExportToString option and do not set a Filename option. After the DoExport command completes, the output is found in the OutputAsString option.

For instance, you can add the following to your FiddlerScript:

BindUIButton("Copy HAR")
ContextAction("Copy HAR")
public static function doCopyHar(arrSess: Session[])
{
var oExportOptions = FiddlerObject.createDictionary();

// If you’d prefer to save to a file, set the Filename instead
//oExportOptions.Add(“Filename”, “C:\\users\\lawrence\\desktop\\out1.har”);

oExportOptions.Add(“ExportToString”, “true”);
oExportOptions.Add(“MaxBinaryBodyLength”, 1000000);
oExportOptions.Add(“MaxTextBodyLength”, 1000000);

FiddlerApplication.DoExport(“HTTPArchive v1.2”, arrSess, oExportOptions, null);
var sOutput: String = oExportOptions[“OutputAsString”];

Utilities.CopyToClipboard(sOutput);
FiddlerApplication.UI.SetStatusText(“Copied Sessions as HAR”);
}

…and Fiddler will add a toolbar button and context menu command that copies the Selected Sessions to the clipboard in HAR format. (Tip: You can choose Edit > Paste As Sessions in Fiddler to create new Sessions based on HAR text that you’ve copied from browser tools.)

Alternatively, you can add a similar command to copy the Selected Sessions as a set of cURL commands:

BindUIButton("Copy cURL")
ContextAction("Copy cURL")
public static function doCopyCurl(arrSess: Session[])
{
var oExportOptions = FiddlerObject.createDictionary();

// If you’d prefer to save to a file, set the Filename instead
//oExportOptions.Add(“Filename”, “C:\\users\\lawrence\\desktop\\out1.bat”);

oExportOptions.Add(“ExportToString”, “true”);

  FiddlerApplication.DoExport("cURL Script", arrSess, oExportOptions, null);
var sOutput: String = oExportOptions["OutputAsString"];

Utilities.CopyToClipboard(sOutput);
FiddlerApplication.UI.SetStatusText(“Copied Sessions as cURL”);
}

Invoking on the UI Thread

Fiddler processes Sessions on background threads, but you should only ever manipulate Fiddler’s UI using the UI thread. Only a few of Fiddler’s UI calls are thread-safe; if you’re not sure, your script should use the new FiddlerObject.uiInvoke or FiddlerObject.uiInvokeAsync methods to avoid crashing or corrupting the user-interface.

Load Extensions at Runtime

To support some exciting new work from the community, Fiddler now has the ability to load additional Extensions and Inspectors at runtime; this enables building of more complex add-on systems atop Fiddler’s existing system. To use these APIs, invoke any of these four methods from the UI thread:


FiddlerApplication.oExtensions.InstantiateInspectorsFromPath(string sPathToInspectors)
FiddlerApplication.oExtensions.InstantiateExtensionsFromPath(string sPathToExtensions)
FiddlerApplication.oExtensions.InstantiateExtensionsInFile(FileInfo oFile, bool bWriteToLog, bool bRethrowExceptions)
FiddlerApplication.oExtensions.InstantiateExtensionByType(Type typeExtension, bool bWriteToLog)

Thank You!

Lastly, I’d like to thank everyone for all of your support over the last twelve years, as Fiddler has evolved from a side project to a fully-supported debugging platform used around the world. I’m excited to see where Telerik takes Fiddler next, and while I’ll be keeping plenty busy in my new job, I expect I’ll remain involved in the Fiddler community (updating the book, and haunting the forum) for quite some time.

Wishing you all the best in 2016 and beyond!

-Eric Lawrence

What’s New in Fiddler 4.6.2