text/plain

Security Software – An Overview

I’ve spent nearly my entire professional career in software security: designing software to prevent abuse by bad actors. I’ve been battling the bad guys for over two decades now, from hunting security bugs in Microsoft Office (I once won an Xbox for finding a vulnerability that allowed malicious clipart take over your computer) to designing security mitigations in Internet Explorer, Google Chrome, Edge, and Windows, to building tools to allow other good actors to find and kill security bugs.

In 2022, I left the world of software security for the world of security software, rejoining some old friends now on the Microsoft Defender team. The difference seems slight but it’s a big one– instead of trying to keep an otherwise-useful software product secure from attackers, now the entire point of my product is to protect users.

If not for adversaries, my product would have no reason to exist.

What Is “Protection”?

The goal of protection is to prevent bad actors from abusing technology to cause loss for protectees. While those losses can take many direct forms, most fall into several categories:

Loss of money (compromise of bank/credit accounts, ransoms paid to recover data)

Loss of productivity (loss of device/service availability, time to recover from incidents, etc.)

Loss of data (destruction of potentially irretrievable data)

Loss of reputation (bad PR, loss of customer/public trust, embarrassing secrets exposed)

Legal ramifications

Beyond losses caused by bad actors, protection features must avoid other losses:

Loss of productivity due to protection in the absence of an attacker (e.g. false positives, application compatibility breakage, performance hits, user confusion, all leading to lower productivity)

Loss of regulatory approval (e.g. failure to demonstrate compliance with infosec standards could result in loss of permission to operate in a regulated industry)

Loss of peace of mind (e.g. failure to explain protection/posture results in user feeling unsafe/unconfident because they worry about attacks)

Avoiding these other losses is absolutely critical– the cure must not be worse than the disease, both for society and for our business. I often joke: “You start having a real hard time selling fire extinguishers if even one of them ever burns down a building.”

Framing: Protection Levers

To provide protection, vendors of security software invest in several related areas:

Sensors – Allow watching events of interest to determine when an attack may be imminent or underway. Examples: AV File system monitor, Network Protection connection monitor.

Throttles – Allow control of the behavior of the system to prevent events when an attack is suspected or detected. Examples: Edge’s Anti-phish blocking, Defender’s File System filter, Defender’s Network Protection connection disruption.

Intelligence – Intelligence guides how to react to sensed data in order to block attacks with throttles, without interfering with the normal operation of the system. Intelligence includes both Threat Intelligence (indicators of compromise and indicators of attack (IoC/IoA) as well as Compatibility Intelligence (“Known good” software and services that are legitimate and should not be blocked.) Examples: SmartScreen Web Defense Service, Application Reputation Service, AV Signatures & Behavior Monitoring. In some cases, customers might provide their own intelligence via 3P TI or via direct knowledge of their own usage of IT.

Beyond these core protection levers, two other levers are important for the overall protection story:

Remediations – The ability to act upon specialized intelligence to recover from an attack. For instance, after a ransomware incident is disrupted, what can we do to help the user? e.g. password change workflows, file recovery from OneDrive or Volume Shadow Copy, etc.

UX / Visibility – The ability of the user or a regulator to understand protection state and feel confident if the protection state is “good”, or to recognize the need for reconfiguration if it is not.

These levers work together to build higher level protection concepts:

Sensor + Intelligence = Detector

Sensor + Throttle + Intelligence = Blocker

Intelligence + Remediations = Recovery

Improvements in each of these areas can increase the level of protection, and we can unlock new protections with either the introduction of new capabilities in each area, or a new combinations of levers.

Other Uses for Sensors and Throttles

Sensors and Throttles are powerful primitives that have uses beyond traditional protection scenarios.

For example, an enterprise may combat insider threats and enforce regulatory compliance by using sensors and throttles to build Information Rights Management products like Microsoft Purview.

Sensors and Throttles are also often used to achieve non-protection goals, sometimes called Control scenarios. For example, combining the Web Defense Sensors and Throttles with a list of site categories enables Web Content Filtering to enforce an organization’s Acceptable Use Policy (“No gambling sites may be used at work“).

My Journey to Security Software

The last two years has been a whirlwind of learning: security software is a huge and very profitable industry, and as such there’s an entire universe of complexity and a new encyclopedia of acronyms and terms of art to learn. Even “Microsoft Defender” is a bit of a misnomer– it’s not one product, but an entire line of products designed to protect both enterprises and consumers from attack.

At a very high level, my product, Microsoft Defender for Endpoint, breaks down into two major components: Endpoint Protection (EPP) and Endpoint Detection and Response (EDR). These days, most of my work accrues to MDE’s EPP.

EPP

The primary goal of Endpoint Protection is to protect the user against threats on their individual endpoint device. Most EPP features focus on preventing initial access (attacks that allow an attacker to get malicious code running on the device in the first place). Modern endpoint protection software does not rely solely on static signatures (file hashes or byte sequences), going beyond to emulate unknown code and watch for suspicious behavior of processes as they execute.

Every Windows user gets many of the most important EPP features for free, including MSAV (Microsoft Antivirus) which protects against viruses and malware, and SmartScreen, which protects against web-borne threats (phishing, malware, and techscams). Beyond these two core protections, there are many niche features (e.g. Exploit Protection) and adjacent features (e.g. the Windows Firewall is branded with “Defender” although it’s mostly built by Windows). These security features are mostly exposed through the Windows Security app.

If a user decides to install 3rd Party security software (e.g. McAfee), it may take over some of the duties of some Defender components (usually replacing MSAV and sometimes the Windows Firewall).

Enterprises that deploy MDE EPP have some additional features that require configuration by an organization’s security team, including Device Control, Attack Surface Reduction rules, AppControl for Business, and Network Protection.

EDR

EDR software aims to address the fact that Endpoint Protection inherently can never be infallible — at some point, through user error, delayed threat intelligence, a novel attack, or other shortcoming, some devices will fall victim to an attacker.

EDR software only makes sense in the context of an enterprise where there’s a security operations center (SOC) tasked with keeping the company’s fleet of devices secure from attackers. EDR is a huge market, and big players include CrowdStrike, TrendMicro, Sentinel One, and others.

EDR software sends a firehose of telemetry to the SOC to allow creation of security alerts for investigation and remediation. For example, if malicious behavior is observed on a cluster of devices, the SOC can isolate those devices from the rest of the enterprise, can use the “Live Response” tool to run investigation packages on the suspicious devices, and can lock down the devices and any credential information used on them to protect the attack from spreading to the rest of the enterprise.

The SOC is also responsible for managing the security posture of the fleet, ensuring that security policies are configured, patches are installed on every endpoint, that risky apps are not present in the environment, etc.

Other Defender Products

Many other products live under the Defender banner, including MDO (Microsoft Defender for Office), MDCA (Microsoft Defender for Cloud Apps), MDIOT (Defender for IoT), etc.

Some of these products reuse threat intelligence, sensors, and throttles from other products (e.g. MDO scans email attachments with MDAV on the server), and some include sensors, throttles, and TI exclusive to each product. Part of our team’s journey has been finding opportunities to bring these product’s components closer together to reduce duplication and improve threat protection.

Stay safe out there!

-Eric

Best Practices for SmartScreen AppRep

Last year, I wrote about how Windows integrates SmartScreen Application Reputation to help ensure users have a secure and smooth experience when running downloaded software.

tl;dr: When a user runs a downloaded program, a call to SmartScreen’s web-based reputation service is made, and four possible outcomes can occur:

“Known Good” SmartScreen’s Web Service indicates that the file is “Known safe”, and the program runs without prompting.
“Known Bad” SmartScreen’s Web Service indicates that the file is “Known malicious”, and the user is blocked from running the software via a red warning dialog.
“Unknown” SmartScreen’s Web Service indicates that the file is “Unknown,” and the user is interrupted from running the software via a blue confirmation dialog.
“Offline”: SmartScreen’s Web Service is offline or unreachable and the user sees a notice that SmartScreen cannot help decide whether the software is safe to run. The user sees a blue confirmation dialog.

As a software developer, it’s natural that you’ll want to have your apps classified as “Known Good” to streamline the installation of your software.

This post explores how to achieve that.

Building Positive Reputation

The Application Reputation service builds reputation based on individual file hashes (SHA256) and/or the certificates used to sign those files. The following are best practices we recommend.

Sign Your Files

Because any update to a file changes its hash, this means that new files that are unsigned have no reputation by default. To avoid triggering warnings after every update, be sure to sign your software using an Authenticode certificate.

Because your certificate can be used to sign files for years, it will accumulate reputation that will accrue to files you sign in the future. To maximize reputation data, you should aim to use the same certificate for as long as possible. Certificates periodically expire, however, and you may find that when you being using a new certificate if must build reputation again.

To help ensure that all of your good reputation accrues in one place, try to sign everything with a single certificate (e.g. do not use a different certificate for each product, or a different certificate for each of your company’s divisions, or anything like that).

From ~2013 to ~2019, all files signed by an Extended Validation Authenticode Certificate were given a “positive” reputation by default, but EV certificates are no longer treated specially — every certificate must build reputation individually.

Sign All Files

Beyond .exe and .dll files, you can also sign .msi Installers, script files (PowerShell, VBScript, JavaScript), and .cab Archives.

While SmartScreen only checks the reputation of the main file that the user executes (aka its “entry point”), you should sign all files to help protect them from tampering, and to ensure your app works correctly when Win d ows 11’s Smart App Control feature is enabled. The Windows 11 Smart App Control feature goes further than SmartScreen and evaluates trust/signatures of all code that is loaded by the Windows OS Loader and script engines.

NOTE: Smart App Control’s signature check does not currently support ECC signatures.

Always Follow the Objective Criteria

Ensure that everything you install follows Microsoft’s Objective Criteria for well-behaved software. Adherence to the criteria ensures that users will not be surprised or upset about the behavior of your software.

Violations of the objective criteria can cause SmartScreen, Microsoft Defender, and 3rd-party security products to treat your app as Malware or Potentially Unwanted Software.

Escalations

If your software is blocked by the red “known bad” dialog box:

… you should investigate to determine whether anything unexpected is included; for instance, was malware injected into your product by a supply-chain attack? You should upload your files to VirusTotal.com and what, if any, detections occur across the universe of security products. If you cannot find any indications of compromise, report the potential false-positive to Microsoft Security Intelligence by uploading your files to the WDSI portal.

If your software is blocked by the blue “unknown” dialog box:

… don’t despair. Ensure that you’ve followed the best practice of signing your software with a certificate, and then just wait. As your software gets downloaded by more users around the world (“increasing prevalence”), its honorable behavior will be noted and eventually its reputation will move into the “Known good” category.

Bypass for Enterprises

Some enterprises build and distribute their own software internally and wish to avoid SmartScreen prompts for their applications. For security reasons, you can and should still sign internally-developed software. However, internally-developed software might never be used broadly enough to organically build a positive reputation within the SmartScreen Service.

Note: Microsoft Defender for Endpoint customers might reasonably expect that creating an Allow Indicator for a certificate would allow files signed by that certificate to bypass SmartScreen AppRep. This is a reasonable expectation, but unfortunately does not presently work (as of November 2024).

Enterprises are in control of how their users get software, and most internal software deployment systems do not result in SmartScreen checks. Only software downloaded by web browsers or copied from network shares using Windows Explorer is likely to get a Mark-of-the-Web that results in consulting SmartScreen’s reputation service.

If using Explorer or a web browser to obtain software is a core scenario for your enterprise, you can avoid SmartScreen checks by correctly configuring your Windows Security Zone settings. Using the Site to Zone Assignment List group policy (or the Internet Control panel), place your trusted software distribution sources into the Local Intranet security zone:

After doing so, files downloaded from the listed locations will no longer be tagged with the Mark-of-the-Web, and executing them will no longer perform reputation checks against SmartScreen.

If you use the Microsoft Edge web browser, you should also use Group Policy to set the SmartScreenForTrustedDownloadsEnabled setting to 0 (Disabled) to prevent the browser from itself performing reputation checks while downloading files from your trusted locations.

Stay safe out there!

-Eric

Defensive Technology: Controlled Folder Access

Most client software’s threat models (e.g. Edge, Chrome) explicitly exclude threats where the local computer was compromised by malware. That’s because, without a trusted computing base, it’s basically impossible to be secure against attackers. This concept was immortalized decades ago in the Ten Immutable Laws of Security:

In the intervening years, new technologies (like Secure Enclaves) have been launched in an attempt to provide some defenses even when a PC has been compromised, but they remain very limited in their capabilities. In my view, preventing initial access represents the most important security investment we can make.

Protecting Files

Nevertheless, there’s value in defense-in-depth and making life harder for attackers even after they get access to a PC.

In that vein, Windows 10 has a feature called Controlled Folder Access (CFA) which aims to help protect against ransomware. In the common ransomware attack, malicious native code running on the user’s machine begins encrypting their files with a randomly-generated key that is sent to the attacker. After files have been encrypted, a ransom note is shown demanding that the user pay money to get the decryption key.

CFA impedes this attack by preventing applications with “unknown” reputation from touching files in sensitive/protected folders, including the user’s Documents, Pictures, and Favorites folders, as well as any folders selected by the user. You can enable CFA using the Ransomware protection section of the Windows Security app:

Rather than moving sensitive files to a protected vault, it’s more like you’ve hired a bouncer to keep questionable apps out. If you want to protect more folders, click the Protected Folders link and choose the folders you’d like protected. If you need to allow an unknown app to access protected files, you can do so using the Allow an app link.

When Defender blocks access, you’ll see a small toast notification:

You can see blocked actions in both the Protection history section of the Windows Security app:

…as well as the Windows Defender node of the Windows Event Viewer:

If you’re managing devices with Intune or Group Policy, you can also enable CFA in “Audit Mode“, which will log untrusted accesses without blocking them.

How?

This feature depends upon the Windows Defender filesystem filter driver (WDFilter). From kernel mode, the filter watches for access to protected folders. If access is requested by an untrusted process, any Write permission bit is stripped from the request.

Non-Obvious Bits

While CFA is conceptually pretty simple, under the covers there’s a fair bit of complexity.

For one thing, various well-known and legitimate applications (like Microsoft Office) offer extensibility models that could be used to load malicious modules. Similarly, well-known applications (e.g. Notepad) could have malicious code injected into them. So, Defender has to watch what’s loaded into each process and may consider an otherwise “friendly” process tainted for the purposes of CFA.

Additionally CFA could be circumvented if a process directly accesses a disk volume using low-level APIs. If CFA blocks an unknown app from accessing a disk in this way, there’s no folder path to show, so the toast (perhaps confusingly) claims that CFA blocked the app from “making changes to memory.”

Finally, CFA must carefully handle all of the myriad equivalent ways in which a file’s path can be represented, for example:

Referring to local files using a UNC path and 8.3 filename syntax

Debugging

In some cases, understanding why a process has triggered CFA could be difficult. For example, several engineers at Microsoft recently found that running most command-line tools (like ping.exe) in the SYSTEM32 folder was triggering the making changes to memory toast. This seems very strange — a network tool like ping isn’t expected to be touching the local disk at all!

Using SysInternals’ Process Monitor to log the process behavior can reveal the root cause. Click Filter > Enable Advanced Output. Use the toolbar toggle button to filter to FileSystem events, and create a filter for Operation is IRP_MJ_CREATE:

Look through the IRP_MJ_CREATE events to find those for the C: volume:

In this case, we see that an AMD GPU driver amdkmdag.sys is opening the C: volume for Write Access as in the context of ping.exe as it shuts down, leading to the toast:

To discover whether a process is being treated as tainted by the engine, see the MPLog-##### file inside C:\ProgramData\Microsoft\Windows Defender\Support. For example, a value of TaintType:0x1 indicates that an untrusted module was loaded into the process.

Online Backup

Beyond Controlled Folder Access, Windows’ other built-in anti-ransomware approach is to enable online backup to a cloud file provider like OneDrive. The recovery experience offered by OneDrive depends on which level of account you have:

If Microsoft Defender detects a ransomware attack, it can send a signal to the cloud file provider, indicating the time of compromise, to stop syncing now-encrypted files to the cloud, and to subsequently ease the process of recovering pre-encryption files from the cloud. You can see the flow from OneDrive’s documentation:

Stay safe out there!

On Politics

I do not come from an especially political family. One parent has not voted in decades, and the other votes regularly, but is not an enthusiast and values harmony over potentially-divisive political discussions. Politically, I am left of center– the middle child in both age and political leanings: one brother to the left of me, and one brother to the right.

As a kid, my parents thought Republicans were the good guys. Bill Clinton’s depravity didn’t help matters at all. In high school, I had one Political Science teacher who challenged my default “I’m a Republican, I guess” posture, but I still skipped voting for Al Gore when I turned 18 as I didn’t care enough to bother in deep-blue Maryland. As a college freshman, I studied early American history and read many of the letters and musings of America’s founders, gaining a much broader understanding of the amazing story behind the founding of this nation, the risks the founders faced, and the compromises that linger to this day. Thus was birthed my interest in politics.

Our Revolutionary Founders (Jefferson in particular), concluded that the only way for a government to be responsive to its citizenry was to have it be periodically overthrown. Rather than suffer the losses and uncertainty of violent overthrow, the notion of regular, peaceful overthrow of the government was seen as the best approach. We call these peaceful revolutions elections.

“I am not an advocate for frequent changes in laws and Constitutions. But laws and institutions must go hand in hand with the progress of the human mind. As that becomes more developed, more enlightened, as new discoveries are made, new truths discovered and manners and opinions change, with the change of circumstances, institutions must advance also to keep pace with the times. We might as well require a man to wear still the coat which fitted him when a boy as civilized society to remain ever under the regimen of their barbarous ancestors.”
Thomas Jefferson

I watched The West Wing and thought it was brilliant. John Kerry visited Microsoft, gave a nice speech, and I read his book and voted for him in 2004. In 2008, I eagerly voted for Obama, although I greatly respected John McCain until his disastrous choice of running mate. I voted for Obama again in 2012 although I believe, gaffes aside, the country would be just fine under Romney. I watched The Newsroom, and I still believe that its opening scene is the best and most important five minutes of television ever made. In 2016, I voted for Hillary, despairing that Trump was even a contender for the nation’s highest office. In 2020, I supported Buttigieg and Warren in the primaries, holding my nose to vote for Biden in the general (I think Biden is an admirable patriot and his reward for a lifetime of public service should be a pleasant retirement, not working into his 80s.)

I was furious that Biden announced that he’d run again in 2024, and that both parties had failed to prepare the next generation of leaders to take over. For durable improvements, Systems > Individuals.

On the first day of early voting, I voted for Harris, and thought she’d be a fine President for all Americans.

On The Issues

Guns are now the leading cause of death for American children. This is an insane travesty, and it’s a stain on our nation that so little has been done about it. More locally, Texas has significantly loosened its gun laws over the last twelve years, with predictably tragic results.

My thoughts on civilian ownership of firearms could fill a book. It’s not a simple issue, but the status quo is barbaric and unacceptable.

Climate Change: Austin just notched its hottest October in history. Atop Kilimanjaro in 2023, I saw the sad state of the depleted glaciers, and I worry that there may be nothing left when I summit again on New Years’ Eve just before the calendar ticks over to 2026. I can’t help but believe that technology is going to bail us out (yet again) and we’ll avoid the worst projections. But we are playing with fire, and we risk activating tipping points we don’t even know about. As with most things, the poor and powerless are going to pay most of the price.

Taxes: It’s common for those on the left to scream “Tax the rich! Make them pay their fair share!” This clip from West Wing is a great and important one– we should tax the rich more (Bill Gates and Warren Buffet, to their credit, both agree). But we must acknowledge that most of the wealthy pay much more in taxes than the average American, and it’s crucial to recognize that even if we taxed the rich at a rate of 100%, we would not be able to balance the budget for long. I’m particularly fond of someone’s flippant proposal that we should require the super-wealthy pay higher taxes, and in return get naming rights for infrastructure– $10M for a Post Office, $5M for a Bridge, etc.

Still, I’ve come around to the notion that the existence of billionaires is a policy failure, and there are absolutely insane loopholes in the tax code (e.g. the IRA one, “step-up basis” on inheritances) that should go away.

I’m infuriated that so many on the right should be voting for higher taxes, but don’t because “the poor see themselves not as an exploited proletariat, but as temporarily embarrassed millionaires.” The rich see you as suckers.

Hard choices must be made, and our system is such that politicians are punished for making them.

The Spending side represents the other side of hard choices; as a society, we must choose how to allocate our limited resources. There are many worthy causes, but we cannot afford them all. Prioritization matters.

For example, the United States is set to spend a trillion dollars ($1,000,000,000,000!) on modernization of our nuclear arsenal, and I don’t think we talk nearly enough about whether that’s the best way to protect Americans against the true threats of the 21st century.

Unintended consequences abound: The US has the most expensive educational system and most expensive healthcare system in the world, and that’s not just because of greed– it’s partly because of some well-meaning policy choices we’ve made have had some awful and unforeseen consequences. We should not be afraid to change things, but we must also refine those changes as we observe their outcomes.

However, it’s almost as great a mistake to think of the United States’ budget as that of a family– money simply works very differently when you control the supply, interest rates, and other key aspects of the worldwide financial system.

Despite the complexity, common sense supports simple heuristics: When considering an expenditure, we should ask “Is it an investment, neutral, or worsening some larger problem? Is the money going to a fellow citizen, an ally, or an enemy?” I’d love to see the $800B a year we spend on the military-industrial complex pivoted to the future-industrial complex.

Abortion: I have a deep ambivalence about abortion– extremists exist on both sides of the divisive issue, and most folks refuse to think deeply about the implications of their own position, let alone listen to those on the other side. Any litmus test is a prime target for abuse.

After decades of thinking about this, I’ve settled into a simpler position: “Abortion is healthcare,” and believe that’s where the political realm should leave it.

Trump: Truer words were never spoken: “Trump is a poor man’s idea of a rich man, a weak man’s idea of a strong man, and a stupid man’s idea of a smart man.” This incurious, impetuous liar is remarkable only for his utter shamelessness.

Trump is both a symptom of great rot in American society, and an accelerator of that decay. Trump claims to “love the poorly educated“– not because he aims to improve their life and help their family attain the American Dream, but because he knows a sucker when he sees one.

Some more moderate Trump voters assume: “Meh, the worst of his plans are just talk, they’ll never actually happen.” They do not understand that Trump 2.0 is a version with far fewer checks, and they failed to heed the (belated) dire warnings of those who worked for him last time. In the same way that Trump uses the poorly educated as “useful idiots,” evil people much smarter than Trump (foreign and domestic) will use our President as their useful idiot.

Politicians: I’ve learned that there’s a class of Trump voters who know he’s an awful person but voted for him because he’s entertaining. They assume that politicians are ineffective and almost interchangeable liars, so the person in the Oval Office doesn’t really matter. That’s a position not entirely without support, but one that falls apart when our institutions and checks-and-balances fail or are subverted. Trump’s backers have been working to undermine our institutions for decades.

Some people have been convinced (often by folks like Harvard-educated Ted Cruz) that politicians are out-of-touch “elitists” who “think they’re better than us.” I’m reminded of this cartoon in the New Yorker:

In the same way that I want my manager at work to be smarter and harder working than I am, when I vote, I want to hire someone for the job who’s better than me in every way possible.

The Democrats: I am still furious that the Democratic Party supported Trump in the 2015 primaries, literally betting the future of our country on the idea that a racist rodeo clown couldn’t possibly lie his way into the most important job in the world. We all lost in that bet.

Bob Menendez should not have been granted the opportunity to resign; he should have been expelled.

Partisanship: Too many of us interested in politics follow it like sports– cheering for the home team (no matter what), and booing the opposition (no matter what), and demanding “loyalty” to their own party regardless of malfeasance or foolishness. Instead of finding common ground with our neighbors, we’re split into factions as many politicians prefer. The truth is that all states are shades of purple. As hard as the Republicans want to convince Texans that it’s a “Deep Red” state, more Texans than New Yorkers voted for Biden.

We should not be afraid to hold our own “side” (and selves) accountable, no matter our opinion of others.

The Supreme Court: For many years, the Supreme Court was the branch of government that I respected most. Justices mostly kept their mouths shut when it came to politics, and an appointment is not a stop on the way to a highly-paid lobbyist job. Justices never have to pander to the base to win their next election. Most importantly, unlike Congress, the Justices have to show their work in written opinions that will be scrutinized for decades. In many of the painful decisions overturning good things, the Court has simply held that a given good thing requires that Congress write a law. Unfortunately, Congress has been mostly dysfunctional for many years now, failing to serve the American people in their constitutional duty.

More recently, however, the court has been tarnished by acceptance of improper gifts, political speech, and other scandals. More practically, a system where we incentivize or demand that the elderly literally work until they die (based on the vagaries of presidential elections) is a grotesque one. Thus far, the best proposal I’ve seen is to term-limit Justices to 18 years– a period long enough to help protect impartiality, but not so long as to make an appointment a life-sentence and the highest stakes action a Senate will ever conduct.

The American Electorate: Unfortunately, fear is a primary driver of voting behavior, and we humans have a set of base fears (particularly “fear of the other”) that politicians have learned to play like an instrument. This year, Ted Cruz spent millions of dollars on ads about immigrants and children’s genitalia, and he, the least popular Senator in a decade, sailed to reelection. Trump spent $65M on ads about genitals. As the world’s real challenges grow greater (aging populace, climate calamities, plagues, AI job displacement), greater fear is sure to follow.

The vote of an ignorant racist counts just as much as the thoughtful patriot, and “all the terrible people vote.” I’m as angry about politically apathetic non-voters as I am about the suckers who voted for the blowhards, the bigoted, and the hateful. Anyone who can look at Harris and Trump and think “Meh, no opinion here” needs to stop sniffing glue. And yes, the Republican party has been working for decades to make it harder for Americans to vote, but it’s just a weak excuse for many of the large block that do not vote:

While it’s easy to lament how disastrous the consequences of the 2024 elections will be, it’s important that elections have consequences. Without consequences, elections don’t matter. And if elections don’t matter, you’re no longer living in a democracy. As Dan Stevens taught in 7th grade civics, at a societal level, Democracy means you get the government you deserve.

Looking Forward

America is about to enter the “Finding out” phase of “Fuck around and find out.” Feedback loops are important, and the tighter they are, the more effective they are. Unfortunately, political feedback loops can take years or even decades between cause and effect.

After the 2024 election, the super-rich (Musk, Bezos, and the like) will be just fine. The merely-rich who supported Trump are going to find themselves like the UK Rich post-Brexit — when your entire economy gets kicked in the nuts, your personal tax rate is not the important factor in your wealth. An ebbing tide lowers all boats. As is common, the middle-class and poor are going to get hurt, and hurt bad, whether they supported Trump or not.

For now, though, I can’t let myself get too lost in despair for our world. Ultimately, we each only have so much control over our reality, and the main thing under our control is how we react to it. Catastrophizing (and anxiety more generally) is only useful if it inspires productive action.

This week, I’m taking a breath and looking around with both trepidation and curiosity. Maybe America, as a country, isn’t better than this. But even if that’s the case, we can be.

Hope is a thing that fights in the dark.

-Eric
PS: I recently moved from Twitter to BlueSky.

Lenovo P1, Gen7: Meh

I’ve been a loyal user of Thinkpads for over twenty-five years now, and I currently own four (with another on loan from Microsoft).

In July, the screen on my Lenovo X1 Yoga Gen 6 failed at an inopportune time, and my 8yo broke the screen on my backup (T480S), so I rush-ordered a Lenovo P1 Gen7 workstation laptop (22 cores, 64gigs of memory) to stand in while awaiting warranty replacement on the Yoga’s screen. After massive discounts, the cost with tax fell to a still-spendy $2,890. (The Yoga screen replacement went well, and I later replaced the T480S’ screen myself for ~$79 from LaptopScreen.com).

Unfortunately, when my new P1 workstation arrived, I discovered to my dismay that it had the same problems as my nearly-unused X1 Extreme Gen 3 — the OLED touchscreen, while mostly beautiful, had super-distracting quirks — it seemed to flicker, and almost have a screen door effect under certain conditions (the newer laptop might be the same panel as on the older one). Fortunately, these screens have now been in the market long enough that I wasn’t gaslit into thinking it was “just a me problem,” and searches turned up the explanation — the screendoor effect comes from the touch-digitizer, and the flicker comes from the way OLED power-management is implemented. Fortunately, I was able to rush-order the same machine with the non-touch-LCD (21KV001CUS) for $2,381 so I could do a side-by-side comparison. Sure enough, the regular LCD seems fine, and saving $500 for a lower-end screen and a cheaper GPU (Ada 3000, RTX 4070) that seemed to benchmark almost as well.

The Gen7 is pretty fast (although building Chrome still takes ages) and the keyboard and big screen are great. The fans seem to run more than they should, (although bizarrely they’re silent as I write this post for the first time in weeks) and it’s not too crazy heavy. Replacing physical buttons for the trackpoint with a tiny section at the top of the trackpad was a surprisingly-annoying downgrade.

Unfortunately, the laptop’s reliability has been poor. First, the Nvidia drivers were blue-screening the system and I remembered why I don’t like gaming GPUs– I don’t play games, and their drivers tend to prioritize performance over stability. So I disabled the Nvidia and started using just the integrated (Intel) GPU.

More recently, the system started ignoring all input shortly after boot — the trackpoint/trackpad wouldn’t move the cursor, wouldn’t accept clicks, the keyboard wouldn’t work, etc. I thought I had narrowed down the problem to the “Virtual Lock” feature that aims to automatically lock the PC when you walk away. I’d never enabled the feature and even after turning off “User Presence Sensing” in the UEFI Settings the problem continued. Setting the Services (Virtual Lock Sensor and Elliptic Human Presence Detection) to Disabled in Windows Services and disabling the Virtual Lock Sensor in Device Manager’s Software Components appears to have helped…

Fingers crossed!

Update: It didn’t. There’s something else that seems to be causing this. I’ve killed off some more of Lenovo’s preloaded software, and learned that the hard hang lasts for ~100 seconds before the system becomes responsive again, forcing me to just “wait” rather than hard-rebooting.

Some recent update to the system also started causing terrible echo for my Teams calls, which I’m working around by using an external speaker/microphone. Lame.

Defensive Technology: Antimalware Scan Interface (AMSI)

Endpoint security software faces a tough challenge — it needs to be able to rapidly distinguish between desired and unwanted behavior with few false positives and false negatives, and attackers work hard to obfuscate (or cloak) their malicious code to prevent detection by security scanners.

To maximize protection, security software wants visibility into attack chains at their weakest — after any obfuscation has been stripped away, immediately before (or during) execution. Unfortunately, this is hard, because most security software hooks are either very low-level (e.g. kernel drivers watching file and process activity) and thus lack context, or very high-level (e.g. scanning downloaded files), where obfuscation is in place.

The Antimalware Scan Interface is a Windows platform mechanism that allows host applications to call out to security software before performing sensitive operations (e.g. running script, elevating UAC, invoking ActiveX objects, etc). The security software can scan the data buffers and return a “Allow”/”Deny” verdict based on its threat intelligence. AMSI is pluggable on both sides — any host application can call into AMSI, and any security software can receive the calls[1].

On a default Windows system, hosts include cscript, wscript, PowerShell, the .NET CLR Platform, UAC elevation prompting, WMI, VSS, and several Microsoft server products (including optionally SharePoint). Microsoft Office desktop applications call AMSI to scan VBA macros.

Security software like Microsoft Defender (and most 3rd party AV products) will scan the data buffers provided by the host and return verdicts based on their threat intelligence (signatures). Microsoft Defender for Endpoint also uses AMSI to implement several of its attack surface reduction rules that constrain the behavior of scripts.

AMSI protections help improve blocking in common initial access vectors (e.g. running a downloaded .js file) and improve protection over traditional AV signatures (which can be fooled by obfuscation) and behavior monitoring (which might detect misbehavior too late). It’s especially useful against fileless threats in which the attack code was never on disk for a traditional AV sensor to scan.

The documentation for application developers who want to call into AMSI is pretty good. For security software developers, there’s both documentation and an sample AMSI scanner on GitHub.

Limitations

The biggest limitation in AMSI is that a host application must call it– unlike most security software sensors, which rely on kernel drivers and process injection techniques to provide security for every application, AMSI requires the explicit participation of an application to call out to AMSI and respect its verdict.

Hosts must choose when and if they will call into AMSI– for example, a host might not choose to call AMSI if it doesn’t think a given script is potentially dangerous (e.g. the Windows Scripting hosts may only call AMSI if they see a potentially-dangerous object created). We call these triggers, and they must be selected carefully to balance performance and security.

For example, AMSI is never called to evaluate JavaScript running inside your web browser (Edge, Chrome, Firefox, etc). Unlike cscript and wscript which run “shell scripts” with access to powerful capabilities (writing files, launching processes, etc), browsers execute “web platform” JavaScript inside tight security sandboxes that prevent interaction with files and other objects on the system. Browsers compete on runtime performance, and the overhead of security scans on sandboxed script would yield a poor cost/benefit.

As with any other security software component, attackers have attempted to tamper with AMSI, using a wide range of techniques ranging from simple to ingenious. The majority of such techniques require that the attacker have at least partially-compromised the victim PC (to manipulate the execution environment), and a major goal of AMSI is to foil the attacks that would grant the attacker initial access to start with.

Future Brainstorming?

AMSI has been around for a long time now, and hasn’t seen a ton of changes over the years. However, there’s recently a greatly renewed interest in empowering security software without requiring that each vendor write its own kernel drivers, and the model used by AMSI is a great one for providing powerful visibility and control in user-space.

To that end, I’d love to see more scenarios for AMSI-like callouts. Most notably among them– URL Reputation. Today, myriad applications present, transmit, and load content from URLs, but security software often is not well-positioned to “see” those URLs. Except for security code directly integrated into browsers (e.g. SafeBrowsing for Chrome, SmartScreen for Edge), security software is often forced to “guess” where a given network request is going via low-level packet-sniffing. This approach (at best) supplies only the hostname (not the full URL) will soon become even less reliable, as browsers further improve their privacy against network sniffers. Imagine if every application could easily call out to the platform security software and ask “Hey, I’m going to go grab these 40 URLs. Any objections?” Apple has recently introduced such an API: NEURLFilter.

Another scenario is malicious browser extensions. Today, security software can watch the browser load an extension’s code out of its filesystem location and block those loads if the files are known to be malicious, but at the browser level this blocking has a poor UX. All the browser “sees” is that an extension failed to load, but it doesn’t know why. It will still think that the extension should be present and loading, and can’t tell the user that the file was blocked for security reasons. Browsers don’t expose an API for security software to even know what extensions are installed and enabled; security inventory software that wants to report extensions up to the SOC must parse browsers’ internal configuration files (which is unsupported) to try to determine which extensions are allowed. If browsers could call into AMSI before loading an extension, this could provide a much better UX.

A further scenario is invocation of App Protocols. App Protocols provide the easiest way for an attacker to escape a browser’s security sandbox and are thus of prime interest to attackers. However, security software typically doesn’t get to directly observe the URL launched by the browser — instead, it must rely on kernel sensors (e.g. this callback) to watch for CreateProcess calls, and this might not be sufficient– not every protocol invocation results in a CreateProcess call (e.g. some result in a COM object creation) and a kernel sensor lacks important context (e.g. the URL of the page that asked to launch the protocol). If browsers could call into AMSI before invoking an App Protocol, they could provide a better UX and improve threat intelligence.

Any other scenarios I haven’t thought of yet?

-Eric

[1] While any app can register their own AMSI provider to get called during UAC elevation prompting, the code must be run out-of-proc in a process protected by PPL and thus only Microsoft Virus Initiative partners can participate in UAC’s AMSI checks.

Content-Blocking in Manifest v3

I’ve written about selectively blocking content in browsers several times over the last two decades. In this post, I don’t aim to convince you that ad-blocking is good or bad, instead focusing on one narrow topic.

Circa 2006, I was responsible for changing IE so that you could simply add an advertising site to the Restricted Sites zone and none of its script would load. Later, in 2010, I wrote a bit about the landscape of ad-blocking on the IEBlog.

More recently, Apple introduced Content-Blocking framework for their browser in 2015, and in 2019 the Edge team released Tracking Prevention, which blocks many ads in its Strict Mode.

Manifest v3

Recently, there’s been a bit of an outcry about Google’s move to require Chrome extensions be built atop a new platform named Manifest v3. This long overdue change attempts to mitigate the overprivileged Chrome extensions framework. V2 poses security, privacy, and performance risks to users, and has been abused (intentionally and unintentionally) by extension authors over the years.

No good deed goes unpunished, however, and conspiracy theorists have argued that Google is doing this to prevent ad-blockers from working. These theories aren’t entirely crazy — Google is, after all, first-and-foremost an advertising company. Where the theory falls apart, however, is that Google’s ads are among the easiest to block. While MV3 may make it more challenging to block some ad providers, you can still trivially choke off more than half of Google’s ad revenue in under a dozen lines of code. If MV3 were a conspiracy on Google’s part, it’s a jaw-droppingly ineffective one.

Step-by-Step: Blocking Google Ads

To save myself some typing, let’s start with a trivial MV3 extension I built for web developers (which should soon be obsolete 🤞).

Clone the code to your local disk. Create a new file named adsense.json:

[{
    "id" : 1,
    "priority": 1,
    "action" : { 
        "type" : "block"
    },
    "condition" : {
        "urlFilter" : "||pagead2.googlesyndication.com",
        "resourceTypes" : ["script"]
  }
}]

As you can see, Manifest v3 makes it utterly trivial to block requests to Google’s ad network.

Modify the existing manifest.json file with new values:

Remove the old localhost.json and add the new adsense.json file:

Testing the Extension

First, visit a page that has a Google ad on it to ensure that it loads as expected:

Now, inside Chrome, visit chrome://extensions, toggle the Developer Mode toggle, and click the Load Unpacked button. Select your extension’s folder:

Revisit the page with the Google ad and observe that it no longer loads. The Developer Tools console notes that the request for the JavaScript was blocked:

The MV3 ad-blocking approach is very easy to work with. Simply declare which sites you want to block, and they’re blocked. No slow and complicated network parsing or anything like that.

But…but… but…

Skeptical readers will note that this trivial approach blocks one type of Google’s ads, but not every type of Google ads. And that’s certainly true.

Fifteen years ago, I described the cat and mouse game between ad-hating users and advertising platforms as “a grenade fight, where both sides get infinite grenades.” Nothing has changed.

Economics rules everything around us.

Because sites need ad revenue to survive, they rely on ads not being blocked. Thus, ad networks have incentives to disrupt ad blockers. When a new blocking technique is adopted, the ad network responds. There are entire businesses built around trying to coax/annoy users to allow ads; one of the biggest is a platform called Admiral. Because I browse with Edge’s Tracking Prevention: Strict enabled, almost every ad-supported site I visit pops a banner like this one:

Some sites allow a “Continue without supporting us” link, and some do not. It’s the site’s choice. Major publishers might not rely on third-party frameworks and instead perform their own ad-blocking detection.

In response, some ad-blockers introduce ad-blocker-blockers, and the publishers then have ad-blocker-blocker-blockers. And the battle rages on, with dueling JavaScript files burning your battery while a quiet war rages under the surface.

-Eric

Attack Techniques: Encrypted Archives

Tricking a user into downloading and opening malware is a common attack technique, and defenders have introduced security scanners to many layers of the ecosystem in an attempt to combat the technique:

Web hosting providers may scan files served from their infrastructure.
Network gateways and proxies may scan files in transit from server to client.
Email web apps may scan files when received as attachments.
Client download managers scan files as they’re downloaded from the internet.
Client AV and OS security features scan downloaded files as they’re stored on disk or opened.

With all this scanning in place, attackers have great incentives to try to prevent their malicious code from detection up until the moment that a user is infected.

One technique attackers use to avoid getting blocked is delivering the malware inside an archive: for example, using a .zip, .rar, or .7z file. Unfortunately for attackers, defenders long ago caught on to this technique and enhanced scanners to peek inside archives. For example, if you download a .zip file in Chrome or Edge, the browser will decompress the archive file and scan the files within it using SafeBrowsing or SmartScreen. Client AV software will scan inside archives on disk, etc.

Attackers were forced to take another step — encrypting the archive using a password that they share with the user.

Requiring that the end-user type a password to get at the malicious content adds some friction– users might not understand how to do so, or might get suspicious if this isn’t something they’re used to doing. However, the tradeoff is that many of the security scanners (at the web host, gateway, and download manager) will not be able to peek inside the archive¹ to hunt out malicious content. Only security scanners run after the archive’s content is extracted will have the opportunity to block the malware.

This attack works best when the attacker has established some pretext for the file being encrypted; e.g. claiming that it contains private content like financial records, a “free trial” of a paid app, or other types of illegal programs like keygen/cracking software.

https://www.404media.co/a-network-of-ai-nudify-sites-are-a-front-for-notorious-russian-hackers-2/

This same encryption technique is sometimes used to hide URLs used in phishing attacks: The attacker sends the user a phishing link inside an encrypted PDF or ZIP file, and thereby scanners that would ordinarily block the phishing link are blinded.

What can a security administrator do to combat this threat vector?

First, help your users understand that encrypted archives are inherently more risky than average. Consider blocking or quarantining encrypted archives, or enable prompting the user for the password if your security software supports the option. Ensure that your users exclusively use archive extraction software that correctly propagates the Mark-of-the-Web, so that client security software is able to detect files extracted from archives that originated from untrusted sources.

Stay safe out there!

-Eric

¹ In rare cases, a security program might be able to decrypt the archive (if it uses an extremely common password, or if it’s configured to detect the password in an email message or download page) but this is, to the best of my knowledge, extremely uncommon.

Welcome to Fall, I guess?

Two months without a blog post? Sheesh. A lot has happened in two months, although perhaps nothing especially interesting.

I splurged on a new laptop, a Lenovo P1 Gen7 (22-core Ultra 9 185H and 64 gigs of memory). It’s big, heavy, and expensive, but it’s nice to have a PC that isn’t super-slow.

In July, the kids and I visited Maryland and went to Hershey Park and Tree Trekkers.

Coasters were ridden, Grandpa’s wallet was lightened

We then joined the family on a cruise to Bermuda and CocoCay. The boat was the older/smaller Vision of the Seas, but we splurged on Suites and it was pretty nice. Bermuda was beautiful and we had a nice time on the beach and snorkeling.

The Bahamas port call was a bit of a letdown, but the worst day on a beach in the Bahamas is better than most days anywhere else. The waterpark on CocoCay was a lot of fun; we took a ton of runs down Daredevils’ Peak, the tallest waterslide in North America. I look forward to going back around New Year’s.

After the cruise, we headed back to Texas. Noah started 6th grade and tackle football. We all were happy to reunite with our (bigger) no-longer-kittens.

At work, I switched from being a Group Product Manager to an Architect, and flew to Redmond for the Windows Ecosystem Security Summit. I’ll be heading back in November for another week.

Coming up? Mostly work, with some daydreaming about winter break mixed in. I’ll be running some races throughout the fall and winter — the 10K Daisy Dash, the 10 Mile Run for the Water, the Austin Half Marathon, and the Galveston Full (gulp!) Marathon.

Browser Features: Find in Page

For busy web users, the humble Find-in-Page feature in the browser is one of the most important features available. While Google or Bing can get you to the page you’re looking for faster than ever before, once you get to that page, you’ve got to find the information you’re looking for¹, and that’s where Find-in-Page comes into play.

Fortunately, with the death of Adobe Flash, Find-in-Page sometimes works better than it did fifteen years decade ago, because 3rd-party plugin content couldn’t participate in the browser’s Find-in-Page feature. (Chromium’s PDF viewer plugin does use the browser’s Find-in-Page).

Unfortunately, the value of Find-in-Page has been on the decline in recent years, largely due to three trends:

Breaking information out over multiple pages
Virtualized DOMs (Lazy-loading)
Non-DOM web applications

How it works

Conceptually, Find-in-Page is simple: simply gather the text of the page, and then search it, highlight the matches, and allow the user to navigate between each.

As a browser developer, the UX simplicity is a facade over a complicated set of conditions:

Pages may be made up of multiple frames; some of those frames may be running in other processes, requiring cross-process, asynchronous communication
Pages are dynamic: their contents can change, and frames can be added/removed/modified at any time, including in the middle of a Find operation. A user can invoke a Find operation as the page loads, or as it’s navigating away.
Providing the user with feedback like a Match Count or playing a “ding” sound when no more matches are found gets quite complicated.
Moving the search bubble so that it doesn’t cover up the highlighted search result may be tricky.
Figuring out how search should behave for invisible, or collapsed elements requires thought.

In Edge, things get even more complicated, with its AI-powered “Find Related” feature making network calls and hunting for related terms:

Beyond all of these complexities, the nature of the modern web makes it harder for Find-in-Page to function as well that users hope it would.

Problem: Paging

The problem with paging is pretty simple– many sites serve ads on each page, and the simplest way to increase page views is to split content out over multiple pages so that the user must navigate to new pages to get all of their content. If the user hits CTRL+F on a page, only the content of that current page is searched. If the content you’re looking for appears on a later page, you won’t find it until you visit that later page.

There’s no easy answer here… many problems in software are the direct result of economics, and this one is no different.

Problem: Virtualized DOMs

In other cases, a page might load content dynamically for performance reasons — loading tons of content “below the fold” might result in wasting the user’s memory or bandwidth for things they’ll never see. Returning more content into the page might put additional load onto the server, so it might use Intersection Observer or other techniques to figure out what content should be visible and not add invisible content to the DOM.

New features like content-visibility aim to allow web developers to get the performance benefits of virtual DOMs while solving some problems like Find-in-Page.

Problem: Non-DOM Pages

On Google Docs, if you invoke the browser’s native Find experience from the … menu, you get this surprising outcome where most instances of what you’re searching for aren’t found even while they’re literally in bold text in the middle of the visible page:

You can see a similar effect in Microsoft’s Web version of Excel:

If you use the Developer Tools, it’s easy to see what’s going on here: The entire content area of the document and spreadsheet are HTML5 Canvas elements, meaning that there’s no DOM to search at all:

To address these problems, web applications may take over the CTRL+F keystroke to pop up their own Find experience, like the Find UX in Google Docs:

Security / Privacy Implications

Most Web users may expect that websites cannot determine what they’re searching for within a web page. That expectation is faulty– there are a number of tricks a website can use to determine what the user is searching for, ranging from detecting how the browser scrolls to matches, to replacing the Find UX entirely with a lookalike (since the Find box is below the Line-of-Death).

¹Unless the search engine takes advantage of a new web platform feature called Scroll-to-Text-Fragment, which deserves a blog post all its own given its usefulness and subtle security implications.