text/plain

Best Practices for SmartScreen AppRep

Last year, I wrote about how Windows integrates SmartScreen Application Reputation to help ensure users have a secure and smooth experience when running downloaded software.

tl;dr: When a user runs a downloaded program, a call to SmartScreen’s web-based reputation service is made, and four possible outcomes can occur:

“Known Good” SmartScreen’s Web Service indicates that the file is “Known safe”, and the program runs without prompting.
“Known Bad” SmartScreen’s Web Service indicates that the file is “Known malicious”, and the user is blocked from running the software via a red warning dialog.
“Unknown” SmartScreen’s Web Service indicates that the file is “Unknown,” and the user is interrupted from running the software via a blue confirmation dialog.
“Offline”: SmartScreen’s Web Service is offline or unreachable and the user sees a notice that SmartScreen cannot help decide whether the software is safe to run. The user sees a blue confirmation dialog.

As a software developer, it’s natural that you’ll want to have your apps classified as “Known Good” to streamline the installation of your software.

This post explores how to achieve that.

Building Positive Reputation

The Application Reputation service builds reputation based on individual file hashes (SHA256) and/or the certificates used to sign those files. The following are best practices we recommend.

Sign Your Files

Because any update to a file changes its hash, this means that new files that are unsigned have no reputation by default. To avoid triggering warnings after every update, be sure to sign your software using an Authenticode certificate.

Because your certificate can be used to sign files for years, it will accumulate reputation that will accrue to files you sign in the future. To maximize reputation data, you should aim to use the same certificate for as long as possible. Certificates periodically expire, however, and you may find that when you being using a new certificate if must build reputation again.

To help ensure that all of your good reputation accrues in one place, try to sign everything with a single certificate (e.g. do not use a different certificate for each product, or a different certificate for each of your company’s divisions, or anything like that).

From ~2013 to ~2019, all files signed by an Extended Validation Authenticode Certificate were given a “positive” reputation by default, but EV certificates are no longer treated specially — every certificate must build reputation individually.

Sign All Files

Beyond .exe and .dll files, you can also sign .msi Installers, script files (PowerShell, VBScript, JavaScript), and .cab Archives.

While SmartScreen only checks the reputation of the main file that the user executes (aka its “entry point”), you should sign all files to help protect them from tampering, and to ensure your app works correctly when Win d ows 11’s Smart App Control feature is enabled. The Windows 11 Smart App Control feature goes further than SmartScreen and evaluates trust/signatures of all code that is loaded by the Windows OS Loader and script engines.

NOTE: Smart App Control’s signature check does not currently support ECC signatures.

Do Not “Cheat” Authenticode

As I’ve explained in the past, it’s possible to shove additional data inside an signed file without Windows considering its signature broken. This technique is commonly used by installers to include additional metadata on every download. However, it’s also possible to set a EnableCertPaddingCheck registry key that rejects any additional data inside the certificate padding.

When that key is set, any signed file with invalid padding is considered unsigned. We recently encountered this issue on Microsoft’s corporate network, when an employee downloaded the Acrobat installer from Adobe. They were worried to see that AppRep said the file had an unknown publisher and was not common:

If you look at the raw bytes of the file, you can see Adobe has injected a base64-encoded blob in the signature padding:

If you compare the underlying network traffic between the working and Unknown Rep cases, you can easily see the problem. On a clean machine where the EnableCertPaddingCheck registry key is not set, the call to AppRep indicates that the file is signed and includes information about the signer:

However, on a machine with EnableCertPaddingCheck set to 1, the call indicates that the file is unsigned, and the “flat file hash” (SHA256) of the file is passed in the hash field:

Because the whole point of putting metadata in the padding is to put different data there for each download, the flat file hash for each download is different, preventing the SmartScreen AppRep cloud service from recognizing that this file is (almost entirely) the same as what others have downloaded.

Always Follow the Objective Criteria

Ensure that everything you install follows Microsoft’s Objective Criteria for well-behaved software. Adherence to the criteria ensures that users will not be surprised or upset about the behavior of your software.

Violations of the objective criteria can cause SmartScreen, Microsoft Defender, and 3rd-party security products to treat your app as Malware or Potentially Unwanted Software.

Escalations

If your software is blocked by the red “known bad” dialog box:

… you should investigate to determine whether anything unexpected is included; for instance, was malware injected into your product by a supply-chain attack? You should upload your files to VirusTotal.com and what, if any, detections occur across the universe of security products. If you cannot find any indications of compromise, report the potential false-positive to Microsoft Security Intelligence by uploading your files to the WDSI portal.

If your software is blocked by the blue “unknown” dialog box:

… don’t despair. Ensure that you’ve followed the best practice of signing your software with a certificate, and then just wait. As your software gets downloaded by more users around the world (“increasing prevalence”), its honorable behavior will be noted and eventually its reputation will move into the “Known good” category.

Bypass for Enterprises

Some enterprises build and distribute their own software internally and wish to avoid SmartScreen prompts for their applications. For security reasons, you can and should still sign internally-developed software. However, internally-developed software might never be used broadly enough to organically build a positive reputation within the SmartScreen Service.

Note: Microsoft Defender for Endpoint customers might reasonably expect that creating an Allow Indicator for a certificate would allow files signed by that certificate to bypass SmartScreen AppRep. This is a reasonable expectation, but unfortunately does not presently work (as of November 2024).

Enterprises are in control of how their users get software, and most internal software deployment systems do not result in SmartScreen checks. Only software downloaded by web browsers or copied from network shares using Windows Explorer is likely to get a Mark-of-the-Web that results in consulting SmartScreen’s reputation service.

If using Explorer or a web browser to obtain software is a core scenario for your enterprise, you can avoid SmartScreen checks by correctly configuring your Windows Security Zone settings. Using the Site to Zone Assignment List group policy (or the Internet Control panel), place your trusted software distribution sources into the Local Intranet security zone:

After doing so, files downloaded from the listed locations will no longer be tagged with the Mark-of-the-Web, and executing them will no longer perform reputation checks against SmartScreen.

If you use the Microsoft Edge web browser, you should also use Group Policy to set the SmartScreenForTrustedDownloadsEnabled setting to 0 (Disabled) to prevent the browser from itself performing reputation checks while downloading files from your trusted locations.

Stay safe out there!

-Eric

Defensive Technology: Controlled Folder Access

Most client software’s threat models (e.g. Edge, Chrome) explicitly exclude threats where the local computer was compromised by malware. That’s because, without a trusted computing base, it’s basically impossible to be secure against attackers. This concept was immortalized decades ago in the Ten Immutable Laws of Security:

In the intervening years, new technologies (like Secure Enclaves) have been launched in an attempt to provide some defenses even when a PC has been compromised, but they remain very limited in their capabilities. In my view, preventing initial access represents the most important security investment we can make.

Protecting Files

Nevertheless, there’s value in defense-in-depth and making life harder for attackers even after they get access to a PC.

In that vein, Windows 10 has a feature called Controlled Folder Access (CFA) which aims to help protect against ransomware. In the common ransomware attack, malicious native code running on the user’s machine begins encrypting their files with a randomly-generated key that is sent to the attacker. After files have been encrypted, a ransom note is shown demanding that the user pay money to get the decryption key.

CFA impedes this attack by preventing applications with “unknown” reputation from touching files in sensitive/protected folders, including the user’s Documents, Pictures, and Favorites folders, as well as any folders selected by the user. You can enable CFA using the Ransomware protection section of the Windows Security app:

Rather than moving sensitive files to a protected vault, it’s more like you’ve hired a bouncer to keep questionable apps out. If you want to protect more folders, click the Protected Folders link and choose the folders you’d like protected. If you need to allow an unknown app to access protected files, you can do so using the Allow an app link.

When Defender blocks access, you’ll see a small toast notification:

You can see blocked actions in both the Protection history section of the Windows Security app:

…as well as the Windows Defender node of the Windows Event Viewer:

If you’re managing devices with Intune or Group Policy, you can also enable CFA in “Audit Mode“, which will log untrusted accesses without blocking them.

How?

This feature depends upon the Windows Defender filesystem filter driver (WDFilter). From kernel mode, the filter watches for access to protected folders. If access is requested by an untrusted process, any Write permission bit is stripped from the request.

Non-Obvious Bits

While CFA is conceptually pretty simple, under the covers there’s a fair bit of complexity.

For one thing, various well-known and legitimate applications (like Microsoft Office) offer extensibility models that could be used to load malicious modules. Similarly, well-known applications (e.g. Notepad) could have malicious code injected into them. So, Defender has to watch what’s loaded into each process and may consider an otherwise “friendly” process tainted for the purposes of CFA.

Additionally CFA could be circumvented if a process directly accesses a disk volume using low-level APIs. If CFA blocks an unknown app from accessing a disk in this way, there’s no folder path to show, so the toast (perhaps confusingly) claims that CFA blocked the app from “making changes to memory.”

Finally, CFA must carefully handle all of the myriad equivalent ways in which a file’s path can be represented, for example:

Referring to local files using a UNC path and 8.3 filename syntax

Debugging

In some cases, understanding why a process has triggered CFA could be difficult. For example, several engineers at Microsoft recently found that running most command-line tools (like ping.exe) in the SYSTEM32 folder was triggering the making changes to memory toast. This seems very strange — a network tool like ping isn’t expected to be touching the local disk at all!

Using SysInternals’ Process Monitor to log the process behavior can reveal the root cause. Click Filter > Enable Advanced Output. Use the toolbar toggle button to filter to FileSystem events, and create a filter for Operation is IRP_MJ_CREATE:

Look through the IRP_MJ_CREATE events to find those for the C: volume:

In this case, we see that an AMD GPU driver amdkmdag.sys is opening the C: volume for Write Access as in the context of ping.exe as it shuts down, leading to the toast:

To discover whether a process is being treated as tainted by the engine, see the MPLog-##### file inside C:\ProgramData\Microsoft\Windows Defender\Support. For example, a value of TaintType:0x1 indicates that an untrusted module was loaded into the process.

Online Backup

Beyond Controlled Folder Access, Windows’ other built-in anti-ransomware approach is to enable online backup to a cloud file provider like OneDrive. The recovery experience offered by OneDrive depends on which level of account you have:

If Microsoft Defender detects a ransomware attack, it can send a signal to the cloud file provider, indicating the time of compromise, to stop syncing now-encrypted files to the cloud, and to subsequently ease the process of recovering pre-encryption files from the cloud. You can see the flow from OneDrive’s documentation:

Stay safe out there!

When a 3rd party antivirus application is installed and Microsoft Defender is disabled, attempting to open “Controlled Folder Access” from the Windows Start menu shows the following end message:

On Politics

I do not come from an especially political family. One parent has not voted in decades, and the other votes regularly, but is not an enthusiast and values harmony over potentially-divisive political discussions. Politically, I am left of center– the middle child in both age and political leanings: one brother to the left of me, and one brother to the right.

As a kid, my parents thought Republicans were the good guys. Bill Clinton’s depravity didn’t help matters at all. In high school, I had one Political Science teacher who challenged my default “I’m a Republican, I guess” posture, but I still skipped voting for Al Gore when I turned 18 as I didn’t care enough to bother in deep-blue Maryland. As a college freshman, I studied early American history and read many of the letters and musings of America’s founders, gaining a much broader understanding of the amazing story behind the founding of this nation, the risks the founders faced, and the compromises that linger to this day. Thus was birthed my interest in politics.

Our Revolutionary Founders (Jefferson in particular), concluded that the only way for a government to be responsive to its citizenry was to have it be periodically overthrown. Rather than suffer the losses and uncertainty of violent overthrow, the notion of regular, peaceful overthrow of the government was seen as the best approach. We call these peaceful revolutions elections.

“I am not an advocate for frequent changes in laws and Constitutions. But laws and institutions must go hand in hand with the progress of the human mind. As that becomes more developed, more enlightened, as new discoveries are made, new truths discovered and manners and opinions change, with the change of circumstances, institutions must advance also to keep pace with the times. We might as well require a man to wear still the coat which fitted him when a boy as civilized society to remain ever under the regimen of their barbarous ancestors.”
Thomas Jefferson

I watched The West Wing and thought it was brilliant. John Kerry visited Microsoft, gave a nice speech, and I read his book and voted for him in 2004. In 2008, I eagerly voted for Obama, although I greatly respected John McCain until his disastrous choice of running mate. I voted for Obama again in 2012 although I believe, gaffes aside, the country would be just fine under Romney. I watched The Newsroom, and I still believe that its opening scene is the best and most important five minutes of television ever made. In 2016, I voted for Hillary, despairing that Trump was even a contender for the nation’s highest office. In 2020, I supported Buttigieg and Warren in the primaries, holding my nose to vote for Biden in the general (I think Biden is an admirable patriot and his reward for a lifetime of public service should be a pleasant retirement, not working into his 80s.)

I was furious that Biden announced that he’d run again in 2024, and that both parties had failed to prepare the next generation of leaders to take over. For durable improvements, Systems must be treated as more important than Individuals.

On the first day of early voting, I voted for Harris, and thought she’d be a fine President for all Americans.

On The Issues

Guns are now the leading cause of death for American children. This is an insane travesty, and it’s a stain on our nation that so little has been done about it. More locally, Texas has significantly loosened its gun laws over the last twelve years, with predictably tragic results.

My thoughts on civilian ownership of firearms could fill a book. It’s not a simple issue, but the status quo is barbaric and unacceptable.

Climate Change: Austin just notched its hottest October in history. Atop Kilimanjaro in 2023, I saw the sad state of the depleted glaciers, and I worry that there may be nothing left when I summit again on New Years’ Eve just before the calendar ticks over to 2026. I can’t help but believe that technology is going to bail us out (yet again) and we’ll avoid the worst projections. But we are playing with fire, and we risk activating tipping points we don’t even know about. As with most things, the poor and powerless are going to pay most of the price.

Taxes: It’s common for those on the left to scream “Tax the rich! Make them pay their fair share!” This clip from West Wing is a great and important one– we should tax the rich more (Bill Gates and Warren Buffet, to their credit, both agree). But we must acknowledge that most of the wealthy pay much more in taxes than the average American, and it’s crucial to recognize that even if we taxed the rich at a rate of 100%, we would not be able to balance the budget for long. I’m particularly fond of someone’s flippant proposal that we should require the super-wealthy pay higher taxes, and in return get naming rights for infrastructure– $10M for a Post Office, $5M for a Bridge, etc.

Still, I’ve come around to the notion that the existence of billionaires is a policy failure, and there are absolutely insane loopholes in the tax code (e.g. this IRA hack, “step-up basis” on inheritances, etc) that should go away.

I’m infuriated that so many on the political right should be voting for higher taxes on those who can afford them, but do not do so because “the poor see themselves not as an exploited proletariat, but as temporarily embarrassed millionaires.” The rich see you as suckers.

Hard choices must be made, and our system is such that politicians are punished for making them.

The Spending side represents the other side of hard choices; as a society, we must choose how to allocate our limited resources. There are many worthy causes, but we cannot afford them all. Prioritization matters.

For example, the United States is set to spend a trillion dollars ($1,000,000,000,000!) on modernization of our nuclear arsenal, and I don’t think we talk nearly enough about whether that’s the best way to protect Americans against the true threats of the 21st century.

Unintended consequences abound: The US has the most expensive educational system and most expensive healthcare system in the world, and that’s not only because of greed– it’s partly because of some well-meaning policy choices we’ve made have had some awful and unforeseen consequences. We should not be afraid to change things, but we must also refine those changes as we observe their outcomes.

However, it’s almost as great a mistake to think of the United States’ budget as that of a family– money simply works very differently when you control the supply, interest rates, and other key aspects of the worldwide financial system.

Despite the complexity, common sense suggests some simple heuristics. When considering an expenditure, we should ask “Is it an investment, neutral, or worsening some larger problem? Is the money going to a fellow citizen, an ally, or an enemy?” I’d love to see the $800B-$1T a year we spend on the military-industrial complex (Trump proposes $1.5T in April 2026) pivoted to the future-industrial complex.

Abortion: I have a deep ambivalence about abortion– extremists exist on both sides of the divisive issue, and most folks refuse to think deeply about the implications of their own position, let alone listen to those on the other side. Any litmus test is a prime target for abuse.

After decades of thinking about this, I’ve settled into a simpler position: “Abortion is healthcare,” and believe that’s where the political realm should leave it.

Trump: Truer words were never spoken: “Trump is a poor man’s idea of a rich man, a weak man’s idea of a strong man, and a stupid man’s idea of a smart man.” This incurious, impetuous liar is remarkable only for his utter shamelessness.

Trump is both a symptom of great rot in American society, and an accelerator of that decay. Trump claims to “love the poorly educated“– not because he aims to improve their life and help their family attain the American Dream, but because he knows a sucker when he sees one.

Some more moderate Trump voters assume: “Meh, the worst of his plans are just talk, they’ll never actually happen.” They do not understand that Trump 2.0 is a version with far fewer checks, and they failed to heed the (belated) dire warnings of those who worked for him last time. In the same way that Trump uses the poorly educated as “useful idiots,” evil people much smarter than Trump (foreign and domestic) will use our President as their useful idiot.

Politicians: I’ve learned that there’s a class of Trump voters who know he’s an awful person but voted for him because he’s entertaining. They assume that politicians are ineffective and almost interchangeable liars, so the person in the Oval Office doesn’t really matter. That’s a position not entirely without support, but one that falls apart when our institutions and checks-and-balances fail or are subverted. Trump’s backers have been working to undermine our institutions for decades.

Some people have been convinced (often by folks like cosplay cowboy Harvard-educated Ted Cruz) that politicians are out-of-touch “elitists” who “think they’re better than us.” I’m reminded of this cartoon in the New Yorker:

In the same way that I want my manager at work to be smarter and harder working than I am, when I vote, I want to hire someone for the job who’s better than me in every way possible.

The Democrats: I am still furious that the Democratic Party supported Trump in the 2015 primaries, literally betting the future of our country on the idea that a racist rodeo clown couldn’t possibly lie his way into the most important job in the world. We all lost in that bet.

Bob Menendez should not have been granted the opportunity to resign; he should have been expelled.

Partisanship: Too many of us interested in politics follow it like sports– cheering for the home team (no matter what), and booing the opposition (no matter what), and demanding “loyalty” to one’s own party regardless of malfeasance or foolishness. Instead of finding common ground with our neighbors, we’re split into factions as many politicians prefer. The truth is that all states are shades of purple. As hard as the Republicans want to convince Texans that it’s a “Deep Red” state, more Texans than New Yorkers voted for Biden.

We should not be afraid to hold our own “side” (and selves) accountable, no matter our opinion of others.

The Supreme Court: For many years, the Supreme Court was the branch of government that I respected most. Justices mostly kept their mouths shut when it came to politics, and an appointment is not a stop on the way to a highly-paid lobbyist job. Justices never have to pander to the base to win their next election. Most importantly, unlike Congress, the Justices have to show their work in written opinions that will be scrutinized for decades. In many of the painful decisions overturning good (and evil) things, the Court has simply held that a given good thing requires that Congress write a law.

Neil Gorsuch concurring on the overturn of Trump’s illegal tariffs

Unfortunately, Congress has been mostly dysfunctional for many years now, failing to serve the American people in their constitutional duty.

More recently, however, the court has been tarnished by acceptance of improper gifts, political speech, and other scandals. More practically, a system where we incentivize or demand that the elderly work until they die (based on the vagaries of presidential elections) is a grotesque one. Thus far, the best proposal I’ve seen is to term-limit Justices to 18 years— a period long enough to help protect impartiality, but not so long as to make an appointment a life-sentence and the highest stakes action a Senate will ever conduct.

The American Electorate: Unfortunately, fear is a primary driver of voting behavior, and we humans have a set of base fears (particularly “fear of the other”) that politicians have learned to play like an instrument. This year, Ted Cruz spent millions of dollars on ads about immigrants and children’s genitalia, and he, the least popular Senator in a decade, sailed to reelection. Trump spent $65M on ads about genitals. As the world’s real challenges grow greater (aging populace, climate calamities, plagues, AI job displacement), greater fear is sure to follow.

The vote of an ignorant racist counts just as much as the thoughtful patriot, and “all the terrible people vote.” I’m as angry about politically apathetic non-voters as I am about the suckers who voted for the blowhards, the bigoted, and the hateful. Anyone who can look at Harris and Trump and think “Meh, no opinion here” needs to stop sniffing glue. And yes, the Republican party has been working for decades to make it harder for Americans to vote, but it’s just a weak excuse for many of the large block that do not vote:

While it’s easy to lament how disastrous the consequences of the 2024 elections will be, it’s important that elections have consequences. Without consequences, elections don’t matter. And if elections don’t matter, you’re no longer living in a democracy. As Dan Stevens taught in 7th grade civics, at a societal level, Democracy means you get the government you deserve.

Looking Forward

America is about to enter the “Finding out” phase of “Fuck around and find out.” Feedback loops are important, and the tighter they are, the more effective they are. Unfortunately, political feedback loops can take years or even decades between cause and effect.

After the 2024 election, the super-rich (Musk, Bezos, and the like) will be just fine. The merely-rich who supported Trump are going to find themselves like the UK Rich post-Brexit — when your entire economy gets kicked in the nuts, your personal tax rate is not the important factor in your wealth. An ebbing tide lowers all boats. As is common, the middle-class and poor are going to get hurt, and hurt bad, whether they supported Trump or not.

For now, though, I can’t let myself get too lost in despair for our world. Ultimately, we each only have so much control over our reality, and the main thing under our control is how we react to it. Catastrophizing (and anxiety more generally) is only useful if it inspires productive action.

This week, I’m taking a breath and looking around with both trepidation and curiosity. Maybe America, as a nation, isn’t better than this. But even if that’s the case, we can be.

Hope is a thing that fights in the dark.

-Eric
PS: I recently moved from Twitter to BlueSky.

Lenovo P1, Gen7: Meh

I’ve been a loyal user of Thinkpads for over twenty-five years now, and I currently own four (with another on loan from Microsoft).

In July, the screen on my Lenovo X1 Yoga Gen 6 failed at an inopportune time, and my 8yo broke the screen on my backup (T480S), so I rush-ordered a Lenovo P1 Gen7 workstation laptop (22 cores, 64gigs of memory) to stand in while awaiting warranty replacement on the Yoga’s screen. After massive discounts, the cost with tax fell to a still-spendy $2,890. (The Yoga screen replacement went well, and I later replaced the T480S’ screen myself for ~$79 from LaptopScreen.com).

Unfortunately, when my new P1 workstation arrived, I discovered to my dismay that it had the same problems as my nearly-unused X1 Extreme Gen 3 — the OLED touchscreen, while mostly beautiful, had super-distracting quirks — it seemed to flicker, and almost have a screen door effect under certain conditions (the newer laptop might be the same panel as on the older one). Fortunately, these screens have now been in the market long enough that I wasn’t gaslit into thinking it was “just a me problem,” and searches turned up the explanation — the screendoor effect comes from the touch-digitizer, and the flicker comes from the way OLED power-management is implemented. Fortunately, I was able to rush-order the same machine with the non-touch-LCD (21KV001CUS) for $2,381 so I could do a side-by-side comparison. Sure enough, the regular LCD seems fine, and saving $500 for a lower-end screen and a cheaper GPU (Ada 3000, RTX 4070) that seemed to benchmark almost as well.

The Gen7 is pretty fast (although building Chrome still takes ages) and the keyboard and big screen are great. The fans seem to run more than they should, (although bizarrely they’re silent as I write this post for the first time in weeks) and it’s not too crazy heavy. Replacing physical buttons for the trackpoint with a tiny section at the top of the trackpad was a surprisingly-annoying downgrade.

Unfortunately, the laptop’s reliability has been poor. First, the Nvidia drivers were blue-screening the system and I remembered why I don’t like gaming GPUs– I don’t play games, and their drivers tend to prioritize performance over stability. So I disabled the Nvidia and started using just the integrated (Intel) GPU.

More recently, the system started ignoring all input shortly after boot — the trackpoint/trackpad wouldn’t move the cursor, wouldn’t accept clicks, the keyboard wouldn’t work, etc. I thought I had narrowed down the problem to the “Virtual Lock” feature that aims to automatically lock the PC when you walk away. I’d never enabled the feature and even after turning off “User Presence Sensing” in the UEFI Settings the problem continued. Setting the Services (Virtual Lock Sensor and Elliptic Human Presence Detection) to Disabled in Windows Services and disabling the Virtual Lock Sensor in Device Manager’s Software Components appears to have helped…

Fingers crossed!

Update: It didn’t. There’s something else that seems to be causing this. I’ve killed off some more of Lenovo’s preloaded software, and learned that the hard hang lasts for ~100 seconds before the system becomes responsive again, forcing me to just “wait” rather than hard-rebooting.

Some recent update to the system also started causing terrible echo for my Teams calls, which I’m working around by using an external speaker/microphone. Lame.

Defensive Technology: Antimalware Scan Interface (AMSI)

Endpoint security software faces a tough challenge — it needs to be able to rapidly distinguish between desired and unwanted behavior with few false positives and false negatives, and attackers work hard to obfuscate (or cloak) their malicious code to prevent detection by security scanners.

To maximize protection, security software wants visibility into attack chains at their weakest — after any obfuscation has been stripped away, immediately before (or during) execution. Unfortunately, this is hard, because most security software hooks are either very low-level (e.g. kernel drivers watching file and process activity) and thus lack context, or very high-level (e.g. scanning downloaded files), where obfuscation is in place.

The Antimalware Scan Interface is a Windows platform mechanism that allows host applications to call out to security software before performing sensitive operations (e.g. running script, elevating UAC, invoking ActiveX objects, etc). The security software can scan the data buffers and return a “Allow”/”Deny” verdict based on its threat intelligence. AMSI is pluggable on both sides — any host application can call into AMSI, and any security software can receive the calls[1].

On a default Windows system, hosts include cscript, wscript, PowerShell, the .NET CLR Platform, UAC elevation prompting, WMI, VSS, and several Microsoft server products (including optionally SharePoint). Microsoft Office desktop applications call AMSI to scan VBA macros.

Security software like Microsoft Defender (and most 3rd party AV products) will scan the data buffers provided by the host and return verdicts based on their threat intelligence (signatures). Microsoft Defender for Endpoint also uses AMSI to implement several of its attack surface reduction rules that constrain the behavior of scripts.

AMSI protections help improve blocking in common initial access vectors (e.g. running a downloaded .js file) and improve protection over traditional AV signatures (which can be fooled by obfuscation) and behavior monitoring (which might detect misbehavior too late). It’s especially useful against fileless threats in which the attack code was never on disk for a traditional AV sensor to scan.

The documentation for application developers who want to call into AMSI is pretty good. For security software developers, there’s both documentation and an sample AMSI scanner on GitHub.

Limitations

The biggest limitation in AMSI is that a host application must call it– unlike most security software sensors, which rely on kernel drivers and process injection techniques to provide security for every application, AMSI requires the explicit participation of an application to call out to AMSI and respect its verdict.

Hosts must choose when and if they will call into AMSI– for example, a host might not choose to call AMSI if it doesn’t think a given script is potentially dangerous (e.g. the Windows Scripting hosts may only call AMSI if they see a potentially-dangerous object created). We call these triggers, and they must be selected carefully to balance performance and security.

For example, AMSI is never called to evaluate JavaScript running inside your web browser (Edge, Chrome, Firefox, etc). Unlike cscript and wscript which run “shell scripts” with access to powerful capabilities (writing files, launching processes, etc), browsers execute “web platform” JavaScript inside tight security sandboxes that prevent interaction with files and other objects on the system. Browsers compete on runtime performance, and the overhead of security scans on sandboxed script would yield a poor cost/benefit.

As with any other security software component, attackers have attempted to tamper with AMSI, using a wide range of techniques ranging from simple to ingenious. The majority of such techniques require that the attacker have at least partially-compromised the victim PC (to manipulate the execution environment), and a major goal of AMSI is to foil the attacks that would grant the attacker initial access to start with.

Future Brainstorming?

AMSI has been around for a long time now, and hasn’t seen a ton of changes over the years. However, there’s recently a greatly renewed interest in empowering security software without requiring that each vendor write its own kernel drivers, and the model used by AMSI is a great one for providing powerful visibility and control in user-space.

To that end, I’d love to see more scenarios for AMSI-like callouts. Most notably among them– URL Reputation. Today, myriad applications present, transmit, and load content from URLs, but security software often is not well-positioned to “see” those URLs. Except for security code directly integrated into browsers (e.g. SafeBrowsing for Chrome, SmartScreen for Edge), security software is often forced to “guess” where a given network request is going via low-level packet-sniffing. This approach (at best) supplies only the hostname (not the full URL) will soon become even less reliable, as browsers further improve their privacy against network sniffers. Imagine if every application could easily call out to the platform security software and ask “Hey, I’m going to go grab these 40 URLs. Any objections?” Apple has recently introduced such an API: NEURLFilter.

Another scenario is malicious browser extensions. Today, security software can watch the browser load an extension’s code out of its filesystem location and block those loads if the files are known to be malicious, but at the browser level this blocking has a poor UX. All the browser “sees” is that an extension failed to load, but it doesn’t know why. It will still think that the extension should be present and loading, and can’t tell the user that the file was blocked for security reasons. Browsers don’t expose an API for security software to even know what extensions are installed and enabled; security inventory software that wants to report extensions up to the SOC must parse browsers’ internal configuration files (which is unsupported) to try to determine which extensions are allowed. If browsers could call into AMSI before loading an extension, this could provide a much better UX.

A further scenario is invocation of App Protocols. App Protocols provide the easiest way for an attacker to escape a browser’s security sandbox and are thus of prime interest to attackers. However, security software typically doesn’t get to directly observe the URL launched by the browser — instead, it must rely on kernel sensors (e.g. this callback) to watch for CreateProcess calls, and this might not be sufficient– not every protocol invocation results in a CreateProcess call (e.g. some result in a COM object creation) and a kernel sensor lacks important context (e.g. the URL of the page that asked to launch the protocol). If browsers could call into AMSI before invoking an App Protocol, they could provide a better UX and improve threat intelligence.

Any other scenarios I haven’t thought of yet?

-Eric

[1] While any app can register their own AMSI provider to get called during UAC elevation prompting, the code must be run out-of-proc in a process protected by PPL and thus only Microsoft Virus Initiative partners can participate in UAC’s AMSI checks.