Cloaking, Detonation, and Client-side Phishing Detection

Today, most browsers integrate security services that attempt to protect users from phishing attacks: for Microsoft’s Edge, the service is Defender SmartScreen, and for Chrome, Firefox, and many derivatives, it’s Google’s Safe Browsing.

URL Reputation services do what you’d guess — they return a reputation based on the URL, and the browser will warn/block loading pages whose URLs are known to be sources of phishing (or malware, or techscams).

Beyond URL reputation, from the earliest days of Internet Explorer 7’s phishing filter, there was the idea: “What if we didn’t need to consult a URL reputation service? It seems like the browser could detect signals of phishing on the client side and just warn the user if they’re encountered. And they could maybe even tattle on the URL to the cloud.“

Client-side Phishing Detection seems to promise a number of compelling benefits.

Benefits

A major benefit to client-side detection is that it reduces the need for service-side detonation, one of the most expensive and error-prone components of running an anti-phishing service. Detonation is the process by which the service takes a URL (collected from an email, text message, post, document, etc) and attempts to determine whether that URL leads to a malicious page.

The problem is that this process is expensive (requiring a fleet of carefully secured virtual machines to navigate to the URLs and process the resulting pages) and under constant attack. Attackers aim to fool service-side detonators by detecting that they’re being detonated and cloaking their attack, playing innocent when they know that they’re being watched by security services.

Many of the characteristics that attackers look for to recognize Detonators (hints of a virtual machine, loading from a particular IP range, etc) inherently fail when client-side detonation is performed from the end-user’s device, because the attackers must show their uncloaked attack page to the intended victim (end-user) if they hope to steal their credentials.

Beyond detonation improvements, browser vendors might find that client-side phishing detection reduces other costs (fewer web service hits, no need to transfer large bloom filters of malicious sites to the client). In the URL Reputation service model, browser vendors must buy expensive threat intelligence feeds from security vendors, or painstakingly generate their own, and must constantly update based on false positives or false negatives as phishers rapidly cycle their attacks to new URLs.

Beyond the benefits to the browser vendors, users might be happy that client-side detection might have better privacy properties (no web service checks) and possibly faster or more-comprehensive protection.

So, how could we detect phish from the client?

Clientside ML

Image Recognition

One obvious approach to detecting a phishing site is to simply take a screenshot the page and compare it to a legitimate login site. If it’s similar enough, but the URL is unexpected, the page is probably a phish. Even back in 2006, when graphics cards had little more power than an Etch-a-Sketch, this approach seemed reasonably practical at first glance.

Unfortunately, if the browser blocks fake login screens based on image analysis, the attacker simply needs to download the browser and tune their attack site until it no longer triggers the browser’s phishing detectors. For example, a legitimate login screen:

…is easily tuned by an attacker such that it no longer trips the clientside detection, while looking equivalent to the vast majority of humans:

Even more subtle changes might work; the field of adversarial ML studies how to confuse image processing models as in this wild example:

However, humans are busy, distracted, and easily fooled, such that attackers don’t even need to be especially clever. Here are two real-world phishing attacks that lured user’s passwords despite not looking much like the legitimate login screen:

Text and Other Metadata

If image processing is too prone to false negatives or has too high a computational cost on low-end devices, perhaps we might look at evaluating other information to recognize spoofing.

For example, we can extract text from the title and body of the page to see whether it’s similar to a legitimate login page. This is harder than it sounds, though, because it’s trivial to add text to a HTML page that is invisible to humans but could trip up an extraction algorithm (white-text-on-white-background, 1 pixel tall, hidden by CSS, etc). Similarly, an attacker might pick synonyms (Login vs. Sign In, Inbox vs Mailbox, etc) such that the text doesn’t match but semantically has the same meaning. An attacker might use multiple character sets to perform a homoglyph spoof (e.g. using a Cyrillic O instead of a Latin O) so that text looks the same to a human but different to a text comparison algorithm. An attacker might use Z-Order or other layout tricks to make text appear in the page in a particular order that differs from the order in the source code. Finally, an attacker might integrate all or portions of text inside carefully-positioned graphics, such that a text-only processor will fail to recognize it.

Making matters more complicated, many websites are dynamic, and their content can change at any time in response to users’ actions, timers, or other factors. Any recognition algorithm must decide how often to run — just on “page load”, or repeatedly as the content of the page changes? The more expensive the recognition algorithm, the more important the timing becomes for performance reasons.

API Observation

For tech scam sites, image and text processing suffer similar shortcomings, but API call observation holds a bit more promise. Most tech scam sites work by abusing specific browser functions, so by watching calls to those functions we may be able to develop useful heuristics to detect a likely attack-in-progress and then do deeper checks for additional evidence. The MalwareBytes security extension uses this approach:

… as does Edge’s new Scareware Blocker.

Given the challenges of client-side recognition (false positives, false negatives, and attacker tuning), what else might we do?

Keystrokes

One compelling approach to finding phish is to just wait for the user to enter their password in the wrong place. While it sounds ridiculous, this approach has a lot of merit — it mitigates (to varying degrees) false positives, false negatives, and attacker-tuning all at once.

Way back in 2006, Microsoft explored this idea and ended up filing a patent (Client side attack resistant phishing protection):

When I joined Google’s Chrome team in 2016, I learned that Google had built and fully deployed a Password Alert extension (open-source) on its employee desktops. If an employee ever typed their Google password into a non-Google Login page, the extension would leap into action. One afternoon, I was distracted while logging into a work site and accidentally switched focus into a different browser window while typing my password. I barely taken my finger off the final key before the browser window was taken over by a warning message and an email arrived in my inbox noting that my password had been automatically locked out because I had inadvertently leaked it. While this extension worked especially well for Google Accounts, available variants allow organizational customization such that an enterprise can force-deploy to their users and trigger reporting to backend APIs when a password reuse event is discovered. The enterprise can then either lock the user’s account or the target site can be added to an allowlist of legitimate login pages.

Chrome later integrated a similar feature directly, not using an extension.

Windows 11 Enhanced Phishing Protection

In 2022, Microsoft took our 2006 idea to the next level with the Enhanced Phishing Protection (EPP) feature of Windows 11 22H2. EPP goes beyond the browser, such that if you type your Windows login password anywhere in Windows (any browser, any chat window, any note-taking app, etc), Defender SmartScreen evaluates the context (what URL was loaded, what network connections are active, etc) and either warns you of your risky action or suggests changing your password:

This service is provided by webthreatdefsvc and its options are controlled by policy or the UI options in the Windows Security App:

Enterprises who have deployed Defender Endpoint Protection receive alerts in their security.microsoft.com portal and can further remediate the threat.

The obvious disadvantage of the “Wait for the bad thing to happen” approach is that it might not protect Patient 0 — the first person to encounter the attack. As soon as the victim has entered their password, we must assume that the bad guy has it: Attackers don’t need to wait for the user to hit Enter, and in many cases would be able to guess the last character if the phishing detector triggered before it was delivered to the app. The best we can do is warn the user and lockdown their account as quickly as possible, racing against the attacker’s ability to abuse the credential.

In contrast, Patient-N and later are protected by this scheme, because the first client that observes the attack sends its “I’ve been phished from <url>” telemetry to the SmartScreen service, which adds the malicious URL and blocks it from subsequently loading in any client protected by that URL reputation service.

Edge’s Scareware Blocker

Updated: 11/21/2025

Scareware sites abuse the HTML5 Fullscreen API to attempt to convince the user that their computer is infected and they must call the attacker for help. These sites now represent the most common attacks on the web, and they tend to rapidly change domains and use cloaking to evade blocking.

In 2025, Edge introduced a clientside detection for scareware sites. If a scareware attack is detected, the attack is interrupted and the user is presented with an explanatory block page. From that block page, they can report the malicious site to SmartScreen for blocking. Want to know more? Check out a demo video.

Conclusions

Attackers and Defenders are engaged in a quiet and ceaseless battle, 24 hours a day, 7 days a week, 366 days a year (Happy leap year!). Defenders are building ingenious protections to speed discovery and blocking of phishing sites, but attackers retain strong financial motivation (many billions of dollars per year) to develop their own ingenious circumventions of those protections.

Ultimately, the war over passwords will only end when we finally achieve our goal of retiring this centuries-old technology entirely — replacing passwords with cryptographically strong replacements like Passkeys that are inherently unphishable.

Stay safe out there!

-Eric

PS: Of course, after we get rid of passwords, attackers will simply move along to other attack techniques, including other forms of social-engineering. I hold no illusions that I’ll get to retire with the end of passwords.

x22i Treadmill Review

I love my treadmill, but two years in, I cannot recommend it.

On New Year’s Day 2022 I bought a NordicTrack x22i Incline Trainer (a treadmill that supports 40% incline and 6% decline) with the aim of getting in shape to hike Kilimanjaro. I was successful on both counts, losing 50 pounds in six months and summiting Kilimanjaro with my brother in mid-2023. Between its arrival January 24, 2022 and today, I’ve run ~1780 miles on it.

The Good

Most people I talk to about running complain about how awful treadmills are, describing them as “dreadmills” and horribly boring. While I’m not an outdoor runner, I’m sympathetic to their criticism, but it doesn’t resonate for me, at all.

The iFit video training series is awesome for me. I’m inspired to get on the treadmill to see what’s next on its 22″ screen (which feels larger). I’ve had the chance to walk, run, and hike all over the world: South America, Hawaii, Japan, Italy, Africa, Europe, Antarctica, and all over the US. I’ve run races I’ll likely never get to run in the real world, including races (mostly marathons) in Hawaii, London, Boston, Jackson Hole, New York, Chicago, Tanzania, and more I’ve probably forgotten. I’ve probably run the Kilimanjaro Half Marathon a dozen times at this point, and I’m currently working my way through a “Kilimanjaro Summit” hiking series, partially retracing my steps up the Western Approach. Along the way, I’ve learned lots of training tips, some phrases in foreign languages and history of lots of interesting places.

The treadmill hardware is pretty nice — the shock absorption of the deck is excellent and I’ve managed not to destroy my knees despite running thousands of miles. Running on pavement in the real world leaves me considerably more sore.

While iFit has a variety of annoyances (there are not nearly enough 10Ks or half marathons, and they don’t add new “hard” workouts fast enough) there’s no question in my mind that the iFit training classes are to thank for the success I’ve had in getting in shape.

The Bad

There are many inexpensive treadmills out there, and most of them don’t seem very sturdy or likely to support a serious and regular running habit.

I was serious about my goals and figured that I should spend enough to ensure that my treadmill would last and never give me a technical excuse not to run. Still, the cost ended up being pretty intimidating, with ~$3800 up-front and $1900 on later expenses.

x22i Treadmill (On Sale)$3170
Delivery and “White Glove” Assembly$299
Sales Tax$286
NordicTrack Heart Rate monitor arm band$100
iFit Video Training Subscription renewal (Years 2-3)$600
20-Amp dedicated circuit$970
Extended warranty (years 2-5)$300ish
Total 3-year costs for the x22i = $5725

Fortunately, Microsoft’s employee fitness program grants $1500 a year, and I was able to put the first year’s payment toward the treadmill and the following year I was able to pay for the subscription content renewal with $900 left over to defray the cost of the Kilimanjaro hike.

The Ugly

Unfortunately, my treadmill has been an escalating source of hassles from the very beginning. The assembly folks failed to fully screw in a few screws (they were sticking so far out that I assumed they used the wrong ones) and they cracked one of the water bottle holders. I complained to the NordicTrack folks and they refunded me the delivery/setup fee and within a few weeks came out to replace the broken water bottle holder.

Throughout the first year, my treadmill frequently tripped the circuit breaker; much to my surprise, the abrupt loss of power never resulted in me crashing into the front handrails, no matter how fast I was going. The treadmill was on a shared 15A circuit and while it was never supposed to approach that level of energy consumption, it clearly did. Sometimes, the trigger was obvious (someone turning on the toaster in the kitchen) while other times the treadmill was the only thing running. Eventually I hooked up a Kill-A-Watt meter and found that it could peak at 16-17 amps when starting or changing the incline, well above what it was supposed to consume, but within the technical specs. I eventually spent the money to get a dedicated 20A circuit, and was angry to discover that it was still periodically tripping. After months of annoyance and research, I eventually discovered that treadmills are infamous for tripping “Arc Fault Circuit Interrupt” breakers that are now required by Texas building code. Since having the electrician swap the AFCI breaker for the “old” type, I don’t think it has tripped again.

After all of the electrical problems, I invested in the extended warranty when it was offered, and I’m glad it did. Somewhere around the one year mark, my treadmill started making a loud banging noise. I looked closer and realized that two screws had broken off the bottom of the left and right rails and I assumed that was the source of the noise. Alas, removing the rails didn’t stop the banging, nor did having them replaced. Over the course of several months, techs came out to replace the side rails, idler roller, drive roller, belt, belt guide, and cushions. As November 2023, the treadmill no longer makes a banging sound, but it’s not nearly as quiet as it once was, and I’m expecting that I’ll probably need more service/parts within a few more months.

UPDATE: In October 2024, at around 2000 miles, the steel of the frame cracked where it holds the motor to the frame. It took two weeks for the technician to come verify that it was, in fact, broken and unfixable. Fortunately, the frame warranty is the longest one, at ten years. A few days later, I was offered either a replacement or a $3170 credit towards a new one. I spend a few days pondering whether to just buy another x22i, add $1500 of my own money for an x24, or get a non-incline 2450. The repair guy suggested that the x22i, with its motor at the back, is an especially unreliable model. :( Ultimately, I decided to get a new x22i, out $380 for shipping, assembly, and removal of the old one. Fingers crossed that the new one holds up better, or that if it does fail, it’s the frame again.

Closing Thoughts

From a cost/hassle point-of-view, I would be much better off getting a membership to the gym a half-mile down the block. I suspect, however, that much of my success with regular running comes from the fact that the treadmill lives between my bedroom and my home office, and it beckons to me every morning on my “commute.” The hassle of getting in the car, needing to dress in more than a pair of sweaty shorts, etc, would give me a lot of excuses to “nope” out of regular runs.

When I first was shopping for a treadmill, someone teased me and suggested that I make sure it had a good bar for hanging clothes on, since that’s probably the most common job for home treadmills. I managed to avoid that trap, and I’ve fallen in love with my treadmill despite its many flaws.

I don’t know whether other treadmills at a similar price point are of higher quality, or whether spending even more would give better results, but it almost doesn’t matter at this point — the iFit video content is the best part of my treadmill, and I don’t think any other ecosystem (e.g. Peloton) is comparable.

-Eric

PS: If I end up replacing my treadmill in a few years, I might get a “regular” treadmill rather than an Incline Trainer, because I don’t use the steep inclines very often and I think that capability adds quite a bit of weight and perhaps some additional components that could fail?

A Cold and Slow 3M Half

My second run of the 3M Half Marathon was Sunday January 21, 2024. My first half-marathon last year was cold (starting at 38F), but this year’s was slated to be even colder (33F) and I was nervous.

For dinner on Saturday night, I had a HelloFresh meal of meatballs and mashed potatoes, and I went to bed around 9:45pm. I set an alarm for 6, but I woke up around 5:15 am and lingered in bed until 5:30. I drank a cup of coffee right away and then had a productive trip to the bathroom. I ate a banana and had another cup of coffee while I prepped my gear and got dressed.

I put on my new UnderArmor leggings and shorts with the number that I’d attached the night before. I packed an additional running shirt in case it was cold enough to double-layer; something I’d never done before but the forecast called for 33 degrees, five degrees colder than last year’s cold run. I also put on a pair of $3 disposable cotton gloves that I’d picked up at the packet pickup expo the day before. I wore new Balega socks and my trusty orange Hokas (my new ones aren’t quite broken in yet).

My water bottle’s pouch would hold my car key, an iPhone SE to provide tunes streamed to one Bluetooth earpiece (the other died months ago) and snacks: a pack of Gu gummies and some Jelly Belly Energy beans (which I ended up liking most).

I left the house around 6:45 for the 7:30 am race. While waiting at a light in the parking traffic I concluded that I definitely was going to need that second shirt, so I put it on under my trusty Decker Challenge shirt that brought me to the top of Kilimanjaro.

By 7:22 I had parked and was waiting in a long line for a porta-potty near the start, debating about whether or not I should just skip it and go find my pace group. Ultimately, the race began just before I had a turn, although it was nice to dispose of that second cup of coffee. Alas, I was forced to start with the 2:40 pacers. I started my Fitbit a minute or two before my group made it across the starting line. Alas, my second watch (an ancient Timex I found somewhere) crashed when I tried to start it as I crossed the starting line.

I spent the first mile passing folks and by the second mile marker I’d reached the 2:05 pace group. For the next few miles, the 2:00 pace group was in sight in the distance, but I’d never caught up to them, despite my hope of running most of the race with the 1:55 pace group. I consoled myself that I’d probably crossed the start line two minutes after the 2:00 group so my dream of finishing in under 2 hours was probably still possible.

Around mile 5, my energy started to flag, but shortly thereafter an 8yo in a tutu running ahead of me guilted me into realizing that this wasn’t as hard as I was making it out to be. I passed her in about half a mile, grateful for the boost.

Shortly after mile 6, I discarded my gloves which had served me well. By this point I was taking short walking breaks and had concluded that I was unlikely to set any PRs.

Miles 6 through 9 were full of signs. My favorite was the 3yo boy holding a sign that said “This seems like a lot of work for a banana“– I told him it was the best one I’d seen. I groaned a bit at some of the signs held by twenty-somethings; one woman’s read “Find a cute butt and follow it” while another proclaimed: “Wow, that looks really long and hard!” Like last year, around mile 9 I stopped for a pee break although this time it was almost nothing… I had sipped under 16 ounces on the entire run.

By mile 7, my torso was starting to get a bit warm in my double shirts, but in another few miles the breeze had picked up and I was glad that I had them. Sheesh, it was chilly.

Amazingly, nothing hurt. My feet felt good. My legs felt good. My throat and lungs felt fine. My lips, wearing a swipe of chapstick, were fine. My thighs and chest, coated in BodyGlide, were not chafing anywhere. I didn’t have any weird aches in my arms or back. The closest thing I had to any pain was the bottom of my nose, which was getting chapped in the cold.

This year, I was anticipating the two hills downtown, and while I can’t say that I ran up them, they were much less demoralizing than last year. As the finish line approached, it had been a few miles since I’d seen my last pacer, but I figured I was somewhere in the 2:13-2:18 range. I idly hoped I’d still beat my time from last year’s slow Galveston Half.

Ultimately, I crossed the finish with a chip time of 2:09:24, a 9:52/m pace.

I wish FitBit made it easier to trim their data to the actual run portion of the race :)

This year, I made sure not to blow by the volunteers passing out the medals just over the finish line.

Tired but feeling like I could’ve easily run a few more miles at a slow pace, I hopped the bus back to the starting point. I grabbed my phone and a jacket from the car and walked a mile to the bagel shop to get a coffee and celebratory breakfast sandwich… the day felt even colder. Back home on the couch after a long warm shower, I signed up for next year’s race.

Next month is the Galveston Half Marathon. I hope to run the race with the 2:00 pacer the whole way, but I’ll settle for beating 2:09, five minutes faster than last year’s effort.

The Blind Doorkeeper Problem, or, Why Enclaves are Tricky

When trying to protect a secret on a client device, there are many strategies, but most of them are doomed. However, as a long-standing problem, many security experts have tried to chip away at its edges over the years. Over the last decade there’s been growing interest in using enclaves as a means to protect secrets from attackers.

Background: Enclaves

The basic idea of an enclave is simple: you can put something into an enclave, but never take it out. For example, this means you can put the private key of a public/private key pair into the enclave so that it cannot be stolen. Whenever you need to decrypt a message, you simply pass the ciphertext into the enclave and the private key is used to decrypt the message and return the plaintext, crucially without the private key ever leaving the enclave.

There are several types of enclaves, but on modern systems, most are backed by hardware — either a TPM chip, a custom security processor, or features in mainstream CPUs that enable enclave-style isolation within a general-purpose CPU. Windows exposes a CreateEnclave API to allow running code inside an enclave, backed by virtualization-based security (VBS) features in modern processors. The general concept behind Virtual Secure Mode is simple: code running at the normal Virtual Trust Level (VTL0) cannot read or write memory “inside” the enclave, which runs its code at VTL1. Even the highly-privileged OS Kernel code running at VTL0 cannot spy on the content of a VTL1 enclave.

DLLs loaded into an enclave must be signed by a specific class of certificate (provided by Azure Trusted Signing) and the code’s signature and integrity are validated before it is loaded into the enclave. After the privileged code is loaded into the enclave, it has access to all of the memory of the current process (both untrusted VTL0 and privileged VTL1 memory). In-enclave code cannot load most libraries and thus can only call a tiny set of external library functions, mostly related to cryptography.

Security researchers spend a lot of time trying to attack enclaves for the same reason that robbers try to rob banks: because that’s where the valuables are. At this point, most enclaves offer pretty solid security guarantees– attacking hardware is usually quite difficult which makes many attacks impractically expensive or unreliable.

However, it’s important to recognize that enclaves are far from a panacea, and the limits of the protection provided by an enclave are quite subtle.

A Metaphor

Imagine a real-world protection problem: You don’t want anyone to get into your apartment, so you lock the door when you leave. However, you’re in the habit of leaving your keys on the bar when you’re out for drinks and bad guys keep swiping them and entering your apartment. Some especially annoying bad guys don’t just enter themselves, they also make copies of your key and share it with their bad-guy brethren to use at their leisure.

You hit on the following solution: you change your apartment’s lock, making only one key. You hire a doorkeeper to hold the key for you, and he wears it on a chain around his neck, never letting it leave his person. Every time you need to get in your apartment, you ask the doorkeeper to let you in and he unlocks the door for you.

No one other than the doorkeeper ever touches the key, so there’s no way for a bad guy to steal or copy the key.

Is this solution secure?

Well, no. The problem is that you never gave your doorkeeper instructions on who is allowed to tell him to unlock the door, so he’ll open it for anyone who asks. Your carefully-designed system is perfectly effective in protecting the key but utterly fails in achieving the actual protection goal: protecting the contents of your apartment.

What does this have to do with enclaves?

Sometimes, security engineers get confused about their goals, and believe that their objective is to keep the private key secret. Keeping the private key secret is simply an annoying requirement in service of the real goal: ensuring that messages can be decrypted/encrypted only by the one legitimate owner of the key. The enclave serves to prevent that key from being stolen, but preventing the key from being abused is a different thing altogether.

Consider, for example, the case of locally-running malware. The malware can’t steal the enclaved key, but it doesn’t need to! It can just hand a message to the code running inside the enclave and say “Decrypt this, please and thank you.” The code inside the enclave dutifully does as it’s asked and returns the plaintext out to the malware. Similarly, the attacker can tell the enclave “Encrypt this message with the key” and the code inside the enclave does as directed. The key remains a secret from the malware, but the crypto system has been completely compromised, with the attacker able to decrypt and encrypt messages of his choice.

So, what can we do about this? It’s natural to think: “Ah, we’ll just sign/encrypt messages from the app into the enclave and the code inside the enclave will validate that the calls are legitimate!” but a moment later you’ll remember: “Ah, but how do we prevent the protect that app’s key?” and we’re back where we started. Oops.

Another idea is that the code inside the enclave will examine the running process and determine whether the app/caller is the expected legitimate app. Unfortunately, this is extremely difficult. While the VTL1 code can read all of app’s VTL0 memory, to confidently determine that the host app is legitimate would require something like groveling all of the executable pages in the process’ memory, hashing them, and comparing them to a “known good” value. If the process contains any unexpected code, it may be compromised. Even if you could successfully implement this process snapshot hash check, an attacker could probably exploit a race condition to circumvent the check, and you’d be forever bedeviled with false-positives caused by non-malicious code injection from accessibility utilities or security tools.

In general, any security checks from inside the enclave that look at memory in VTL0 are potentially subject to a TOCTOU attack — an attacker can change any values at any time unless they have been copied into VTL1 memory. Microsoft Security wrote an important summary of things your code needs to do to in order to program securely with enclaves.

Another idea would be to prompt the user: the code inside the enclave could pop up an unspoofable dialog asking “Hey, would you like me to sign this message [data] with your key?” Unfortunately, in the Windows model this isn’t possible — code running inside an enclave can’t show any UI, and even if it could, there’s nothing that would prevent such a confirmation UI from being redressed by VTL0 code. Rats.

Conclusion

Before you reach for an enclave, consider your full threat model and whether using the enclave would meaningfully mitigate any threats.

For example, token binding uses an enclave to render cookie theft useless– while token binding doesn’t mitigate the threat of locally-running malware abusing cookies via a sock-puppet browser, it does mitigate the threat of cookies theft via XSS/cookie leaks. Furthermore, it complicates the lives of malicious insiders using tightly-locked down corporate PCs at financial services firms– those PCs are heavily monitored and audited, so forcing an attacker to abuse the key on device is a significant improvement in security posture. (The upcoming Device Bound Session Credentials feature has similar properties and may be more deployable than token binding was).

A similar scenario involves Hardware Security Modules (HSMs) used in code-signing and certificate issuance scenarios: while the overall security goal is to prevent misuse of the key, preventing the attacker from egressing the key improves the threat model because it allows other components of the overall system (auditing, alerting, AV, EDR/XDR, etc) to combat attackers attempting to abuse the unexportable key.

-Eric

PS: A great discussion of hardware-backed enclaves on popular phones can be read here.

PPS: I turns out I wrote a much shorter version of this post (with a similar metaphor) previously in response to a bug report.