dev

ShellExecute Doesn’t

My oldest supported Windows application is a launcher app named SlickRun, and it’s ~24 years old this year. I haven’t done much to maintain it over the last few years, although it’s now available in 64-bit and runs great on Windows 10. (Thanks go to Embarcadero, who now offer a free “Community” edition of Delphi, the language/platform I ported SlickRun to circa 1994).

I still fix bugs in SlickRun from time to time, and as I was playing with Rust a few days ago I was reminded of one of the oldest limitations in my code– if you update your system’s %PATH% variable, those changes aren’t seen by applications/consoles spawned by SlickRun until you restart it. It’s particularly annoying because it’s so unexpected– users expect that command consoles launched by Win+R,cmd.exe,Enter will behave the same way as Win+Q,cmd,Enter, but the former consoles have the updated %PATH% while the latter do not.

While ShellExecute() sounds like it’s an API that causes the shell (aka Explorer) to execute something, in fact it does nothing of the sort.

Updating the Environment Block

The root cause of the “outdated path” problem is that processes launched via ShellExecute inherit the environment variables of their spawning process, and those environment variables (typically) are assigned as the process launches and never touched again. Because SlickRun starts with Windows, the %PATH% when it starts is the %PATH% that every process it launches inherits. (You can easily view a process’ environment block using the Properties > Environment tab in Process Explorer).

So, how does Explorer detect the change? That part I figured out ages ago– after updating an environment variable, the System Properties > Environment Variables Control Panel UI (or the SetX.exe console tool) broadcast a WM_SETTINGCHANGE message to all top-level windows with an lparam containing the string “Environment”. I could easily add code to SlickRun to detect that the variables had changed, but for decades I didn’t really know what to do next… I didn’t know how to read the updated variables (without doing something hacky like restarting the process) nor ensure that they were passed to the applications spawned by ShellExecute.

Yesterday, I got fed up and started Googling. A few posts on StackOverflow mentioned a promising-sounding function, RegenerateUserEnvironment. And while that function appears to be undocumented, there’s an amazing issue filed in an open-source tracker that explains exactly how Windows Explorer uses this function– basically, just wait for the WM_SETTINGCHANGE event, then call the API. The RegenerateUserEnvironment will replace the calling process’ current environment block with the latest values.

Launching at Medium Integrity

While we’re on the topic of executing applications “like the shell”, another scenario came up twelve years ago when Windows Vista was first introduced. The SlickRun installer, written in NSIS, launches SlickRun when installation completes. Unfortunately, the installer runs with Admin rights (High integrity), which means that, by default, all of the programs it launches inherit that integrity. For SlickRun, this is especially bad because it means that any programs that it, in turn, launches during that first session (e.g. your browser!) will run at High integrity too. Not good.

While you can easily use the “Runas” verb to ShellExecute to launch a High integrity application from a Medium integrity application, there (depressingly) isn’t a way to do the opposite. For years, the official recommendation was to do some fancy coding to clone Explorer’s tokens and use those. Unfortunately, this is quite complicated to implement, especially within a NSIS script.

As it turns out, however, there’s a trivial workaround which works quite well– while ShellExecute doesn’t run things as the shell, applications can easily get Explorer to launch anything they like at Explorer’s integrity. The trick is to simply invoke explorer.exe and pass the filename to be executed as the first command line argument:

While this approach isn’t technically supported, I expect it is likely to continue to work for the foreseeable future.

 

It’s depressing that together these tricks have taken me almost twenty years to discover, but I’m happy that I have. I hope they help you out.

-Eric

Standard

2 thoughts on “ShellExecute Doesn’t

  1. Here’s a tip for NSIS. Change the script to use set “RequestExecutionLevel” to “user”.
    Then change the install path to install to the user’s AppData\Local instead (Microsofts recommendation for per user actually).
    If a admin really need to run a installer as admin they can right click and “Run as administrator” anyway, and a admin is able to manually copy the files anyway.
    You can do fancy things and detect elevated mode and present different paths (i.e. program files folder instead of local) if detecting a elevated script. Let me know if you want to steal ideas from a NSIS script I use.

    As to programs and ENV try to use CreateProcess to launch processes.
    https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-createprocessa
    It lets you specify the environment block or use NULL (in witch case the parent process ENV is used).

    Use GetEnvironmentStrings (and FreeEnvironmentStrings when done with it obviously) to get the block.
    This should be a copy of the current env block for the process. (this also means your launch program should re-read as well if it put anything into variables).

    AFAIK this is the only “proper” way to do it (other than reading the env vars from the registry where user and system env stuff are in different places AFAIK)

    While I see no direct mention of it after a WM_SETTINGCHANGE the GetEnvironmentStrings call should return a updated block.
    A test should be easy enough, call the API, don’t free it, wait for the window message then call the API again. (BTW! it’s possible there may be fired multiple WM_SETTINGCHANGE messages).

    The wording in the description for GetEnvironmentStrings is kinda vague, it could mean the block is is the block given to the current process at launch, but it could also mean the block currently available to the current process (which I believe to be true, I have not tested this myself yet).

    ShellEx for urls should be fine as the browsers probably handles paths/env fetching itself, I’ve at least never seen the system default browser have issues with that.

    Like

  2. Yes, changing to a per-user install will mean that you get per-user behavior. I don’t want a per-user install.

    Yes, using CreateProcess gives you the ability to pass an environment block, but no, constructing that block is not trivial, and CreateProcess differs from ShellExecute in fundamental and important ways.

    No, GetEnvironmentStrings isn’t sensitive to changes in the system; it simply returns the current process’ (startup) environment block. Hence, this article.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s