Application restart #1

I'm a pretty carefree guy, experimenting with all kinds of weird applications from dubious sources. The domain and the network is pretty well secured, but my virtual lab computers are like people in the old west; unregulated and easily killed off if they become to unwieldy.

Still, when something suspicious happens I wan't to know why, because of course I wan't.

Some time ago the following dialog suddenly popped up when I logged in to my work computer;

What? No, I would not like to allow Regedit.exe to make changes to my device. Searching through the startup locations with Sysinternals excellent Autoruns didn't reveal anything and concurrent logons did not start Regedit.exe. Probably, it was launched by the RunOnce registry key. Difficult to troubleshoot when the traces are gone.

Weeks later it appeared again! What program is causing this? Why does it want to start Regedit.exe? How do I find out when it only happens once every month or so? None of my coworkers have had the same experience and even though I have a black belt in Google-fu searching the web was hard, only resulting in how to use Regedit. I know this already!

Then it appeared on my clean virtual lab computer, not in any way related to my domain joined work computer. Luckily I had just done a snapshot of it and Regedit.exe wanted to start every time I restored. Finally some progress! Using my incredible skills I finally found the culprit using Procmon, the greatest tool of all time;

Indeed it was a RunOnce registry key and the name of it, "Application Restart #1" is the clue. Since some version ago (who knows which, versioning sucks) Windows restarts select applications that were running when the computer was shut down. Not only during forced restarts from updates, but when you manually shut down the computer. Apparently I am not very good at closing programs before shutting the PC down.

Not all programs are restarted and how Windows decides this is still a mystery to me. It's a badly documented feature with a badly thought through workaround. This is what Microsoft Answers says about it.

Regardless, Regedit.exe, or any program requiring elevation, should not be started automatically!

Or is it requireing? Require-ing? Reqiruing? My Google-fu might not be as advanced as I thought.

Installation failed because reasons

Sometimes installations fail. Often, the reason for failure is logical and easily found in a log file, sometimes you just get an 1603 with no indication whatsoever what went wrong.

And sometimes the install fails by design. A retarded design.

Solid Works 2016 installs Microsoft SQL Server 2008, Microsoft SQL Server 2012 and Microsoft SQL Server 2014.

I think most people see a probable cause for disaster right there. But surprisingly it usually works.

Unless your organisation has done the rare but sensible and rational thing and set a password policy:

As it happens, Microsoft SQL Server adheres to the group policy dictating the password policy for the domain.

There are workarounds found when googleing, but they either require disabling the password policy or modifying the way the database is created. Neither of these methods might be possible for an unattended install from a third party vendor.

Mostly because I don't feel like talking to the vendor, nor to my collegues managing the domain.

So here is how you install an SQL instance without supervision, without modifying the install and without modifying the domain:


; Removing password-requirements with secedit:

RunWait('secedit.exe /export /cfg "' & @TempDir & '\sectmporg.cfg" /quiet',"",@SW_HIDE)

FileCopy(@TempDir & '\sectmporg.cfg', @TempDir & '\sectmpcpy.cfg',1)
IniWrite(@TempDir & '\sectmpcpy.cfg',"System Access","MinimumPasswordLength","0")
IniWrite(@TempDir & '\sectmpcpy.cfg',"System Access","PasswordComplexity","0")

RunWait('secedit.exe /configure /db secedit.sdb /cfg "' & @TempDir & '\sectmpcpy.cfg" /quiet',"",@SW_HIDE)
RunWait("gpupdate.exe","",@SW_HIDE)

; **********************
;
; THE INSTALL GOES HERE!
;
; **********************

; Restoring password-requirements:

RunWait('secedit.exe /configure /db secedit.sdb /cfg "' & @TempDir & '\sectmporg.cfg" /quiet',"",@SW_HIDE)
RunWait("gpupdate.exe","",@SW_HIDE)

Code is AutoIt, but I'm sure you figure it out in any language.

Windows on Windows action

If you are running a 64 bit Windows, and you are aren't you, you are actually running two operating systems, one 64 bit, and one 32 bit. While this is an extreme simplification, it is for all intents and purposes true enough, while at the same time maybe not true at all. But it is.

Anyway, an Intel processor runs 16, 32 or 64 bit code in different modes, thus a process can only run in either mode, not several. If I designed a CPU I would make sure future modes could run current code, but I don't design CPUs, probably because I don't have a clue. To Intels defense, the x86 range of CPUs are backwards compatible back into a different century. Hilariously it was AMD, not Intel, who designed the 64 bit mode and thus it is sometimes called AMD64.

The consequence of these modes is that 32 bit binaries and 64 bit binaries are pretty incompatible. 32 bit processes cannot load 64 bit binaries and vise versa. To run both kind of programs, Windows therefor has an almost full set of both 32 and 64 bit components.

The system files of Windows have since Windows 95 been stored in the "System32" folder. In 32-bit Windows, which can natively run 16 bit code, there is also a "System" folder, pretty much containing a Windows 3.11 set of 16 bit files. But 64 bit Windows dropped the 16 bit support, so the "System" folder is no longer present. Well, it's there, but pretty much empty.

For stupid reasons Microsoft decided to keep the name of the system folder as "System32" even on a 64 bit Windows.

The System32 folder contains the 64 bit system files.

I can see why, but I believe it's stupid because we are going to run 64 bit for much longer than the 20 years we used 32 bit operating systems. Then again, the future plans for Windows executables are uncertain.

But as you've figured out, 32 bit programs can't use these 64 bit system files, how does that work? Misdirection! Or rather, redirection. Or virtualization, if you prefer.

A 32 bit program trying to access System32 will be redirected to the "SysWOW64" folder. The program believes it loads from System32, but it isn't. It's running a 32 bit System Windows On Windows 64. SysWOW64.

The SysWOW64 contains the 32 bit system files.

There are ways to get around this if your program needs to for some reason, but this is the default behaviour. You can access the true System32 folder from a 32 bit process by the alias C:\Windows\Sysnative.

The registry is divided in much the same way. 32 bit programs accessing HKLM\Software are redirected to HKLM\Software\Wow6432Node. They couldn't choose a consistent name such as RegWOW64, because why would they. Also note that HKEY_CLASSES_ROOT is actually a mirror of HKLM\Software\Classes and HKCU\Software\Classes, therefor 32 bit and 64 bit class registrations can exist at the same time, the 32 bit under HKLM\Software\Wow6432Node\Classes.

The Program Files folders are not redirected, you can install a 64 bit program in Program Files (x86) and a 32 bit program in Program Files without much trouble. However, the environment variables are different for each architecture, the %ProgramFiles% points to a different folder depending on the process.

You can try all this by launching one 64 bit CMD from C:\Windows\System32 and one 32 bit CMD from C:\Windows\SysWOW64.

I made a tool, CheckArch, that shows the architecture of a file, sometimes needed for troubleshooting. How you managed so far without it, I don't know.

Download CheckArch.exe

Self Repair

There are things that probably had good intentions early in its developement but eventually didn't turn out as great as one had hoped. Communism. The DeLorean DMC-12. MSI Self Repair.

An msi normaly register itself into the system during the end of the installation, running actions such as "PublishFeatures". With an msi you don't actually have to install the program, you only need to "publish" it. Some files and registry entries in the msi is set as keys, and if one of those is missing when the program needs it, Windows Installer will perform a Self Repair.

I don't know anyone who has ever published a program instead of installing it. But then again, I don't know that many people.

However, Self Repair has effect even when a program is fully installed, because the keys always have to be present. An msi usually applies source resiliency. In the rare case of an install that becomes corrupt it will try to reinstall the broken feature, but the more common scenario is that a user is launching the program for the first time and files and registry keys that are supposed to be in the user space are missing. When this works, it is ok. Not great, but ok.

This brings us to when it doesn't work, which is probably where a vast majority of people encounter Self Repair:

This was a very common problem during the days of removable media, but thankfully most computers don't even have a CD reader anymore. It's still a problem though, because users don't always have access to the company network, or the cache that used to contain the install has been purged. The latter is typical of an environment running the later versions of Microsoft System Center.

Some deployers seem to think Self Repair is the greatest thing since sliced bread, but it isn't. A proper program does NOT rely on Windows Installer for user files and a proper configured computer does NOT need to constantly repair broken installs.

Worse, many badly authored msi triggers unnecessary and sometimes repeating Self Repairs not only for itself but for features belonging to other, completely unrelated, programs. And for the love of God, don't set key paths to any startup locations! Do you hear me, Citrix?

There are a couple of work-arounds if you want to avoid actually solving the problem. The first and easiest is to create a new shortcut to the offending program and see if it launches correctly. Normally, the repair is triggered when the program is called by its advertised shortcut, a special lnk that goes to a launcher i C:\Windows\Installer. Repair is triggered by more events than launching the advertised shortcut though, so this might not always work.

The cause of the Self Repair should be noted in the Event Viewer in the Application Event log with an ID of 1004, and a quick fix could be to create a dummy file or registry key in place of the one missing.

The most brutal one is to use MsiZap to simply clean all the installer metadata, leaving only the program itself, completely unaware about any published features or components. This is a last resort solution that might cause new problems that are hard to troubleshoot further down the road, try to avoid it!

How to deploy an application that needs data in the user space will be covered, I promise.

Procmon, the greatest tool of all time

If you don't use Procmon, you're not an IT professional. If you're not an expert at quickly sifting through the Procmon output you are definitely not a deployer!

Procmon is the greatest trouble shooting tool for PC ever made and you should be using it.

My work day is 25% Procmon and 25% other work.

The system corrupts

An update is due again, as it always is. This time it is the digital signage program Dise. Previously deployed version is 1.6 and when I try on my lab computer both uninstalling this version and installing the new one works beautiful with just a /S switch. A good five minutes of work that I can charge $500 for. I can lean back, take a sip of coffee and browse the internet for "news".

I deploy the application but curiously I don't get any statistics in System Center, nothing seems to happen. Suddenly I hear screaming. The staff at the call center seem distressed. I find that after a restart the computers don't boot anymore, all thats left of our entire environment is this:

It turns out that uninstalling Dise 1.6 with the SYSTEM accound wipes the boot sector for no reason. The computer is now a brick.

Fortunately the above scenario didn't play out, because I actually spent more than five minutes and caught this before deploying.

Many installs behave differently when installing as SYSTEM as opposed to an ordinary administrator account. Every install MUST be tested with this account before being deployed sitewide. I recommend at least having a CMD shortcut on the desktop that uses psexec.exe to launch as SYSTEM.

Some old issetup.dll, a component in Installshield built setups, had a bug that launched the install as the currently logged on user, which of course failed with tons of access denied. Some installs deploy settings only to the running users profile, which in this case is C:\Windows\System32\config\systemprofile or might fail to write user settings altogether. Some installs get confused for other reasons when the environment is different from a standard user.

Few installs are as awful as Dise, but there are more ways to completely destroy a computer. More about that in a later post.