Initially, when I tested a VB6 IDE install on an Windows 11 ‘out-of-the-box’ (RTM) install, I encountered no new issues. I just got a new laptop and tried the same install method on a 21H2 (Build 22000.376) and immediately ran into an issue. The install ended with a message it had not worked and a reference to an install setting. I had never seen this previously.
When Visual Studio 6, or VB6 as a part of it, installs it creates a directory, ~MSSETUP.T, at the root of the boot drive. The installer uses this directory during the install. I did not debug the exact reasons for the problem but it appears the installer has run afoul of the inherited security — probably the MAC (Mandatory Access Control) security from the drive root. I fixed the issue by blocking inheritance on the ~MSSETUP.T directory, converting none of it, and adding full control permissions to myself, the Administrator, and the Administrators group. I then restarted the install and it completed normally.
Not sure why this is new with 21H2 but I assume the security descriptor at the root of the boot drive has been changed for security reasons. I am also not sure this is isolated to Windows 11. It could also impact the latest versions of Windows 10. In any case, it looks like an additional step needs to be added for Windows 11. In addition to creating the MSJAVA.DLL file to bypass the java install before the starting the install:
- Create the ~MSSETUP.T directory manually,
- Block security inheritance converting none of the inherited security to explicit,
- Add the account you are installing with, the local Administrator, and the local Administrators Group, all with ‘full control’ permissions (This may be more than is needed – I just know it works)
Based on my experience, you can fix the security during the install (after an install failure) but it is probably best to just create the directory before starting. With this step added to the list in my Installing the VB6 IDE on Windows 10 post, the install runs normally and deletes the ~MSSETUP.T directory when it completes.
If I discover exactly this is happening, I will update this post but it doesn’t really matter, if this approach works.