Secrets Engine startup fix in 4.89.0 does not work: rename of engine.sock fails the same way remove did

File at: https://github.com/docker/for-win/issues

Summary

Docker Desktop 4.89.0 shipped "Fixed a bug where Docker Desktop could fail to
start after an ungraceful shutdown left behind a stuck socket file." On this
machine that fix does not work. The startup failure is identical; only the
failing syscall changed, from remove to rename.

The cause is that the fix operates on the socket file, which requires
opening it. On this machine the orphaned AF_UNIX reparse point cannot be opened
by anything. Renaming the parent directory succeeds, because a rename never
opens the child. That is a one-line change in strategy and it fixes the failure
completely.

Error text, before and after the fix

4.88.1.237512:

starting services: initializing Secrets Engine: listening on
unix://C:/Users/<user>/AppData/Local/docker-secrets-engine/engine.sock:
remove C:/Users/<user>/AppData/Local/docker-secrets-engine/engine.sock:
The file cannot be accessed by the system.
(listener: The file cannot be accessed by the system.)

4.90.0.238679:

starting services: initializing Secrets Engine: listening on
unix://C:/Users/<user>/AppData/Local/docker-secrets-engine/engine.sock:
rename C:/Users/<user>/AppData/Local/docker-secrets-engine/engine.sock
C:/Users/<user>/AppData/Local/docker-secrets-engine/engine.sock.stale:
The file cannot be accessed by the system.
(listener: The file cannot be accessed by the system.)

Underlying Windows error in both cases: ERROR_CANT_ACCESS_FILE (1920, 0x780).

Reproduction

On 4.90.0.238679:

1. Start Docker Desktop, let the engine come up.
2. Kill the backend ungracefully:
   Get-Process 'Docker Desktop','com.docker.backend' | Stop-Process -Force
3. Confirm %LOCALAPPDATA%\docker-secrets-engine\engine.sock still exists.
4. Launch Docker Desktop.exe.
5. Startup fails with the 4.90.0 error above.

Reproduced reliably. This has occurred roughly ten times over two weeks on this
machine, on both 4.88.1 and 4.90.0.

The orphaned socket cannot be opened by any userspace call

With Docker fully stopped, every one of these fails with ERROR_CANT_ACCESS_FILE
on engine.sock:

• [System.IO.File]::Delete()
• Remove-Item
• cmd /c del /f /q
• robocopy <empty> <dir> /MIR
• fsutil reparsepoint delete (elevated)
• fsutil reparsepoint query (elevated)

The file is 0 bytes with attributes Archive, ReparsePoint.

The working fix: rename the directory, not the file

Renaming the parent directory succeeds every time, because rename does not open
the child:

Rename-Item -LiteralPath "$env:LOCALAPPDATA\docker-secrets-engine" `
            -NewName "docker-secrets-engine.broken-$(Get-Date -f yyyyMMddHHmmss)"

Docker Desktop then recreates a clean docker-secrets-engine directory and
starts normally, in 15 to 30 seconds. This has worked on every occurrence, on
both versions.

Suggested change

When the Secrets Engine cannot remove or rename a stale engine.sock, fall back
to moving the containing directory aside and recreating it, rather than failing
startup. Binding into a freshly created uniquely-named directory would also
sidestep the problem entirely.

The orphaned directories are inert and roughly 0 bytes, and they cannot be
deleted afterward, so a periodic best-effort sweep would be worthwhile too.

Environment

• Docker Desktop 4.90.0.238679 (previously 4.88.1.237512, same failure)
• Windows 10 Home 19045
• WSL2 backend, distros Ubuntu and docker-desktop, both version 2
• Engine 29.7.2

Ruled out by testing

These were each tested and eliminated, so they do not need revisiting:

• Disk space. Free space raised from 4.85 GB to 36.99 GB. No effect on the
  delete or the failure.
• Reboot and uptime. The orphaned sockets survive a reboot and remain
  undeletable. The fault recurred the day after a reboot.
• Antivirus. Norton is present but loads no filesystem minifilter;
  fltmc filters shows only Microsoft filters.
• Privilege level. Elevated attempts fail identically.
• Docker version. 4.88.1 to 4.90.0 changed the failing call, not the outcome.

Possibly relevant, unproven

fltmc instances shows bindflt attached to C: at altitude 409800, the highest
of any filter on the volume. wcifs is loaded but has no instances.

Separately, and possibly the same underlying cause: a Task Scheduler process on
this machine cannot see the docker-secrets-engine* directories at all. It
enumerates 173 of the 175 entries in %LOCALAPPDATA%, and the two it misses are
exactly those, including a plain empty directory created seconds earlier by
New-Item. An interactive shell, elevated or not, sees all 175. I could not
take this further because fltmc cannot enumerate bindflt mappings.

Note on the 4.90.0 installer

The 4.88.1 to 4.90.0 upgrade rewrote both the HKCU\...\Run entry and the
Start Menu shortcut for Docker Desktop, discarding local modifications to them
without warning. Worth flagging separately if that is not intended.