Skip to content

ci: stabilize v5 compatibility tests and macOS patch host resolution - #295

Merged
Mikola Lysenko (mikolalysenko) merged 4 commits into
release/v5-prereleasefrom
v5/fix-macos-hosted-flake
Sep 30, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 4 commits into
release/v5-prereleasefrom
v5/fix-macos-hosted-flake

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

GitHub-hosted macOS runners intermittently fail to resolve patch.socket.dev for minutes, including midway through the Bun, vlt, and Poetry compatibility jobs. Package-manager tarball downloads fail even when the production service and the CLI's API calls are healthy.

The historical compatibility matrix also had deterministic fixture failures: Yarn before 1.10 does not write an integrity lock field, and early vlt releases record explicit npmjs URLs that v5's ledger-free upstream restore does not reproduce verbatim.

Changes

  • Add a macOS-only composite action that resolves the patch hosts through the system resolver or DNS-over-HTTPS, verifies each address with a TLS handshake for its hostname, and pins verified addresses in /etc/hosts. The package managers continue exercising the production service over verified TLS.
  • Mirror older Yarn lock hashes into registry metadata as SHA-1 SRI. Assert the reconstructed upstream lock, including the integrity line introduced by hosted mode, and prove a fresh frozen install restores pristine package contents.
  • Allow only the known target-URL differences in vlt 0.0.0-1 and 0.0.0-11 rollback assertions. Preserve checks for integrity, bystanders, line endings, and fresh installs of pristine packages; keep exact lock comparisons for later releases.
  • Preserve failed CLI JSON output in vlt result rows so the existing bounded transport retry catches API 5xx errors during initial, repeat, and rollback calls. Functional failures remain failures.
  • Print the failing Yarn suite's captured output directly in CI logs.
  • Update the branch with release/v5-prerelease and reconcile the compatibility documentation.

Validation

  • Real Yarn migration tests pass on 1.0.2, 1.6.0, 1.7.0, 1.9.4, 1.10.1, and 1.22.22.
  • Real vlt hosted capstones pass on 0.0.0-1, 0.0.0-11, and 1.2.0: 38 tests per release, with the expected per-era skips verified by check-vlt-legs.py.
  • Python script suite: 125 tests, one expected skip.
  • Clippy with warnings denied for the affected test targets; rustfmt for the changed Rust files; shellcheck; actionlint; diff whitespace checks.

The Bun, vlt and Poetry compatibility workflows drive real package managers
against the production patch service. On GitHub's hosted macOS runners the
system resolver intermittently answers patch.socket.dev with EAI_NONAME
("[Errno 8] nodename nor servname provided"; bun: FailedToOpenSocket; Bun
1.3.x workspace installs never exit) for minutes at a time, at job start or
mid-job, while the service is up: ubuntu and windows legs of the same run
pass, and the same macOS cells pass before and after the window. Over the
last 60 Bun runs (69 attempts) 29 macOS native jobs failed this way and no
other OS did; the CLI's own API calls in those cells succeeded.

A pre-flight wait cannot cover a mid-job window, and the failing processes
are bun / vlt / poetry / python rather than the CLI, so a product retry
cannot help. The runner's resolver is not under test, so take it out of the
path: .github/actions/pin-socket-hosts runs scripts/pin-socket-hosts.py on
macOS, which resolves patch.socket.dev and patches-api.socket.dev (system
resolver, then DNS-over-HTTPS by IP literal, with bounded backoff), keeps
only addresses whose TLS handshake verifies the hostname, and pins them in
/etc/hosts. Every cell still hits production over TLS verified for the
hostname, so the captures depscan imports stay production captures.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#279 landed on release/v5-prerelease as f6bdad5; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#282 landed on release/v5-prerelease as 14a9cb0; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#291 landed on release/v5-prerelease as f9cb7e1; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#292 landed on release/v5-prerelease as a7b0d00; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#296 landed on release/v5-prerelease as 1e3ace6; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#293 landed on release/v5-prerelease as b9e106d; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

#294 landed on release/v5-prerelease as 180f10f; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.


Generated by Claude Code

@mikolalysenko Mikola Lysenko (mikolalysenko) changed the title ci: pin TLS-verified patch hosts on macOS compat legs (fix hosted DNS flake) ci: stabilize v5 compatibility tests and macOS patch host resolution Sep 30, 2026
@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 909c08c into release/v5-prerelease Sep 30, 2026
365 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the v5/fix-macos-hosted-flake branch September 30, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant