Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/actions/pin-socket-hosts/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Pin Socket patch hosts
description: >-
On macOS runners, resolve the production patch hosts once (system resolver,
then DNS-over-HTTPS by IP literal), TLS-verify every address for its host,
and pin them in /etc/hosts for the rest of the job
inputs:
hosts:
description: Space-separated hostnames to pin
default: patch.socket.dev patches-api.socket.dev
runs:
using: composite
steps:
# GitHub's hosted macOS runners intermittently answer patch.socket.dev
# with EAI_NONAME ("[Errno 8] nodename nor servname provided", bun's
# `FailedToOpenSocket`) for minutes at a time — at job start or mid-job —
# while the service is up: the ubuntu / windows legs of the same run pass
# and the same macOS cells pass before and after the window. A pre-flight
# wait cannot cover a mid-job window and the failing processes are the
# real package managers, not the CLI, so the job takes the runner's
# resolver out of the path instead. Every request still goes to the
# production service over TLS verified for the hostname.
# scripts/pin-socket-hosts.py documents the resolution and verification.
- name: Pin hosts
if: runner.os == 'macOS'
shell: bash
env:
PIN_HOSTS: ${{ inputs.hosts }}
run: |
set -euo pipefail
# shellcheck disable=SC2086 # PIN_HOSTS is a space-separated list
lines=$(python3 "$GITHUB_WORKSPACE/scripts/pin-socket-hosts.py" $PIN_HOSTS)
printf '%s\n' "$lines"
printf '\n# pinned by .github/actions/pin-socket-hosts\n%s\n' "$lines" | sudo tee -a /etc/hosts >/dev/null
sudo dscacheutil -flushcache
sudo killall -HUP mDNSResponder || true
for host in $PIN_HOSTS; do
got=$(python3 -c 'import socket, sys; print(" ".join(sorted({i[4][0] for i in socket.getaddrinfo(sys.argv[1], 443)})))' "$host" || true)
echo "$host now resolves to: ${got:-nothing}"
if [ -z "$got" ] || ! grep -qE "^(${got// /|}) $host\$" <<<"$lines"; then
echo "::warning::$host does not resolve to its pinned address after pinning (got: ${got:-nothing})"
fi
done
9 changes: 9 additions & 0 deletions .github/workflows/bun-compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ on:
pull_request:
paths:
- '.github/actions/upload-artifact/**'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- '.github/workflows/bun-compatibility.yml'
- 'scripts/backtest-bun*.py'
- 'scripts/probe-bun-historical-linux.py'
Expand Down Expand Up @@ -57,6 +59,8 @@ on:
branches: [main]
paths:
- '.github/workflows/bun-compatibility.yml'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- 'scripts/backtest-bun*.py'
- 'scripts/probe-bun-historical-linux.py'
- 'scripts/bun-historical-shas.json'
Expand Down Expand Up @@ -187,6 +191,11 @@ jobs:
with:
python-version: '3.12'

- name: Pin the production patch hosts (macOS)
# The hosted macOS resolver intermittently loses patch.socket.dev for
# minutes (EAI_NONAME) while the service is up; see the action.
uses: ./.github/actions/pin-socket-hosts

- name: Download Bun ${{ matrix.bun }}
id: bun
# Pre-populate the exact directory layout the script's install_tool()
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/poetry-compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ on:
pull_request:
paths:
- '.github/actions/upload-artifact/**'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- '.github/workflows/poetry-compatibility.yml'
- 'scripts/backtest-poetry.py'
- 'crates/socket-patch-core/src/utils/poetry_lock.rs'
Expand All @@ -29,6 +31,8 @@ on:
branches: [main]
paths:
- 'scripts/backtest-poetry.py'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- 'crates/socket-patch-core/src/utils/poetry_lock.rs'
- 'crates/socket-patch-core/src/patch/redirect/**'
- 'crates/socket-patch-core/src/vendor/pypi*.rs'
Expand Down Expand Up @@ -91,6 +95,10 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: '3.12'
- name: Pin the production patch hosts (macOS)
# The hosted macOS resolver intermittently loses patch.socket.dev for
# minutes (EAI_NONAME) while the service is up; see the action.
uses: ./.github/actions/pin-socket-hosts
# uv bootstraps every pinned Poetry release (and its interpreter)
# itself; pinning uv keeps the bootstrap reproducible.
- run: python -m pip install uv==0.11.19
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/vlt-compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ on:
pull_request:
paths:
- '.github/actions/upload-artifact/**'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- '.github/workflows/vlt-compatibility.yml'
- 'Cargo.lock'
- 'rust-toolchain.toml'
Expand Down Expand Up @@ -60,6 +62,8 @@ on:
branches: [main]
paths:
- '.github/actions/upload-artifact/**'
- '.github/actions/pin-socket-hosts/**'
- 'scripts/pin-socket-hosts.py'
- '.github/workflows/vlt-compatibility.yml'
- 'Cargo.lock'
- 'rust-toolchain.toml'
Expand Down Expand Up @@ -407,6 +411,10 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: '3.12'
- name: Pin the production patch hosts (macOS)
# The hosted macOS resolver intermittently loses patch.socket.dev for
# minutes (EAI_NONAME) while the service is up; see the action.
uses: ./.github/actions/pin-socket-hosts
- name: Backtest against production
# Every hosted cell probes the artifact first; it records
# blocked-by-server-encoding only when that probe saw a non-identity
Expand Down
44 changes: 28 additions & 16 deletions crates/socket-patch-cli/tests/e2e_redirect_vlt_build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,24 @@ fn hosted_leg(name: &'static str) -> Option<Leg> {
Leg::start(SUITE, name)
}

/// v5 restores upstream pins without a saved lock fragment. The earliest
/// vlt releases record npmjs URLs even with the harness registry configured;
/// restore may omit that redundant slot or point it at the harness registry.
/// All other bytes, including bystanders and line endings, must still match.
fn assert_restored_lock(fx: &Fixture, before: &[u8]) {
let mut expected = String::from_utf8(before.to_vec()).unwrap();
let mut actual = String::from_utf8(lock_bytes(&fx.proj)).unwrap();
if fx.leg.version() <= VltVersion::zero(11) {
for target in &fx.svc.targets {
let bare = target.name.rsplit('/').next().unwrap();
let path = Registry::tarball_path(&target.name, bare, &target.version);
expected = expected.replace(&format!(",\"https://registry.npmjs.org{path}\""), "");
actual = actual.replace(&format!(",\"{}{path}\"", fx.reg.server.uri()), "");
}
}
assert_eq!(actual, expected, "rollback restores the upstream lock");
}

// ── drivers and fresh checkouts ───────────────────────────────────────────

/// `scan --mode hosted --vex`: the lock pins the artifact (one preflight
Expand Down Expand Up @@ -167,7 +185,7 @@ async fn vlt_pinned_matrix_hosted_tamper_cold_eintegrity() {

// ── rollback, rerun, heal ─────────────────────────────────────────────────

/// Rollback restores the lock byte-for-byte, heals the patched store copy
/// Rollback restores the upstream lock, heals the patched store copy
/// (and nothing else), and the next `vlt install` is pristine.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "real vlt: SOCKET_PATCH_VLT_E2E_JS"]
Expand All @@ -184,11 +202,7 @@ async fn vlt_pinned_matrix_hosted_rollback_byte_exact() {
let out = fx.rollback(&[]);
assert_eq!(out.code, 0, "{out}");
let doc = out.json();
assert_eq!(
String::from_utf8_lossy(&lock_bytes(&fx.proj)),
String::from_utf8_lossy(&fx.lock_before),
"rollback restores vlt-lock.json byte-for-byte"
);
assert_restored_lock(&fx, &fx.lock_before);
assert!(
fx.ledger().is_none(),
"no hosted ledger is ever written (v5)"
Expand Down Expand Up @@ -762,11 +776,10 @@ async fn vlt_pinned_matrix_hosted_crlf_lock() {
assert_eq!(state(&co, fx.t()), State::Patched);
let out = fx.rollback(&[]);
assert_eq!(out.code, 0, "{out}");
assert_eq!(
lock_bytes(&fx.proj),
crlf,
"rollback restores the CRLF lock"
);
assert_restored_lock(&fx, &crlf);
let co = fx.checkout("restored-crlf");
fx.vlt_ok_profile(&co, &fx.leg.locked_install_args(), "restored-crlf");
assert_eq!(state(&co, fx.t()), State::Pristine);
fx.leg.ran();
}

Expand Down Expand Up @@ -1185,11 +1198,7 @@ async fn vlt_pinned_matrix_hosted_idempotence() {
assert!(fx.ledger().is_none());
let out = fx.rollback(&[]);
assert_eq!(out.code, 0, "{out}");
assert_eq!(
String::from_utf8_lossy(&lock_bytes(&fx.proj)),
String::from_utf8_lossy(&fx.lock_before),
"rollback restores the registry lock byte-for-byte: {out}"
);
assert_restored_lock(&fx, &fx.lock_before);
assert!(fx.ledger().is_none());
let files = package_files(&fx.proj);
let out = fx.rollback(&[]);
Expand All @@ -1199,6 +1208,9 @@ async fn vlt_pinned_matrix_hosted_idempotence() {
"a second rollback finds no state: {out}"
);
assert_eq!(package_files(&fx.proj), files, "and writes nothing");
let co = fx.checkout("restored-idempotence");
fx.vlt_ok_profile(&co, &fx.leg.locked_install_args(), "restored-idempotence");
assert_eq!(state(&co, fx.t()), State::Pristine);
fx.leg.ran();
}

Expand Down
81 changes: 62 additions & 19 deletions crates/socket-patch-cli/tests/mode_migration_npm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -388,11 +388,15 @@ async fn mount_hosted_mocks(
/// Serve, from `server` (as `SOCKET_NPM_REGISTRY`), the npm registry version
/// document the v5 upstream restore reads for DEP — mirrored from what the
/// PRISTINE classic lock recorded (`resolved "<tarball>#<sha1>"`,
/// `integrity`). The restore of a hosted classic entry must reproduce the
/// registry entry yarn wrote from exactly that document; mirroring it keeps
/// `integrity`, or the SHA-1 fragment on pre-1.10 releases). The restore must
/// reproduce the registry entry yarn wrote from that document; mirroring it keeps
/// the unwind hermetic (the binary's TLS stack need not reach the real
/// registry). Returns the registry base to hand the binary.
async fn mount_registry_from_classic_lock(server: &MockServer, lock: &str) -> String {
/// registry). Returns the registry base and expected upstream lock. Hosted
/// mode adds an integrity line even on pre-1.10 yarn, and v5 restores that
/// line's registry hash without a saved fragment to recover its absence.
async fn mount_registry_from_classic_lock(server: &MockServer, lock: &str) -> (String, String) {
use base64::Engine as _;

let block = lock
.split("\n\n")
.find(|b| {
Expand All @@ -404,23 +408,43 @@ async fn mount_registry_from_classic_lock(server: &MockServer, lock: &str) -> St
.lines()
.find_map(|l| l.trim().strip_prefix(&format!("{name} ")))
.map(|v| v.trim_matches('"').to_string())
.unwrap_or_else(|| panic!("no `{name}` in {block}"))
};
let resolved = field("resolved");
let resolved = field("resolved").unwrap_or_else(|| panic!("no `resolved` in {block}"));
let (tarball, shasum) = resolved
.split_once('#')
.map(|(t, s)| (t.to_string(), Some(s.to_string())))
.unwrap_or((resolved.clone(), None));
let integrity = field("integrity").unwrap_or_else(|| {
let sha1 = hex::decode(
shasum
.as_ref()
.expect("pre-1.10 yarn pins a SHA-1 fragment"),
)
.expect("the resolved fragment is hex SHA-1");
format!(
"sha1-{}",
base64::engine::general_purpose::STANDARD.encode(sha1)
)
});
let upstream_lock = if field("integrity").is_some() {
lock.to_string()
} else {
lock.replacen(
&format!(" resolved \"{resolved}\""),
&format!(" resolved \"{resolved}\"\n integrity {integrity}"),
1,
)
};
Mock::given(method("GET"))
.and(path(format!("/registry/{DEP}/{DEP_VERSION}")))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"name": DEP,
"version": DEP_VERSION,
"dist": { "tarball": tarball, "integrity": field("integrity"), "shasum": shasum }
"dist": { "tarball": tarball, "integrity": integrity, "shasum": shasum }
})))
.mount(server)
.await;
format!("{}/registry", server.uri())
(format!("{}/registry", server.uri()), upstream_lock)
}

fn run_hosted_scan(proj: &Path, server_uri: &str) -> (i32, String, String) {
Expand Down Expand Up @@ -642,7 +666,7 @@ fn assert_pure_vendored_and_round_trip(
"fresh vendored install must carry the PATCHED bytes ({tag})"
);

// (b) Round trip: `vendor --revert` restores the REGISTRY lock
// (b) Round trip: `vendor --revert` restores the expected REGISTRY lock
// byte-identically (pre-fix it restored the hosted fragment, with no CLI
// path back to registry state).
let (code, stdout, stderr) = run_socket(
Expand All @@ -659,8 +683,8 @@ fn assert_pure_vendored_and_round_trip(
assert_eq!(
std::fs::read(proj.join("yarn.lock")).unwrap(),
lock_pristine,
"yarn.lock must be restored byte-identical to the pre-hosted \
REGISTRY pristine ({tag}); got:\n{}",
"yarn.lock must be restored byte-identical to the expected \
upstream REGISTRY lock ({tag}); got:\n{}",
read(proj, "yarn.lock")
);
assert_eq!(
Expand Down Expand Up @@ -809,7 +833,7 @@ async fn classic_hosted_then_vendored_takeover_round_trips_to_registry() {
// upstream restore re-resolves the registry entry (mirrored from the
// pristine lock), and the mock origin is named hosted via
// --patch-server-url.
let registry =
let (registry, lock_upstream) =
mount_registry_from_classic_lock(&server, &String::from_utf8_lossy(&lock_pristine)).await;
stage_patch(&proj, &fx.orig, &fx.patched);
let (code, stdout, stderr) = run_socket_env(
Expand Down Expand Up @@ -847,7 +871,7 @@ async fn classic_hosted_then_vendored_takeover_round_trips_to_registry() {
"classic",
false,
&hosted_url,
&lock_pristine,
lock_upstream.as_bytes(),
&pkg_json_pristine,
&stdout,
);
Expand Down Expand Up @@ -1055,10 +1079,10 @@ async fn classic_vendored_then_hosted_takeover_leaves_pure_hosted() {
)
});

// The originals chain across migrations: `rollback` restores the hosted
// pin's upstream registry entry, which is the pristine lock byte for
// byte (online: the entry is re-resolved from the registry document).
let registry =
// Rollback re-resolves the upstream registry entry. Hosted mode added an
// integrity line even on pre-1.10 yarn; v5 has no saved fragment to tell
// whether it was originally absent, so it restores the registry hash.
let (registry, lock_upstream) =
mount_registry_from_classic_lock(&server, &String::from_utf8_lossy(&lock_pristine)).await;
let (code, stdout, stderr) = run_socket_env(
&proj,
Expand All @@ -1076,8 +1100,27 @@ async fn classic_vendored_then_hosted_takeover_leaves_pure_hosted() {
assert_eq!(code, 0, "rollback failed: {stdout}\n{stderr}");
assert_eq!(
read(&proj, "yarn.lock"),
String::from_utf8_lossy(&lock_pristine),
"rollback lands on the pristine registry lock"
lock_upstream,
"rollback restores the registry lock, allowing the added upstream integrity"
);
let fresh = fresh_checkout(&proj, fx.tmp.path(), "classic-rollback", false);
let fresh_cache = fx.tmp.path().join("fresh-cache-classic-rollback");
let ci = corepack(
&fresh,
&yarn_classic_vex::yarn_classic(),
&["install", "--frozen-lockfile", "--no-progress"],
&[("YARN_CACHE_FOLDER", fresh_cache.to_str().unwrap())],
);
assert!(
ci.status.success(),
"fresh-checkout rollback install must succeed.\nstdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&ci.stdout),
String::from_utf8_lossy(&ci.stderr),
);
assert_eq!(
std::fs::read(fresh.join("node_modules").join(DEP).join("index.js")).unwrap(),
fx.orig,
"rollback installs the pristine registry bytes"
);
}

Expand Down
12 changes: 12 additions & 0 deletions docs/testing/bun-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,18 @@ matrix to six concurrent jobs, with three cells per job. Each cell has its own
temporary directory so historical Bun processes cannot collide while extracting
identically named packages.

On macOS the job first runs `.github/actions/pin-socket-hosts`
(`scripts/pin-socket-hosts.py`): the hosted macOS resolver intermittently
answers `patch.socket.dev` with EAI_NONAME for minutes at a time, at job start
or mid-job, while the service is up, which failed every hosted cell in the
window (`FailedToOpenSocket`, `[Errno 8] nodename nor servname provided`, or a
Bun 1.3.x workspace install that never exits). The action resolves the patch
hosts once (the system resolver, then DNS-over-HTTPS by IP literal), keeps only
addresses whose TLS handshake verifies the hostname, and pins them in
`/etc/hosts`, so cells still reach the production service over verified TLS
without depending on the runner's resolver. The vlt and Poetry workflows run
the same step.

**Pinned versions:** 0.8.1, 1.0.0, 1.0.36, 1.1.0, 1.1.38 (binary lock),
1.1.39 (first text lock, version 0), 1.1.43 (first `--lockfile-only`), 1.1.45
(last version-0 writer), 1.2.0, 1.2.23, 1.3.0 (version 1), 1.3.9 / 1.3.10
Expand Down
Loading
Loading