Hatch rollback and remove refuse with "configuration drifted" after the project version is bumped or a dependency is added, in both hosted and vendored mode #385

[agent] Found by the scheduled Hatch bug-hunt routine (ledger #314).

Summary

For a Hatch project, socket-patch records whole pyproject.toml snapshots (hatch_document wiring and the HatchDocument hosted ledger edit) and reverts them with the three-way TOML merge vendor::pypi_lock::restore_document. That merge was written for lock [[package]] tables, and it treats two ordinary pyproject edits as drift:

1. [project].version (or name) changed. restore_item / restore_value call same_identity, which requires the live table's name and version to equal the recorded ones. Applied to the [project] table, that fails as soon as the project releases a new version. Even a trailing comment on name = "app" trips it, because the comparison is textual.
2. Any element added to or removed from the array that holds the patched pin (for example a new entry in project.dependencies). restore_value returns drift when live.len() != new.len().

After either edit, socket-patch rollback and socket-patch remove refuse, and the patch stays wired. A re-scan (scan --mode hosted or --mode vendored) exits 0 but doesn't re-baseline the ledger, so the next rollback fails the same way. Edits in other tables, such as a new env dependency or a new [tool.ruff] table, merge fine.

Impact

Bumping [project].version is part of every release, and adding a dependency is routine. After either, a Hatch project can't unpatch through socket-patch in either mode. remove exits with hosted_revert_failed / vendor_revert_failed, and the only way out is to edit pyproject.toml by hand. That includes removing allow-direct-references correctly, which socket-patch otherwise tracks ownership of. It fails closed (nothing is corrupted), but it blocks the documented rollback and remove workflow.

Repro (Linux, Hatch 1.18.1 and 1.7.0, main f6b7fb9)

Mock patch API serving a patched six 1.16.0 wheel: the same mock as the #335 probe (https://github.com/SocketDev/socket-patch/actions/runs/36740025279), modeled on tests/vex_pypi_real_common.

SP="socket-patch --api-url http://127.0.0.1:18080 --api-token fake --org test-org --patch-server-url http://127.0.0.1:18080"
mkdir -p app/src/app && cd app && touch src/app/__init__.py
cat > pyproject.toml <<'EOF'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "app"
version = "0.1.0"
dependencies = ["six==1.16.0"]

[tool.hatch.build.targets.wheel]
packages = ["src/app"]
EOF
hatch env create
VIRTUAL_ENV=$(hatch env find default) $SP scan --mode hosted --json --yes --ecosystems pypi   # redirected: 1
sed -i 's/0.1.0/0.2.0/' pyproject.toml                  # a release bump
$SP rollback --json      # exit 1, partial_failure: hosted.failed[0].error = "pyproject.toml: Hatch configuration drifted (pyproject.toml)"
$SP remove pkg:pypi/[email protected] --json --yes   # hosted_revert_failed, same message
$SP scan --mode hosted --json --yes --ecosystems pypi && $SP rollback --json   # still exit 1
grep -c 'six @' pyproject.toml                          # 1: still wired

With --mode vendored --vendor-source build (Hatch on PATH), the same steps give vendoredFailed: "pyproject.toml changed since patching", and remove gives vendor_revert_failed.

Control: without the edit, rollback restores pyproject.toml byte for byte in both modes (this passed on the previous run for 13 hosted and 12 vendored shapes).

Expected vs actual

• Expected: docs/testing/hatch.md says both modes "record reversible document edits", and that selective and preserved rollback restore allow-direct-references after the last reference is unwired. The only refusals it documents are drifted sources, ledgerless direct references, concurrent edits and symlinks. An unrelated key in [project], or a sibling entry in the dependency array, is none of those, so rollback should put six==1.16.0 back, keep the user's edits, and drop the permission.
• Actual: rollback and remove refuse with a drift error on every run, and the re-scan doesn't recover.

Matrix (Linux)

The failure comes from socket-patch's TOML merge, not from Hatch, so it's the same on every Hatch version and OS. No macOS or Windows probe was run for that reason.

First bad

Hatch support landed in #244 (649d457), which is after v4.0.0, so no release is affected yet. It has been present since Hatch support landed.

Suspect code

• crates/socket-patch-core/src/vendor/pypi_lock.rs:400 same_identity (called at :462 and :495) applies the lock-package identity check to pyproject tables.
• crates/socket-patch-core/src/vendor/pypi_lock.rs:470 / :505 treat any array-length change as drift.
• The callers are crates/socket-patch-core/src/patch/redirect/replay.rs:652 (hosted HatchDocument) and crates/socket-patch-core/src/vendor/pypi_hatch.rs:253 (vendored revert).

Related, but a different code path: #379 (uv, ReplaceFragment whole-document fragments) and #382 (PDM re-scan never normalizes the ledger).