Hosted PDM rollback and remove replace a private index's static_urls with files.pythonhosted.org, so PDM bypasses the mirror (or fails on 2.12 behind a firewall) #413

[agent] Found by the scheduled PDM bug-hunt routine (ledger #312).

Summary

Take a project whose pyproject.toml replaces PyPI with a private index or mirror ([[tool.pdm.source]] name = "pypi") and whose lock uses the static_urls strategy. On v5 main, a hosted rollback (or remove <purl>) rewrites the restored files entries to https://files.pythonhosted.org/... URLs taken from PyPI's JSON API. The mirror URLs the lock had before the scan are gone. The command reports success, and --dry-run doesn't warn either.

The uv restore refuses a lock whose registry isn't PyPI (upstream/uv.rs:368, "the lock's registry … is not PyPI"), and the Pipenv restore checks _meta.sources the same way (pipenv_index_is_pypi, upstream/pypi.rs:193). restore_pdm has no equivalent check. It never looks at [[tool.pdm.source]] or at the original file URLs.

This is a regression from v4. v4 (f6b7fb9) replayed the recorded original fragment and round-tripped the lock byte-exactly. #277 (2463257) replaced that replay with the upstream restore.

Impact

• PDM 2.20 / 2.29: after rollback, pdm sync downloads the wheel directly from files.pythonhosted.org, and the configured index gets zero requests. That silently bypasses an organisation's mirror, proxy or allow-list policy.
• PDM 2.12.4: on a network where only the mirror is reachable, pdm sync fails after rollback (ProxyError … host='files.pythonhosted.org'). The original lock installs fine on the same network.
• The lock committed after rollback no longer matches what PDM writes for this project, and pdm lock --check doesn't notice, because content_hash is unchanged.

Repro

Requirements: PDM, a local PEP 503 index serving urllib3-1.26.18 (wheel and sdist) at http://127.0.0.1:18780/simple, and a mock patch API on :18765 that grants a hosted wheel for pkg:pypi/[email protected] (the routes from vex_e2e_common/uv.rs::ScanApi). Set SOCKET_PATCH_SERVER_URL to the mock's origin.

cat > pyproject.toml <<'EOF'
[project]
name = "proj"
version = "0.1.0"
requires-python = ">=3.8"
dependencies = ["urllib3==1.26.18"]
[tool.pdm]
distribution = false
[[tool.pdm.source]]
name = "pypi"
url = "http://127.0.0.1:18780/simple"
verify_ssl = false
EOF
pdm lock --static-urls && cp pdm.lock pdm.lock.orig
socket-patch scan --json --yes --ecosystems pypi    # redirected: 1
socket-patch rollback --json --yes                  # status: success, hosted.reverted: [urllib3]
diff pdm.lock.orig pdm.lock
# -    {url = "http://127.0.0.1:18780/files/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b9…"},
# -    {url = "http://127.0.0.1:18780/files/urllib3-1.26.18.tar.gz", hash = "sha256:f8ec…"},
# +    {url = "https://files.pythonhosted.org/packages/0c/39/…/urllib3-1.26.18.tar.gz", hash = "sha256:f8ec…"},
# +    {url = "https://files.pythonhosted.org/packages/b0/53/…/urllib3-1.26.18-py2.py3-none-any.whl", hash = "sha256:34b9…"},
rm -rf .venv; pdm sync -v | grep Downloading     # unearth: Downloading https://files.pythonhosted.org/… (mirror log: 0 hits)
# Mirror-only network: the original lock syncs, the rolled-back lock doesn't (PDM 2.12.4):
HTTPS_PROXY=http://127.0.0.1:9 NO_PROXY=127.0.0.1 pdm sync   # ProxyError host='files.pythonhosted.org'

socket-patch remove pkg:pypi/[email protected] produces the same lock. Each cell below was reproduced at least twice.

Expected vs actual

• Expected: CLI_CONTRACT.md "Hosted unwind coverage" says that "only the hosted entries change and every other byte stays the file's own". The uv and Pipenv restores refuse when the lock's index isn't PyPI, because "the upstream hashes cannot be re-derived". docs/testing/pdm-compatibility.md also says the backtest's rollback "restores the lock … byte for byte". For a PDM lock whose source isn't PyPI, the restore should either keep the lock's own file locations or refuse, telling the user to restore it from version control the way the uv restore does.
• Actual: the restore reports success and swaps the project's index for PyPI's CDN.

Matrix (Linux)

macOS and Windows weren't probed; the restore is platform-independent logic.

First bad commit: 2463257 (#277, "consolidate the v5 patching workflow"). f6b7fb9 is good.

Suspect code

• crates/socket-patch-core/src/patch/redirect/upstream/pypi_locks.rs:341-367 (restore_pdm): static_urls → files_value(release, static_urls) renders PypiFile.url from PyPI's JSON API, with no check of the project's source.
• Compare crates/socket-patch-core/src/patch/redirect/upstream/uv.rs:366-368 and upstream/pypi.rs:193 (pipenv_index_is_pypi).

The same root cause probably affects a non-static_urls lock whose private index serves different bytes under the same name and version: the restored hashes would be PyPI's. I haven't tested that.