Skip to content

Hosted and vendored requirements.txt rewrites add a --hash to one line, which turns on pip's hash-checking mode and breaks pip install -r for every other unhashed line #376

Description

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

When requirements.txt has no hashes (the common hand-written or pip freeze case), both scan --mode hosted and scan --mode vendored rewrite the patched pin into a line that carries --hash=sha256:…. pip turns on hash-checking mode for the whole install as soon as any requirement has a --hash (pip docs: "turns on automatically when any package has a hash"). After that, every other line in the file, and every transitive dependency of every line, has to be ==-pinned and hashed too. So the first pip install -r requirements.txt after the scan fails, and nothing gets installed.

The scan reports status: success with no warning. pip's own error even says: "If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash."

The vendored writer's module doc already notes this behaviour ("any --hash on any line turns hash-checking on", crates/socket-patch-core/src/vendor/pypi_requirements.rs:8), but neither writer checks whether the rest of the file can satisfy it. The existing real-pip tests only use a one-line six==1.16.0 file, and six has no dependencies, which is the one shape that still installs.

Impact

  • Any project whose requirements.txt has a second unhashed requirement, or whose patched package has dependencies (e.g. patching requests pulls in urllib3, idna, certifi and charset-normalizer unhashed), can't install at all after a hosted or vendored scan. CI breaks on the next run.
  • In vendored mode, vex still attests not_affected for the patch, even though the wired requirements file can't be installed.

Repro

This uses a local mock of the patch API that serves a patched six-1.16.0 wheel (the same mock the Pipenv, Poetry and Hatch routines use). Any real pypi patch behaves the same.

mkdir h && cd h && git init -q
printf 'six==1.16.0\nidna==3.7\n' > requirements.txt
export SOCKET_API_URL=http://127.0.0.1:18080 SOCKET_API_TOKEN=fake SOCKET_ORG_SLUG=test-org
socket-patch scan --mode hosted --json --yes      # status success, redirected 1, warnings []
cat requirements.txt
# six @ http://127.0.0.1:18080/patch/pypi/six/1.16.0/<token>/<uuid>/six-1.16.0-py2.py3-none-any.whl --hash=sha256:063404c3…
# idna==3.7
python -m venv v && v/bin/pip install -r requirements.txt
# ERROR: Hashes are required in --require-hashes mode, but they are missing from some requirements. ...
#     idna==3.7 --hash=sha256:82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0

Vendored gives the same result (with .venv holding the pristine six):

socket-patch scan --mode vendored --json --yes    # status success
# requirements.txt: ./.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl --hash=sha256:01663f71…  # socket-patch vendor: six==1.16.0
#                   idna==3.7
pip install -r requirements.txt                   # same "Hashes are required" error

As controls: a one-line six==1.16.0 file installs PATCHED, and a file that was already fully hashed (pip-compile --generate-hashes style) installs PATCHED. Both reproduced twice on the current main.

Expected vs actual

  • Expected: after a hosted or vendored scan, pip install -r requirements.txt installs the patched artifact and leaves every other requirement as it was. docs/testing/uv-compatibility.md says for hosted "Exact version pins become direct artifact URLs with the patched SHA-256… hashes for the replaced artifact are removed", and for vendored that the file refers to the committed wheel "with its hash". Neither says the other requirements get pulled into hash-checking mode. If the file can't be made hash-complete, the scan should refuse or warn. (Possible fixes: emit the pin without --hash when no other line is hashed, relying on the URL/#sha256= fragment instead; or refuse with a dedicated code. That's the maintainers' call.)
  • Actual: status: success, no warning, and pip then refuses to install anything.

OS × version

Probe run https://github.com/SocketDev/socket-patch/actions/runs/36771795909 (plus local Linux runs):

OS Python pip hosted, 2 unhashed lines vendored, 2 unhashed lines hosted, single six line (control)
Linux 3.8 20.3.4 / 23.3.2 / bundled fail fail pass
Linux 3.10 (local) 20.3.4 / 23.3.2 / 26.2.1 fail — —
Linux 3.11 / 3.13 24.0 / 23.3.2 / 26.2.1 fail fail pass
macOS 3.8 / 3.13 20.3.4 / 23.3.2 / bundled (26.2.1) fail fail pass
Windows 3.8 / 3.13 20.3.4 / 23.3.2 / bundled (26.2.1) fail fail pass

uv pip was not checked: the local mock doesn't answer the HEAD request uv sends.

First bad release

The released 4.0.0 (from PyPI) reproduces in both modes. 3.3.0 has no hosted or vendored mode.

Suspect code

  • crates/socket-patch-core/src/patch/redirect/requirements.rs:295: rewritten.push_str(&format!(" --hash=sha256:{sha256}")) runs unconditionally.
  • crates/socket-patch-core/src/vendor/pypi_requirements.rs:581 (vendor_line): always emits --hash, and wire_requirements (:215) doesn't check whether the other lines are hashed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions