Skip to content

Pipenv recognizes hosted PyPI patch URLs with two private grammars that disagree with the shared one #563

Description

[agent] Filed by the scheduled architecture audit routine (ecosystems and formats). Register: #560 (comment).

Kind: bug. Source: review Part 5.4 ("Hosted-URL recognition is duplicated"), register E04. The hosted-side half is a new finding, register E49.

Problem

Four functions decide whether a URL in a Python lock is a Socket-hosted PyPI patch reference:

function origin path artifact
redirect::hosted_patch_uuid (redirect/mod.rs#L4682-L4740) patch.socket.dev or a configured --patch-server-url origin any (a path prefix is allowed) any
lock_inventory::pypi::hosted_artifact_url (lock_inventory/pypi.rs#L109-L162),`` used by inventory and VEX any …/patch/pypi/<n>/<v>/<grant>/<uuid>/<leaf> matched from the end ("a configured origin may carry a path prefix") wheel or sdist
hosted Pipenv owned_url (redirect/pipenv.rs#L245-L279) the grant's origin or patch.socket.dev exactly 8 segments from the root (no prefix) .whl only
vendored Pipenv is_socket_hosted_reference (vendor/pypi_pipenv.rs#L612-L622) any https host exactly 7 segments from the root .whl only

Reproduced on d63ae5f (unit tests, run twice):

  • Hosted Pipenv cannot rotate its own pin on a path-prefixed patch server. With artifactUrl = https://patches.internal.example/socket/patch/pypi/urllib3/1.26.18/token/patch-one/urllib3-1.26.18-py3-none-any.whl, owned_url(artifact_url, dep) is false for the URL it just wrote. The second plan() (grant rotated) returns Conflict("Pipenv source for urllib3 already exists") (redirect/pipenv.rs#L355-L360).`` On the same URL hosted_artifact_url returns `uuid_level = Some("patch-one")`, so VEX and inventory treat the entry as Socket's while hosted treats it as a user source. The same origin without a path prefix (`:8443`) rotates fine.
  • Vendored Pipenv calls a foreign host's URL Socket-hosted. is_socket_hosted_reference("https://evil.example/patch/pypi/requests/2.28.1/<grant>/<uuid>/requests-2.28.1-py3-none-any.whl") is true, while hosted_patch_uuid returns None. A real hosted sdist URL, or a path-prefixed hosted URL, returns false in the vendored check and Some(uuid) in hosted_patch_uuid. The refusal code is the same either way (pypi_pipenv_source_already_exists, #L229-L240),`` but the remedy it prints ("run socket-patch rollback" versus "user-authored") is wrong in both directions.

Symptoms

None filed.

Impact:

  • Hosted Pipenv on a path-prefixed --patch-server-url deployment cannot re-run or rotate grants: every scan after the first refuses.
  • Vendored Pipenv gives misleading remediation.
  • The grammar will drift again with the next shape change. Poetry, pdm and uv use other recognizers; check them in the same PR.

Size: small.

Proposed change

  • Make one recognizer the authority: hosted_artifact_url for the coordinates, combined with hosted_patch_uuid's origin allowlist (the configured origins plus patch.socket.dev) for "is it ours". Put it next to hosted_artifact_url, or in redirect::hosted_url.
  • Hosted owned_url becomes "the shared recognizer accepts it, and its name, version and leaf match dep".
  • Vendored is_socket_hosted_reference becomes "the shared recognizer accepts it with the configured origins".
  • Delete both private grammars (the segment-count checks).

Size and scope

patch/redirect/pipenv.rs, vendor/pypi_pipenv.rs, and possibly vendor/lock_inventory/pypi.rs. About −40 / +25 production lines. Out of scope: the other pypi lock writers, beyond a note in the PR if their checks differ.

Acceptance criteria

  • Neither private grammar remains.
  • Regression test: hosted Pipenv rotates its pin on a path-prefixed --patch-server-url origin.
  • Regression test: vendored Pipenv treats a foreign-host /patch/pypi/… URL as user-authored, and treats a hosted sdist or path-prefixed URL on an accepted origin as Socket-hosted.
  • owned_url_follows_the_grant_origin, compatibility_tests, the vendored Pipenv tests and vex/discover/pypi_* stay green.

Dependencies

None. This is independent of E23 (the pypi backend skeleton) but shrinks it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)bugSomething isn't workingpm:pipenvPipenvpriority:p1

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions