feat(pypi): support non-sha256 --hash pins and index digests

[abcdabcd987]

Previously, pip.parse only recognized --hash=sha256: pins in requirements
files and #sha256= URL fragments on Simple API pages. Pins using any other
hash algorithm were silently dropped: the requirement was treated as hash-less,
so with experimental_index_url the resolver fell back to trusting whatever
digests the index listed for that version, and in the pip fallback the hashes
were stripped from the requirement line entirely. The artifact was no longer
verified against what was locked.

This PR parses --hash=<algo>:<digest> (and #<algo>=<digest> URL fragments)
for all hashlib algorithms, matches those pins against the index metadata,
and verifies downloads with the pinned digest.

Fixes #3972.

Before / after behavior

Scenario	Before	After
sha256 pin, index advertises sha256	matched, verified via sha256	unchanged
non-sha256 pin, index advertises the same algorithm	pin dropped; all index digests for the version trusted instead	matched; verified via ctx.download(integrity = ...)
non-sha256 pin, index advertises a different algorithm	same as above	no match; falls back to pip with the pins kept in the requirement line, so pip verifies them; a warning explains the algorithm mismatch
direct URL requirement with only a non-sha256 pin	downloaded unverified	verified via integrity
uv_lock entry with a non-sha256 hash	mangled value passed through as sha256 (download fails)	verified via integrity

Design

Hash representation

A new private module python/private/pypi/hashes.bzl centralizes hash
handling:

• HASH_ALGOS — the accepted algorithm names (hashlib.algorithms_guaranteed,
  the set PEP 691 defines as valid and PEP 503 recommends for fragments).
  Unknown algorithms are still dropped rather than propagated.
• hex_to_sri(algo, hex_digest) / integrity_from_hashes(hashes) — pure
  Starlark hex→base64 conversion to the Subresource Integrity format accepted
  by ctx.download(integrity = ...) (sha256/sha384/sha512 only, strongest
  preferred).
• preferred_digest(hashes) — picks a stable hex digest for e.g. repo naming.

Two representations are used, chosen to fit each side of the match:

• Requirement side (index_sources()): hashes: list[str] of
  "<algo>:<digest>" strings, replacing the old shas: list[str] of bare
  sha256 digests. A requirement can pin several files, possibly with mixed
  algorithms, so a flat list keyed by nothing is the right shape.
• Index side (parse_simpleapi_html() dists, parse_requirements() src
  entries): hashes: dict[str, str] mapping algorithm → hex digest. One
  artifact has at most one digest per algorithm, and this shape is
  forward-compatible with the multi-digest PEP 691 hashes dict.

The existing sha256 fields keep their exact semantics everywhere ("" when
no sha256 digest is known) so the common path is untouched.

Matching

parse_simpleapi_html() keys the whls/sdists dicts by the bare hex digest
from the URL fragment, whatever its algorithm (previously all non-sha256
dists collapsed onto the "" key). _add_dists() strips the <algo>: prefix
from each pin and looks the digest up directly. Digests of different
algorithms have different lengths, so cross-algorithm key collisions are not a
concern, and no dual-key index is needed.

Matching therefore succeeds whenever the index advertises a digest with the
same algorithm as the pin. When it does not, the requirement now falls back to
pip with the pins preserved in the requirement line (previously they were
stripped), so the install is still verified — just without the bazel
downloader.

Download verification

whl_library (both whl_archive and pip_archive) gains an integrity
attribute (SRI string). The hub builder sets it only when sha256 is empty
and a convertible digest is available, so sha256 and integrity are
mutually exclusive by construction and existing lock outputs are byte-for-byte
unchanged for sha256 requirements.

Lockfile facts (MODULE.bazel.lock)

dist_hashes values are now either a bare sha256 hex digest (unchanged legacy
form) or "<algo>:<digest>" for other algorithms; dist_yanked is keyed by
the same stored value. fact_version stays at v1: old facts parse
identically under the new reader, and non-sha256 dists never round-tripped
before this change, so there is nothing to migrate.

This also fixes a latent facts bug: dist_filenames is written keyed by the
distribution URL but was read back keyed by the digest, so stored filenames
(e.g. for URL-encoded wheel names) were never found and the reader always fell
back to deriving the filename from the URL.

API changes

• whl_library / whl_archive / pip_archive: new integrity string
  attribute (documented, :::{versionadded}). Only used when urls is set
  and sha256 is empty.
• Internal (python/private/pypi, not public API):
  • index_sources() returns hashes ("algo:digest" strings) instead of
    shas.
  • parse_simpleapi_html() dist structs and parse_requirements() src
    entries gain a hashes dict; dict keys of whls/sdists are the
    advertised digest of any algorithm.
  • New hashes.bzl module as described above.

Out of scope

Requesting and parsing the PEP 691 JSON Simple API (whose hashes dict can
carry several algorithms per file) would let pins match even when the HTML
fragment uses a different algorithm; that is left as a follow-up feature.

Testing

• New unit tests: tests/pypi/hashes (SRI conversion vectors), non-sha256
  cases in index_sources, parse_simpleapi_html, parse_requirements
  (requirements + uv_lock paths), hub_builder (end-to-end integrity
  arg), and a pypi_cache facts round-trip.
• bazel test //tests/pypi/... — 237/237 pass.

🤖 Generated with Claude Code

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.