Skip to content

[BUG]: CI may build cuda.core against published cuda-bindings instead of the same-commit wheel #2468

Description

@rwgk

codex gpt-5.6-sol


Type of bug

Silent failure

Component

Infrastructure

Describe the bug

The wheel and sdist CI workflows build a cuda-bindings development wheel from the current checkout and expose its directory through PIP_FIND_LINKS when building cuda.core. However, the cuda.core build backend requests only:

cuda-bindings==<CUDA major>.*

PIP_FIND_LINKS adds candidates; it does not prioritize those candidates over the package index or enable prerelease selection. When a stable release satisfies the requirement, pip can therefore select the published stable wheel instead of the same-checkout .dev wheel.

This means cuda.core can be Cythonized against declarations from an older published cuda-bindings package even though CI has just built a newer cuda-bindings wheel from the PR. The resulting green build does not validate that the current cuda.core and cuda-bindings sources are compatible.

This occurred in PR #2395 (and was then discovered while locally testing PR #2393):

  • The same-checkout bindings artifact was 13.3.2.dev....
  • The cuda.core build requirement was cuda-bindings==13.*.
  • Stable cuda-bindings 13.3.1 was available from the package index.
  • The published 13.3.1 cydriver.pxd declared CUDA_MEMCPY3D.srcHost as const void*.
  • The current in-tree cydriver.pxd declared the field as void*.
  • Consequently, params.srcHost = <const void*>... compiled in CI but failed
    when Cython consumed the current in-tree PXD.

The failure from a same-source editable QA build is recorded in cuda-python_qa_bindings_linux_2026-07-31+003242_build_log.txt:

Assigning to 'void *' from 'const void *' discards const qualifier

The standard wheel tests do not expose the problem. They install the current bindings wheel only after the core wheel has already been Cythonized. The const qualification does not change the struct layout or ABI, so runtime testing continues to pass.

The paired pixi source-build workflow could catch this class of incompatibility, but its pull-request path filter does not include ordinary cuda_core/cuda/**/*.pyx changes.

How to reproduce

  1. Build a .dev cuda-bindings wheel from the current checkout.
  2. Leave the normal package index enabled and expose the wheel only through PIP_FIND_LINKS.
  3. Build cuda.core; its backend requests cuda-bindings==13.*.
  4. Observe that the build succeeds against the published stable bindings PXD.
  5. Editable-install the current bindings source and then build the same core source.
  6. Observe that Cython instead consumes the in-tree PXD and reports the discarded-const error.

Expected behavior

When CI builds cuda.bindings and cuda.core from the same checkout, the core build must use the exact bindings artifact produced from that checkout. CI should fail if it cannot resolve that artifact, and the selected bindings version and location should be visible in the build log.

Possible fixes

  • Make the dynamic core build requirement select the exact same-checkout bindings version or artifact. A direct local requirement or exact constraint is more robust than relying on PIP_FIND_LINKS; merely enabling prereleases would still leave room for a different index candidate to win later.
  • Add a build-time assertion that reports and validates the resolved cuda-bindings version and PXD location.
  • Expand the paired source-build workflow's pull-request path filter to cove ordinary cuda_core Cython sources.

Metadata

Metadata

Assignees

No one assigned

    Labels

    CI/CDCI/CD infrastructureP0High priority - Must do!bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions