Skip to content

Move dependencies from requirements.txt to an optional packaging extra #638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 16, 2025
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions .github/workflows/test-wheel-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -318,14 +318,13 @@ jobs:
pushd "${CUDA_BINDINGS_ARTIFACTS_DIR}"
if [[ "${{ matrix.LOCAL_CTK }}" == 1 ]]; then
ls $CUDA_PATH
pip install *.whl
pip install $(ls *.whl)[test]
else
pip install $(ls *.whl)[all]
pip install $(ls *.whl)[all,test]
fi
popd

pushd ./cuda_bindings
pip install -r requirements.txt
${SANITIZER_CMD} pytest -rxXs -v tests/

# It is a bit convoluted to run the Cython tests against CTK wheels,
Expand Down Expand Up @@ -356,11 +355,10 @@ jobs:
fi
TEST_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${{ matrix.CUDA_VER }})"
pushd "${CUDA_CORE_ARTIFACTS_DIR}"
pip install $(ls *.whl)["cu${TEST_CUDA_MAJOR}"]
pip install $(ls *.whl)["cu${TEST_CUDA_MAJOR}","test-cu${TEST_CUDA_MAJOR}"]
popd

pushd ./cuda_core
pip install -r "tests/requirements-cu${TEST_CUDA_MAJOR}.txt"
${SANITIZER_CMD} pytest -rxXs -v tests/

# It is a bit convoluted to run the Cython tests against CTK wheels,
Expand Down
8 changes: 3 additions & 5 deletions .github/workflows/test-wheel-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -246,14 +246,13 @@ jobs:
if ('${{ matrix.LOCAL_CTK }}' -eq '1') {
Get-ChildItem $env:CUDA_PATH
echo $PATH
pip install (Get-ChildItem -Filter *.whl).FullName
pip install "$((Get-ChildItem -Filter *.whl).FullName)[test]"
} else {
pip install "$((Get-ChildItem -Filter *.whl).FullName)[all]"
pip install "$((Get-ChildItem -Filter *.whl).FullName)[all,test]"
}
Pop-Location

Push-Location ./cuda_bindings
pip install -r requirements.txt
pytest -rxXs -v tests/
# skip Cython tests for now (NVIDIA/cuda-python#466)
Pop-Location
Expand All @@ -273,11 +272,10 @@ jobs:
}
$TEST_CUDA_MAJOR = '${{ matrix.CUDA_VER }}' -split '\.' | Select-Object -First 1
Push-Location $env:CUDA_CORE_ARTIFACTS_DIR
pip install "$((Get-ChildItem -Filter *.whl).FullName)[cu${TEST_CUDA_MAJOR}]"
pip install "$((Get-ChildItem -Filter *.whl).FullName)[cu${TEST_CUDA_MAJOR},test-cu${TEST_CUDA_MAJOR}]"
Pop-Location

Push-Location ./cuda_core
pip install -r "tests/requirements-cu${TEST_CUDA_MAJOR}.txt"
pytest -rxXs -v tests/
Pop-Location

Expand Down
2 changes: 1 addition & 1 deletion cuda_bindings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This subpackage adheres to the developing practices described in the parent meta

## Testing

Latest dependencies can be found in [requirements.txt](https://github.com/NVIDIA/cuda-python/blob/main/cuda_bindings/requirements.txt).
Testing dependencies can be installed using the `[test]` optional dependency identifier. For example, `pip install -v -e .[test]`.

Multiple testing options are available:

Expand Down
11 changes: 10 additions & 1 deletion cuda_bindings/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# SPDX-License-Identifier: LicenseRef-NVIDIA-SOFTWARE-LICENSE

[build-system]
requires = ["setuptools>=77.0.0", "cython", "pyclibrary"]
requires = ["setuptools>=77.0.0", "cython>=3.0", "pyclibrary>=0.1.7"]
build-backend = "setuptools.build_meta"

[project]
Expand Down Expand Up @@ -37,6 +37,15 @@ all = [
"nvidia-nvjitlink-cu12>=12.3",
]

test = [
"cython>=3.0",
"distutils",
"numpy>=1.21.1",
"pytest>=6.2.4",
"pytest-benchmark>=3.4.1",
"llvmlite"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but just to point out our options:

  • This could be left commented out (status quo), with
# Uncomment to generate MINIMAL_NVVMIR_BITCODE_STATIC for test_nvvm.py (see PR #443).
  • If we choose to make this an unconditional test dependency: I'll work on a follow-on PR to simplify the logic in cuda_bindings/tests/test_nvvm.py. I believe we can then remove the MINIMAL_NVVMIR_BITCODE_STATIC dictionary entirely.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something I wanted to discuss. I could pull it out into a separate optional dependency group of something like test-nvvm or something of the likes. One of my motivations for uncommenting it was to have it start being tested in CI.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote, based on the quick experiment below (pip install llvmlite takes less than 1 second):

  • Make llvmlite an unconditional test dependency, as you have it right now.
  • I'll do the follow-on PR (quickly, should be easy to squeeze in).

0db6015-lcedt.nvidia.com:~ $ python -m venv junk
0db6015-lcedt.nvidia.com:~ $ . junk/bin/activate
(junk) 0db6015-lcedt.nvidia.com:~ $ pip install --upgrade pip
...
Successfully installed pip-25.1.1
(junk) 0db6015-lcedt.nvidia.com:~ $ time pip install llvmlite
Collecting llvmlite
  Using cached llvmlite-0.44.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.0 kB)
Using cached llvmlite-0.44.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.4 MB)
Installing collected packages: llvmlite
Successfully installed llvmlite-0.44.0

real    0m0.867s
user    0m0.662s
sys     0m0.151s

]

[project.urls]
Repository = "https://github.com/NVIDIA/cuda-python"
Documentation = "https://nvidia.github.io/cuda-python/"
Expand Down
11 changes: 0 additions & 11 deletions cuda_bindings/requirements.txt

This file was deleted.

3 changes: 3 additions & 0 deletions cuda_core/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ dependencies = [
[project.optional-dependencies]
cu11 = ["cuda-bindings==11.8.*"]
cu12 = ["cuda-bindings==12.*"]
test = ["cython>=3.0", "distutils", "pytest>=6.2.4"]
test-cu11 = ["cuda-core[test]", "cupy-cuda11x", "nvidia-cuda-runtime-cu11"] # runtime headers needed by CuPy
test-cu12 = ["cuda-core[test]", "cupy-cuda12x", "nvidia-cuda-runtime-cu12"] # runtime headers needed by CuPy

[project.urls]
homepage = "https://nvidia.github.io/cuda-python/"
Expand Down
4 changes: 0 additions & 4 deletions cuda_core/tests/requirements-cu11.txt

This file was deleted.

4 changes: 0 additions & 4 deletions cuda_core/tests/requirements-cu12.txt

This file was deleted.

Loading