fix tensorrt dependencies issue #3105

lanluo-nvidia · 2024-08-20T17:22:51Z

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project (You can use the linters)
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes
I have added the relevant labels to my PR in so that relevant reviewers are notified

pytorch-bot · 2024-08-20T17:22:53Z

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

lanluo-nvidia · 2024-08-20T22:02:48Z

I have verified it is now only installing cu11 related dependencies for cu118:
Successfully installed nvidia-cuda-runtime-cu11-2022.4.25 nvidia-cuda-runtime-cu117-11.7.60 tensorrt-cu11-10.1.0 tensorrt-cu11-bindings-10.1.0 tensorrt-cu11-libs-10.1.0 torch-tensorrt-2.5.0.dev20240820+cu118

narendasan · 2024-08-21T00:57:13Z

any idea why this gets installed nvidia-cuda-runtime-cu117-11.7.60? is it a TRT dep?

narendasan · 2024-08-21T00:58:01Z

Seems like theres failures specific to TRT-cu11

HolyWu · 2024-08-21T16:35:31Z

tensorrt should be remove from pyproject.toml, py/requirements.txt and tests/py/requirements.txt so as to actually solve the issue, leaving only tensorrt-cuXX/bindings/libs in pyproject.toml. Otherwise the tensorrt meta package will still pull in CUDA 12 variants, resulting in both cu11 and cu12 being installed in the cu118 case.

(trt) holywu@HOLYWU:~$ pip list
Package                  Version
------------------------ ------------------------
filelock                 3.13.1
fsspec                   2024.6.1
Jinja2                   3.1.4
MarkupSafe               2.1.5
mpmath                   1.3.0
networkx                 3.3
numpy                    1.26.4
nvidia-cublas-cu11       11.11.3.6
nvidia-cuda-cupti-cu11   11.8.87
nvidia-cuda-nvrtc-cu11   11.8.89
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cudnn-cu11        9.1.0.70
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.3.0.86
nvidia-cusolver-cu11     11.4.1.48
nvidia-cusparse-cu11     11.7.5.86
nvidia-nccl-cu11         2.21.5
nvidia-nvtx-cu11         11.8.86
Pillow                   10.1.0
pip                      24.2
pytorch-triton           3.0.0+dedb7bdf33
setuptools               73.0.1
sympy                    1.13.1
torch                    2.5.0.dev20240821+cu118
torchvision              0.20.0.dev20240821+cu118
typing_extensions        4.12.2
wheel                    0.44.0
(trt) holywu@HOLYWU:~$
(trt) holywu@HOLYWU:~$ pip install torch_tensorrt-2.5.0.dev20240820+cu118-cp312-cp312-linux_x86_64.whl
Processing ./torch_tensorrt-2.5.0.dev20240820+cu118-cp312-cp312-linux_x86_64.whl
Requirement already satisfied: torch<2.6.0,>=2.5.0.dev in ./trt/lib/python3.12/site-packages (from torch-tensorrt==2.5.0.dev20240820+cu118) (2.5.0.dev20240821+cu118)
Collecting tensorrt==10.1.0 (from torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading tensorrt-10.1.0.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Collecting tensorrt-cu11==10.1.0 (from torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading tensorrt-cu11-10.1.0.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Collecting tensorrt-cu11-bindings==10.1.0 (from torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading tensorrt_cu11_bindings-10.1.0-cp312-none-manylinux_2_17_x86_64.whl.metadata (627 bytes)
Collecting tensorrt-cu11-libs==10.1.0 (from torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading tensorrt_cu11_libs-10.1.0.tar.gz (629 bytes)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting packaging>=23 (from torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Requirement already satisfied: numpy in ./trt/lib/python3.12/site-packages (from torch-tensorrt==2.5.0.dev20240820+cu118) (1.26.4)
Requirement already satisfied: typing-extensions>=4.7.0 in ./trt/lib/python3.12/site-packages (from torch-tensorrt==2.5.0.dev20240820+cu118) (4.12.2)
Collecting tensorrt-cu12 (from tensorrt==10.1.0->torch-tensorrt==2.5.0.dev20240820+cu118)
  Downloading tensorrt-cu12-10.3.0.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: nvidia-cuda-runtime-cu11 in ./trt/lib/python3.12/site-packages (from tensorrt-cu11-libs==10.1.0->torch-tensorrt==2.5.0.dev20240820+cu118) (11.8.89)
Requirement already satisfied: filelock in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (3.13.1)
Requirement already satisfied: networkx in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (3.3)
Requirement already satisfied: jinja2 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (3.1.4)
Requirement already satisfied: fsspec in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (2024.6.1)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.8.89 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.8.89)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.8.87 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.8.87)
Requirement already satisfied: nvidia-cudnn-cu11==9.1.0.70 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu11==11.11.3.6 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.11.3.6)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (10.9.0.58)
Requirement already satisfied: nvidia-curand-cu11==10.3.0.86 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (10.3.0.86)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.1.48 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.4.1.48)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.5.86 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.7.5.86)
Requirement already satisfied: nvidia-nccl-cu11==2.21.5 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu11==11.8.86 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (11.8.86)
Requirement already satisfied: pytorch-triton==3.0.0+dedb7bdf33 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (3.0.0+dedb7bdf33)
Requirement already satisfied: setuptools in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (73.0.1)
Requirement already satisfied: sympy==1.13.1 in ./trt/lib/python3.12/site-packages (from torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./trt/lib/python3.12/site-packages (from sympy==1.13.1->torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./trt/lib/python3.12/site-packages (from jinja2->torch<2.6.0,>=2.5.0.dev->torch-tensorrt==2.5.0.dev20240820+cu118) (2.1.5)
Downloading tensorrt_cu11_bindings-10.1.0-cp312-none-manylinux_2_17_x86_64.whl (1.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 3.8 MB/s eta 0:00:00
Downloading packaging-24.1-py3-none-any.whl (53 kB)
Building wheels for collected packages: tensorrt, tensorrt-cu11, tensorrt-cu11-libs, tensorrt-cu12
  Building wheel for tensorrt (setup.py) ... done
  Created wheel for tensorrt: filename=tensorrt-10.1.0-py2.py3-none-any.whl size=16334 sha256=c6befc2314ef55b3859023e4d8084c510990307b354f85e3c87b5d6427d739e5
  Stored in directory: /home/holywu/.cache/pip/wheels/6c/fc/c4/7e30345c05db37c229e8c91c5e3de381dc2ea9707106d60519
  Building wheel for tensorrt-cu11 (setup.py) ... done
  Created wheel for tensorrt-cu11: filename=tensorrt_cu11-10.1.0-py2.py3-none-any.whl size=17552 sha256=95709f214e9e6386b74dbf0be4a560bdbfc17dd394815a7ab15cae4db82993dd
  Stored in directory: /home/holywu/.cache/pip/wheels/e6/24/88/424588eb0d1f25a4a0f3ca3731c83b35f770b3ed5b4fee44c3
  Building wheel for tensorrt-cu11-libs (pyproject.toml) ... done
  Created wheel for tensorrt-cu11-libs: filename=tensorrt_cu11_libs-10.1.0-py2.py3-none-manylinux_2_17_x86_64.whl size=1055732539 sha256=41e8bda53a5cbd5f70b867f02abf45b7951e5a26c3ee044677a27edf5763ce05
  Stored in directory: /home/holywu/.cache/pip/wheels/bc/cb/e1/8a3f90f242d9c033da712c483991891bf67a14edc5244760e1
  Building wheel for tensorrt-cu12 (setup.py) ... done
  Created wheel for tensorrt-cu12: filename=tensorrt_cu12-10.3.0-py2.py3-none-any.whl size=17554 sha256=33fd8794ff31ee650831f23b261555cef8e97f60ab8428a100b870e51cb1b102
  Stored in directory: /home/holywu/.cache/pip/wheels/6f/a9/59/d4b78cd2fbbde9f0441fa38ac4127b774b03576f4d0f5515be
Successfully built tensorrt tensorrt-cu11 tensorrt-cu11-libs tensorrt-cu12
Installing collected packages: tensorrt-cu11-bindings, tensorrt-cu12, tensorrt-cu11-libs, tensorrt-cu11, packaging, tensorrt, torch-tensorrt
Successfully installed packaging-24.1 tensorrt-10.1.0 tensorrt-cu11-10.1.0 tensorrt-cu11-bindings-10.1.0 tensorrt-cu11-libs-10.1.0 tensorrt-cu12-10.3.0 torch-tensorrt-2.5.0.dev20240820+cu118
(trt) holywu@HOLYWU:~$
(trt) holywu@HOLYWU:~$ pip list
Package                  Version
------------------------ ------------------------
filelock                 3.13.1
fsspec                   2024.6.1
Jinja2                   3.1.4
MarkupSafe               2.1.5
mpmath                   1.3.0
networkx                 3.3
numpy                    1.26.4
nvidia-cublas-cu11       11.11.3.6
nvidia-cuda-cupti-cu11   11.8.87
nvidia-cuda-nvrtc-cu11   11.8.89
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cuda-runtime-cu12 12.6.37
nvidia-cudnn-cu11        9.1.0.70
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.3.0.86
nvidia-cusolver-cu11     11.4.1.48
nvidia-cusparse-cu11     11.7.5.86
nvidia-nccl-cu11         2.21.5
nvidia-nvtx-cu11         11.8.86
packaging                24.1
Pillow                   10.1.0
pip                      24.2
pytorch-triton           3.0.0+dedb7bdf33
setuptools               73.0.1
sympy                    1.13.1
tensorrt                 10.1.0
tensorrt-cu11            10.1.0
tensorrt-cu11-bindings   10.1.0
tensorrt-cu11-libs       10.1.0
tensorrt-cu12            10.3.0
tensorrt-cu12-bindings   10.3.0
tensorrt-cu12-libs       10.3.0
torch                    2.5.0.dev20240821+cu118
torch_tensorrt           2.5.0.dev20240820+cu118
torchvision              0.20.0.dev20240821+cu118
typing_extensions        4.12.2
wheel                    0.44.0

lanluo-nvidia · 2024-08-21T20:24:10Z

@narendasan @HolyWu please review again
I have removed tensorrt and put tensorrt-cu11/12 as dependencies instead, the pipeline looks good except the existing failed testcases which is not related to this change.
I also did another test which is download the wheel and make sure it is no longer pulling tensorrt-cu12 as dependency for cu118.

narendasan · 2024-08-21T20:26:34Z

As long as TRT gets installed automatically and the library is ready to go if you do pip install torch-tensorrt I am ok with this change, but we don't want to start forcing people to install trt separately.

lanluo-nvidia · 2024-08-21T20:30:07Z

any idea why this gets installed nvidia-cuda-runtime-cu117-11.7.60? is it a TRT dep?

@narendasan this is due to my local environment does not have cuda118 runtime installed, it pulled this as dependency, not sure why it is pulling cuda runtime cu117 not cu118.
I checked in the pipeline in cu118, it is as expected as below:
Successfully installed tensorrt-cu11-10.1.0 tensorrt-cu11-bindings-10.1.0 tensorrt-cu11-libs-10.1.0 torch-tensorrt-2.5.0.dev20240821+cu118

lanluo-nvidia · 2024-08-21T20:31:33Z

As long as TRT gets installed automatically and the library is ready to go if you do pip install torch-tensorrt I am ok with this change, but we don't want to start forcing people to install trt separately.

pip install torch-tensorrt will get tensorrt installed automatically:
Successfully installed tensorrt-cu11-10.1.0 tensorrt-cu11-bindings-10.1.0 tensorrt-cu11-libs-10.1.0 torch-tensorrt-2.5.0.dev20240821+cu118

HolyWu · 2024-08-22T14:04:20Z

Not related to this PR's purpose, but could you also remove pip install --pre -r tests/py/requirements.txt --use-deprecated legacy-resolver from both pre_build_script? The packages in tests/py/requirements.txt are only required when running testcases. They are of no use in the build phase when creating the wheel.

Otherwise the changes LGTM.

fix tensorrt dependencies issue

bc39698

lanluo-nvidia added WIP Work is in progress, pull request should not be merged yet ciflow/binaries/all Build for all Python Versions labels Aug 20, 2024

lanluo-nvidia self-assigned this Aug 20, 2024

facebook-github-bot added the cla signed label Aug 20, 2024

github-actions bot added the component: build system Issues re: Build system label Aug 20, 2024

github-actions bot requested a review from narendasan August 20, 2024 17:25

test

7e8824b

lanluo-nvidia mentioned this pull request Aug 20, 2024

🐛 [Bug] The version for CUDA 11 depends on CUDA 12 libraries #3102

Closed

lanluo-nvidia added 2 commits August 20, 2024 13:32

test

806a0ee

test

440dc06

lanluo-nvidia removed the WIP Work is in progress, pull request should not be merged yet label Aug 20, 2024

lanluo-nvidia marked this pull request as ready for review August 20, 2024 22:02

test according to Holy's comments

ed27551

github-actions bot added component: tests Issues re: Tests component: api [Python] Issues re: Python API labels Aug 21, 2024

test

2d27a30

lanluo-nvidia merged commit 3e7be44 into main Aug 22, 2024
213 of 263 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix tensorrt dependencies issue #3105

fix tensorrt dependencies issue #3105

Uh oh!

lanluo-nvidia commented Aug 20, 2024

Uh oh!

pytorch-bot bot commented Aug 20, 2024

Uh oh!

lanluo-nvidia commented Aug 20, 2024

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

HolyWu commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024 •

edited

Loading

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024 •

edited

Loading

Uh oh!

HolyWu commented Aug 22, 2024

Uh oh!

Uh oh!

Uh oh!

fix tensorrt dependencies issue #3105

fix tensorrt dependencies issue #3105

Uh oh!

Conversation

lanluo-nvidia commented Aug 20, 2024

Description

Type of change

Checklist:

Uh oh!

pytorch-bot bot commented Aug 20, 2024

Uh oh!

lanluo-nvidia commented Aug 20, 2024

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

HolyWu commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

narendasan commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024

Uh oh!

lanluo-nvidia commented Aug 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HolyWu commented Aug 22, 2024

Uh oh!

Uh oh!

Uh oh!

lanluo-nvidia commented Aug 21, 2024 •

edited

Loading

lanluo-nvidia commented Aug 21, 2024 •

edited

Loading