Skip to content

Test Only: Windows Build failure issue #3072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 15 commits into from

Conversation

lanluo-nvidia
Copy link
Collaborator

@lanluo-nvidia lanluo-nvidia commented Aug 8, 2024

Description

For the windows build issue, I am able to identify the issue is caused by the torchtrt.dll:
It is complaining the getinstance method in the UndefinedTensorImpl class

The procedure entry point
?getinstance@UndefinedTensorImpl@c10@@CAAEAU12@XZ
could not be located in the dynamic link library
C:\Users\lanl\git\venv_py310\Lib\site-packages\torch_tensorrt\
lib\torchtrt.dll.

I believe this is related to the libtorch version:
currently we use the one from:

https://download.pytorch.org/libtorch/${CHANNEL}/${CU_VERSION}/libtorch-win-shared-with-deps-latest.zip

which is causing the issue.

Fixes # (issue)
need to find out a good libtorch mapping.

Type of change

Please delete options that are not relevant and/or add your own.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@lanluo-nvidia lanluo-nvidia self-assigned this Aug 8, 2024
@lanluo-nvidia lanluo-nvidia added WIP Work is in progress, pull request should not be merged yet and removed cla signed labels Aug 8, 2024
@github-actions github-actions bot added component: build system Issues re: Build system component: api [Python] Issues re: Python API labels Aug 8, 2024
@lanluo-nvidia lanluo-nvidia changed the title test on windows Windows Build failure issue Aug 13, 2024
@lanluo-nvidia lanluo-nvidia changed the title Windows Build failure issue Test Only: Windows Build failure issue Aug 13, 2024
@HolyWu
Copy link
Contributor

HolyWu commented Aug 14, 2024

The CI failure issue in Windows workflow is caused by version mismatch between the torch library being used for building the wheel and the torch library being used at runtime. In Build torch-tensorrt whl package phase, it's always using the latest nightly version of libtorch. But in the subsequent Test phases, it's installing the version specified in py/requirements.txt. However, torchvision specified >=0.19.0.dev,<0.20.0, and the last torch version matching torchvision 0.19.0 is 2.5.0.dev20240617. The dll from 20240617 doesn't contain the symbol appeared in the newer library, hence the issue. I believe the issue should be gone if you upgrade torch and torchvision by pip3 install --pre -U torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu124 on your local machine. Albeit now there is an upstream breaking change causing maybe_disable_fake_tensor_mode import error after upgrading, it's not related to our issue here.

@lanluo-nvidia
Copy link
Collaborator Author

The CI failure issue in Windows workflow is caused by version mismatch between the torch library being used for building the wheel and the torch library being used at runtime. In Build torch-tensorrt whl package phase, it's always using the latest nightly version of libtorch. But in the subsequent Test phases, it's installing the version specified in py/requirements.txt. However, torchvision specified >=0.19.0.dev,<0.20.0, and the last torch version matching torchvision 0.19.0 is 2.5.0.dev20240617. The dll from 20240617 doesn't contain the symbol appeared in the newer library, hence the issue. I believe the issue should be gone if you upgrade torch and torchvision by pip3 install --pre -U torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu124 on your local machine. Albeit now there is an upstream breaking change causing maybe_disable_fake_tensor_mode import error after upgrading, it's not related to our issue here.

Yes, ultimately if we can always keep update with pytorch latest version, then nightly latest is alright for us.
However it is possible there is some reason we cannot keep up with the latest pytorch version, so I changed it to local_repository instead of getting from nightly latest.
this way libtorch is always synced with torch.
Here is the new PR:
#3086
Closing this one, since it is a test only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed component: api [Python] Issues re: Python API component: build system Issues re: Build system WIP Work is in progress, pull request should not be merged yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants