Skip to content

Commit 090b9b9

Browse files
HolyWugs-olive
HolyWu
authored and
gs-olive
committed
Fix C++ and Python building on Windows
Add build-test-windows job 9.3.0 Upgrade and version fixing Add bazelisk installation Update installation of Torch-TRT Bazel install fix Python only for initial Windows build Conda install latest cuda toolkit CUDA Driver Update CUDA Driver Update 2 Get windows version info Attempt driver upgrade within system Switch installer script to curl Switch installer script to network Try testing E2E models on Windows Try linux testing action for Windows Update Windows build/test Use local version of yml Use local version of yml 2 Use local version of yml 3 Use local version of yml 4 Remove CONDA_RUN references Update Python source Add torch-trt install script packing Remove extraneous python references Replace pipefail command Replace pipefail command 2 Replace source command Update cat command for Powershell Avoid script overwrite in runner Avoid script overwrite in runner 2 Update path concat to use echo fix: Use forked infra repository Skip things not supported by GitHub-hosted Windows runner Upgrade setuptools and wheel Rework windows test Restore root WORKSPACE to original Attribute naming fix in YML Enable one test in Windows Enable GPU Access Enable GPU Access Attempt 2 Enable GPU Access Attempt 3 - revert Switch off miniconda setup Specify Python 3.9 as default for testing Specify Python 3.9 as default for testing 2 Revert py39 changes, make changes to infra fork, re-enable cu121 installation Failing cuda installation Failing cuda installation 2 Failing cuda installation 3 Differentiate pre-build script during build and test Install trt bindings + lib manually Switch back to pytorch as repo source Specify validation runner Revert "Specify validation runner" This reverts commit 6d664c0. Enable more tests Enable more tests 2 Remove Python runtime specification Clean up and enable additional Windows tests Use .bat for driver upgrade Use .bat for driver upgrade 2 UTF8 encoding for windows testing
1 parent 38642bb commit 090b9b9

File tree

27 files changed

+676
-97
lines changed

27 files changed

+676
-97
lines changed

‎.bazelrc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ build --cxxopt="-fdiagnostics-color=always"
2525
build --cxxopt='-std=c++17'
2626
#build --linkopt="-Wl,--no-as-needed"
2727

28+
build:windows --cxxopt="/GS-" --cxxopt="/std:c++17" --cxxopt="/permissive-"
29+
build:windows --cxxopt="/wd4244" --cxxopt="/wd4267" --cxxopt="/wd4819"
30+
build:windows --features=windows_export_all_symbols
2831

2932
build:python --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"
3033
build:python --linkopt="-D_GLIBCXX_USE_CXX11_ABI=0"

‎.github/scripts/install-torch-tensorrt-windows.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
source "${BUILD_ENV_FILE}"
2+
${CONDA_RUN} ${PIP_INSTALL_TORCH} torchvision
3+
${CONDA_RUN} pip install pyyaml mpmath==1.3.0
4+
${CONDA_RUN} pip install ${RUNNER_ARTIFACT_DIR}/torch_tensorrt*.whl tensorrt==9.3.0.post12.dev1 tensorrt-bindings==9.3.0.post12.dev1 tensorrt-libs==9.3.0.post12.dev1 --extra-index-url https://pypi.nvidia.com
5+
6+
echo -e "Running test script";

‎.github/workflows/build-test-windows.yml

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
name: Build and test Windows wheels
2+
3+
on: [pull_request]
4+
5+
jobs:
6+
generate-matrix:
7+
uses: pytorch/test-infra/.github/workflows/generate_binary_build_matrix.yml@main
8+
with:
9+
package-type: wheel
10+
os: windows
11+
test-infra-repository: pytorch/test-infra
12+
test-infra-ref: main
13+
with-rocm: false
14+
with-cpu: false
15+
16+
build:
17+
needs: generate-matrix
18+
permissions:
19+
id-token: write
20+
contents: read
21+
strategy:
22+
fail-fast: false
23+
matrix:
24+
include:
25+
- repository: pytorch/tensorrt
26+
pre-script: packaging/pre_build_script_windows.sh
27+
smoke-test-script: packaging/smoke_test_windows.py
28+
package-name: torch_tensorrt
29+
name: Build torch-tensorrt whl package
30+
uses: pytorch/test-infra/.github/workflows/build_wheels_windows.yml@main
31+
with:
32+
repository: ${{ matrix.repository }}
33+
ref: ""
34+
test-infra-repository: pytorch/test-infra
35+
test-infra-ref: main
36+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
37+
pre-script: ${{ matrix.pre-script }}
38+
smoke-test-script: ${{ matrix.smoke-test-script }}
39+
package-name: ${{ matrix.package-name }}
40+
trigger-event: ${{ github.event_name }}
41+
42+
# tests-py-torchscript-fe:
43+
# name: Test torchscript frontend [Python]
44+
# needs: [generate-matrix, build]
45+
# strategy:
46+
# fail-fast: false
47+
# matrix:
48+
# include:
49+
# - repository: pytorch/tensorrt
50+
# package-name: torch_tensorrt
51+
# pre-script: packaging/pre_build_script.sh
52+
# uses: pytorch/tensorrt/.github/workflows/windows-test.yml@main
53+
# with:
54+
# job-name: tests-py-torchscript-fe
55+
# repository: "pytorch/tensorrt"
56+
# ref: ""
57+
# test-infra-repository: pytorch/test-infra
58+
# test-infra-ref: main
59+
# build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
60+
# pre-script: ${{ matrix.pre-script }}
61+
# script: |
62+
# export USE_HOST_DEPS=1
63+
# export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
64+
# pushd .
65+
# cd tests/modules
66+
# ${CONDA_RUN} python -m pip install --pre -r requirements.txt --use-deprecated=legacy-resolver
67+
# ${CONDA_RUN} python hub.py
68+
# popd
69+
# pushd .
70+
# cd tests/py/ts
71+
# ${CONDA_RUN} python -m pip install --pre pytest timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
72+
# ${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_api_test_results.xml api/
73+
# ${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_models_test_results.xml models/
74+
# ${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/ts_integrations_test_results.xml integrations/
75+
# popd
76+
77+
tests-py-dynamo-converters:
78+
name: Test dynamo converters [Python]
79+
needs: [generate-matrix, build]
80+
strategy:
81+
fail-fast: false
82+
matrix:
83+
include:
84+
- repository: pytorch/tensorrt
85+
package-name: torch_tensorrt
86+
uses: ./.github/workflows/windows-test.yml
87+
with:
88+
job-name: tests-py-dynamo-converters
89+
repository: "pytorch/tensorrt"
90+
ref: ""
91+
test-infra-repository: pytorch/test-infra
92+
test-infra-ref: main
93+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
94+
pre-script: packaging/driver_upgrade.bat
95+
script: |
96+
export USE_HOST_DEPS=1
97+
pushd .
98+
cd tests/py/dynamo
99+
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
100+
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_converters_test_results.xml -n 10 conversion/
101+
popd
102+
103+
tests-py-dynamo-fe:
104+
name: Test dynamo frontend [Python]
105+
needs: [generate-matrix, build]
106+
strategy:
107+
fail-fast: false
108+
matrix:
109+
include:
110+
- repository: pytorch/tensorrt
111+
package-name: torch_tensorrt
112+
uses: ./.github/workflows/windows-test.yml
113+
with:
114+
job-name: tests-py-dynamo-fe
115+
repository: "pytorch/tensorrt"
116+
ref: ""
117+
test-infra-repository: pytorch/test-infra
118+
test-infra-ref: main
119+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
120+
pre-script: packaging/driver_upgrade.bat
121+
script: |
122+
export USE_HOST_DEPS=1
123+
pushd .
124+
cd tests/py/dynamo
125+
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
126+
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dynamo_fe_test_results.xml --ir dynamo models/test_models_export.py
127+
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/dyn_models_export.xml --ir dynamo models/test_dyn_models.py
128+
popd
129+
130+
tests-py-dynamo-serde:
131+
name: Test dynamo export serde [Python]
132+
needs: [generate-matrix, build]
133+
strategy:
134+
fail-fast: false
135+
matrix:
136+
include:
137+
- repository: pytorch/tensorrt
138+
package-name: torch_tensorrt
139+
uses: ./.github/workflows/windows-test.yml
140+
with:
141+
job-name: tests-py-dynamo-serde
142+
repository: "pytorch/tensorrt"
143+
ref: ""
144+
test-infra-repository: pytorch/test-infra
145+
test-infra-ref: main
146+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
147+
pre-script: packaging/driver_upgrade.bat
148+
script: |
149+
export USE_HOST_DEPS=1
150+
pushd .
151+
cd tests/py/dynamo
152+
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
153+
${CONDA_RUN} python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/export_serde_test_results.xml --ir dynamo models/test_export_serde.py
154+
popd
155+
156+
tests-py-torch-compile-be:
157+
name: Test torch compile backend [Python]
158+
needs: [generate-matrix, build]
159+
strategy:
160+
fail-fast: false
161+
matrix:
162+
include:
163+
- repository: pytorch/tensorrt
164+
package-name: torch_tensorrt
165+
uses: ./.github/workflows/windows-test.yml
166+
with:
167+
job-name: tests-py-torch-compile-be
168+
repository: ${{ matrix.repository }}
169+
ref: ""
170+
test-infra-repository: pytorch/test-infra
171+
test-infra-ref: main
172+
pre-script: packaging/driver_upgrade.bat
173+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
174+
script: |
175+
export USE_HOST_DEPS=1
176+
pushd .
177+
cd tests/py/dynamo
178+
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
179+
${CONDA_RUN} python -m pytest -n 10 --junitxml=${RUNNER_TEST_RESULTS_DIR}/torch_compile_be_test_results.xml backend/
180+
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/torch_comple_be_e2e_test_results.xml --ir torch_compile models/test_models.py
181+
popd
182+
183+
tests-py-dynamo-core:
184+
name: Test dynamo core [Python]
185+
needs: [generate-matrix, build]
186+
strategy:
187+
fail-fast: false
188+
matrix:
189+
include:
190+
- repository: pytorch/tensorrt
191+
package-name: torch_tensorrt
192+
uses: ./.github/workflows/windows-test.yml
193+
with:
194+
job-name: tests-py-dynamo-core
195+
repository: "pytorch/tensorrt"
196+
ref: ""
197+
test-infra-repository: pytorch/test-infra
198+
test-infra-ref: main
199+
build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
200+
pre-script: packaging/driver_upgrade.bat
201+
script: |
202+
export USE_HOST_DEPS=1
203+
pushd .
204+
cd tests/py/dynamo
205+
${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
206+
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_test_results.xml runtime/
207+
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_partitioning_test_results.xml partitioning/
208+
${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_lowering_test_results.xml lowering/
209+
popd
210+
211+
# tests-py-core:
212+
# name: Test core [Python]
213+
# needs: [generate-matrix, build]
214+
# strategy:
215+
# fail-fast: false
216+
# matrix:
217+
# include:
218+
# - repository: pytorch/tensorrt
219+
# package-name: torch_tensorrt
220+
# pre-script: packaging/pre_build_script.sh
221+
# uses: pytorch/tensorrt/.github/workflows/windows-test.yml@main
222+
# with:
223+
# job-name: tests-py-core
224+
# repository: "pytorch/tensorrt"
225+
# ref: ""
226+
# test-infra-repository: pytorch/test-infra
227+
# test-infra-ref: main
228+
# build-matrix: ${{ needs.generate-matrix.outputs.matrix }}
229+
# pre-script: ${{ matrix.pre-script }}
230+
# script: |
231+
# export USE_HOST_DEPS=1
232+
# pushd .
233+
# cd tests/py/core
234+
# ${CONDA_RUN} python -m pip install --pre pytest-xdist timm transformers parameterized expecttest==0.1.6 --use-deprecated=legacy-resolver
235+
# ${CONDA_RUN} python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_core_test_results.xml .
236+
# popd

‎.github/workflows/windows-test.yml

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
name: Test on Windows
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
repository:
7+
description: 'Repository to checkout, defaults to ""'
8+
default: ""
9+
type: string
10+
ref:
11+
description: 'Reference to checkout, defaults to "nightly"'
12+
default: "nightly"
13+
type: string
14+
test-infra-repository:
15+
description: "Test infra repository to use"
16+
default: "pytorch/test-infra"
17+
type: string
18+
test-infra-ref:
19+
description: "Test infra reference to use"
20+
default: ""
21+
type: string
22+
build-matrix:
23+
description: "Build matrix to utilize"
24+
default: ""
25+
type: string
26+
pre-script:
27+
description: "Pre script to run prior to build"
28+
default: ""
29+
type: string
30+
script:
31+
description: 'Script to utilize'
32+
default: "python setup.py bdist_wheel"
33+
type: string
34+
job-name:
35+
description: "Name for the job, which is displayed in the GitHub UI"
36+
default: "windows-job"
37+
type: string
38+
continue-on-error:
39+
description: "Prevents a job from failing when a step fails. Set to true to allow a job to pass when exec script step fails."
40+
default: false
41+
type: boolean
42+
43+
jobs:
44+
test:
45+
strategy:
46+
fail-fast: false
47+
matrix: ${{ fromJSON(inputs.build-matrix) }}
48+
env:
49+
PYTHON_VERSION: ${{ matrix.python_version }}
50+
PACKAGE_TYPE: wheel
51+
REPOSITORY: ${{ inputs.repository }}
52+
REF: ${{ inputs.ref }}
53+
CU_VERSION: ${{ matrix.desired_cuda }}
54+
SCRIPT: ${{ inputs.script }}
55+
PYTHONUTF8: 1
56+
name: ${{ inputs.job-name }}-${{ matrix.desired_cuda }}
57+
runs-on: windows.8xlarge.nvidia.gpu.nonephemeral
58+
defaults:
59+
run:
60+
shell: bash -l {0}
61+
# If a build is taking longer than 60 minutes on these runners we need
62+
# to have a conversation
63+
timeout-minutes: 60
64+
steps:
65+
- name: Clean workspace
66+
run: |
67+
echo "::group::Cleanup debug output"
68+
rm -rfv "${GITHUB_WORKSPACE}"
69+
mkdir -p "${GITHUB_WORKSPACE}"
70+
echo "::endgroup::"
71+
- name: Checkout repository (${{ inputs.test-infra-repository }}@${{ inputs.test-infra-ref }})
72+
uses: actions/checkout@v3
73+
with:
74+
# Support the use case where we need to checkout someone's fork
75+
repository: ${{ inputs.test-infra-repository }}
76+
ref: ${{ inputs.test-infra-ref }}
77+
path: test-infra
78+
- name: Setup Windows
79+
uses: ./test-infra/.github/actions/setup-windows
80+
- name: Setup SSH
81+
uses: ./test-infra/.github/actions/setup-ssh
82+
with:
83+
github-secret: ${{ github.token }}
84+
- name: Add Conda scripts to GitHub path
85+
run: |
86+
echo "C:/Jenkins/Miniconda3/Scripts" >> $GITHUB_PATH
87+
- uses: ./test-infra/.github/actions/set-channel
88+
- uses: ./test-infra/.github/actions/setup-binary-builds
89+
with:
90+
repository: ${{ inputs.repository }}
91+
ref: ${{ inputs.ref }}
92+
setup-miniconda: false
93+
python-version: ${{ env.PYTHON_VERSION }}
94+
cuda-version: ${{ env.CU_VERSION }}
95+
arch: ${{ env.ARCH }}
96+
- name: Run Pre-Script with Caching
97+
if: ${{ inputs.pre-script != '' }}
98+
uses: ./test-infra/.github/actions/run-script-with-cache
99+
with:
100+
cache-path: ${{ inputs.cache-path }}
101+
cache-key: ${{ inputs.cache-key }}
102+
repository: ${{ inputs.repository }}
103+
script: ${{ inputs.pre-script }}
104+
is_windows: 'enabled'
105+
- name: Download artifacts
106+
uses: actions/download-artifact@v3
107+
with:
108+
name: ${{ env.ARTIFACT_NAME }}
109+
path: ${{ runner.temp }}/artifacts/
110+
- name: Pack script
111+
continue-on-error: ${{ inputs.continue-on-error }}
112+
working-directory: ${{ inputs.repository }}
113+
env:
114+
ALL_SECRETS: ${{ toJSON(secrets) }}
115+
run: |
116+
set -eou pipefail
117+
source "${BUILD_ENV_FILE}"
118+
{
119+
echo "${SCRIPT}";
120+
} > "user_script"
121+
cat .github/scripts/install-torch-tensorrt-windows.sh user_script > exec_script
122+
- name: Run script
123+
uses: ./test-infra/.github/actions/run-script-with-cache
124+
with:
125+
repository: ${{ inputs.repository }}
126+
script: exec_script
127+
is_windows: 'enabled'
128+
- name: Surface failing tests
129+
if: always()
130+
uses: pmeier/[email protected]
131+
with:
132+
path: ${{ env.RUNNER_TEST_RESULTS_DIR }}
133+
fail-on-empty: false
134+
- name: Teardown Windows
135+
if: ${{ always() }}
136+
uses: ./test-infra/.github/actions/teardown-windows
137+
138+
concurrency:
139+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}-${{ inputs.repository }}-${{ github.event_name == 'workflow_dispatch' }}-${{ inputs.job-name }}
140+
cancel-in-progress: true

0 commit comments

Comments
 (0)