(//bazel): Native compilation support for Jetson AGX platform #113

andi4191 · 2020-06-24T20:22:03Z

Description

It includes native compilation support for NVIDIA Jetson AGX platform.

Dependencies:
Cross-compiled libraries from PyTorch on Jetson. Refer https://ngc.nvidia.com/catalog/containers/nvidia:l4t-pytorch

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

Breaking change (x86_64 bazel system needs to adapt and co-exist with aarch64 support)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation and have regenerated the documentation (make html in docsrc)
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes

narendasan

We need to find a way to elegantly integrate these changes so that it doesn't wildly increase complexity or introduce confusion. We might be able to do some work with select in the BUILD file of the dependencies, this might get us around having 4 cases in every BUILD file. Also can you run all this through buildifier (https://github.com/bazelbuild/buildtools/blob/master/buildifier/README.md)? there are quite a few formatting issues

.bazelrc

BUILD

narendasan · 2020-06-24T20:45:21Z

README.md

-| Linux aarch64 / GPU | **Planned/Possible with Native Compiation but untested** |
-| Linux aarch64 / DLA | **Planned/Possible with Native Compilation but untested** |
+| Linux aarch64 / GPU | **Native Compilation Supported on JetPack-4.4** |
+| Linux aarch64 / DLA | **Native Compilation Supported on JetPack-4.4 but untested** |


Can you try resnet 50 on DLA with trtorchc?

WORKSPACE

cpp/trtorchc/BUILD

tests/core/converters/BUILD

third_party/libs/cuda.BUILD

third_party/libs/cudnn.BUILD

narendasan · 2020-06-24T21:06:05Z

Also we need to change the sphinx documentation to document the new usage for aarch64 (//docsrc/tutorials/installation.rst)

narendasan · 2020-06-26T18:21:09Z

Closes #36

tests/core/converters/converter_test.bzl

tests/modules/BUILD

tests/util/BUILD

…nd bilinear2d ops Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…d op Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…ilinear ops. removed redundant tests Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…late plugin Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…ompiles now. time to test it. Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Added support for interpolate plugin, used when align_corners=False and mode is linear Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…plugin, works for mode='linear' Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

… sized tensors Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…ut tensors Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected] Signed-off-by: Abhiram Iyer <[email protected]>

…r adaptive_pool2d Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

… support for aten::select.int Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

…ndex Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

Signed-off-by: anuragd <[email protected]>

Signed-off-by: Anurag Dixit <[email protected]>

cuBLAS into its own dependency should follow the same workflow as CUDA (i.e. local only) Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

@andi4191

The issue was in //third_party/libtorch/BUILD, there was numpy libraries being picked as headers. Also significantly simplifies the build system to only using the cpu flag for configuring precompiled third party dependecies and also removes the pre_cxx11_abi_aarch64 flag. Streamlines WORKSPACE, deduplicating repositories. Now the user workflow for aarch64 compilation is to configure the WORKSPACE to use all local sources. These should be able to be used the same way on x86_64 Also discovered that the NVIDIA PyTorch distribution for aarch64 uses the CXX11 abi so default paths for both will point to the default torch install location for local. TODO: Move from cpu to platforms (@andi4191) TODO: Test on both x86 and aarch64 + DLA (@andi4191) TODO: Before merge reset the WORKSPACE file to the default settings Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

Signed-off-by: andi4191 <[email protected]>

narendasan · 2020-07-07T19:14:53Z

@andi4191 can you rebase this on master the PR? is trying to recommit a bunch of old changes

narendasan · 2020-07-07T21:25:42Z

There were some issues with the PR, I filed one against your branch to resolve them. I think that we need to get this merged ASAP, it is currently blocking CUDA 11 work. I think if you rebase to remove those extra commits we should be good to merge this and then do documentation and DLA work in separate PRs.

andi4191 · 2020-07-08T02:45:05Z

Closing this PR. Updated PR tracked here #124

andi4191 requested a review from narendasan June 24, 2020 20:22

andi4191 self-assigned this Jun 24, 2020

andi4191 changed the title ~~Anuragd/aarch64 jetpack~~ WIP: (//bazel): Native compilation support for Jetson AGX platform Jun 24, 2020

narendasan requested changes Jun 24, 2020

View reviewed changes

narendasan marked this pull request as draft June 24, 2020 21:09

narendasan added this to the v0.1.0 milestone Jun 26, 2020

narendasan added the component: build system Issues re: Build system label Jun 26, 2020

narendasan requested changes Jul 1, 2020

View reviewed changes

tests/core/converters/converter_test.bzl Outdated Show resolved Hide resolved

tests/modules/BUILD Outdated Show resolved Hide resolved

tests/util/BUILD Outdated Show resolved Hide resolved

abhi-iyer added 20 commits July 2, 2020 14:29

feat(//core/conversion/converters/impl): added support for linear1d a…

e05ed17

…nd bilinear2d ops Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl): added support for trilinear3…

d67cecf

…d op Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(/tests/core/converters/): added 3 tests for linear, bilinear, tr…

41108c9

…ilinear ops. removed redundant tests Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl/plugins): template for interpo…

8bc0531

…late plugin Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl/plugins): interpolate plugin c…

a9ad817

…ompiles now. time to test it. Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

fix(): trying to resolve interpolate plugin problems

a072542

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

fix(plugin): trying to fix bug in plugin

a33d89b

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

fix(BUILD): modified BUILD

0f37912

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl):

b26aca7

Added support for interpolate plugin, used when align_corners=False and mode is linear Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(tests/core/converters): Added test for plugins

cebae40

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl/plugins): Created interpolate …

3e5c8c6

…plugin, works for mode='linear' Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(tests/util): added RunGraphEngineDynamic to handle dynamic input…

ef4389a

… sized tensors Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(): added adaptive_avg_pool2d plugin, and added test for it

2e853cc

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(): support for adaptive_avg_pool2d plugin

ccf649f

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(): addressed some PR comments, refactored code

ba821d5

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

fix(): RunGraphEngineDynamic fixed to work with dynamically sized inp…

029ea66

…ut tensors Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(): added interpolate model for engine serialization testing

e1bdb47

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(): Added engine serialization testing for interpolation model

2d9e44a

Signed-off-by: Abhiram Iyer <[email protected] Signed-off-by: Abhiram Iyer <[email protected]>

fix(): fixed interpolate_plugin to handle dynamically sized inputs fo…

92343d5

…r adaptive_pool2d Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

fix(): fixed FP16 bug, fixed README, addressed some other PR comments

5336cd2

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

narendasan and others added 11 commits July 2, 2020 14:29

docs: Remove known limitations section

5d497f9

Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

fix(): need to fix gather converter

05a6b65

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(aten::select.int): added test for aten::select.int

c9f567a

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(trt_util): from Naren, added unpadDims tool

0f25b86

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl): select converter, which adds…

335a006

… support for aten::select.int Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

refactor(): removed unecessary imports from aten::select.int converter

adc1d53

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(aten::stack): Added test for aten::stack

30909cc

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

feat(//core/conversion/converters/impl): added support for aten::stack

74fb809

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

tool(core/util/): added tool to unsqueeze dimensions at a specified i…

f2d2fdc

…ndex Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

test(aten::select): added test case for single call to select

d05d93d

Signed-off-by: Abhiram Iyer <[email protected]> Signed-off-by: Abhiram Iyer <[email protected]>

(//bazel): Added support for native compilation on NVIDIA Jetson AGX

3ae414e

Signed-off-by: anuragd <[email protected]>

narendasan linked an issue Jul 6, 2020 that may be closed by this pull request

Create a toolchain for aarch64 #36

Closed

Anurag Dixit and others added 5 commits July 6, 2020 12:13

Incorporate review comments

3747489

Signed-off-by: Anurag Dixit <[email protected]>

Update build system

28fc9ff

Signed-off-by: Anurag Dixit <[email protected]>

fix(//third_party/cublas): cuBLAS source was incorrect, breaking out

20415f0

cuBLAS into its own dependency should follow the same workflow as CUDA (i.e. local only) Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

fix(//cpp/trtorchexec): typo extra comma

c809eab

Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>

andi4191 force-pushed the anuragd/aarch64-jetpack branch from 5e2c081 to 199caa1 Compare July 6, 2020 20:01

andi4191 changed the title ~~WIP: (//bazel): Native compilation support for Jetson AGX platform~~ (//bazel): Native compilation support for Jetson AGX platform Jul 6, 2020

(//bazel): Incorporate review comments

813289f

Signed-off-by: andi4191 <[email protected]>

andi4191 force-pushed the anuragd/aarch64-jetpack branch from 199caa1 to 813289f Compare July 6, 2020 20:05

Merge branch 'master' into anuragd/aarch64-jetpack

98ec7f7

narendasan marked this pull request as ready for review July 6, 2020 21:50

andi4191 requested a review from narendasan July 7, 2020 01:33

andi4191 closed this Jul 8, 2020

andi4191 deleted the anuragd/aarch64-jetpack branch June 1, 2021 18:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

(//bazel): Native compilation support for Jetson AGX platform #113

(//bazel): Native compilation support for Jetson AGX platform #113

Uh oh!

andi4191 commented Jun 24, 2020 •

edited

Loading

Uh oh!

narendasan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan Jun 24, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan commented Jun 24, 2020

Uh oh!

narendasan commented Jun 26, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan commented Jul 7, 2020

Uh oh!

narendasan commented Jul 7, 2020

Uh oh!

andi4191 commented Jul 8, 2020

Uh oh!

Uh oh!

(//bazel): Native compilation support for Jetson AGX platform #113

(//bazel): Native compilation support for Jetson AGX platform #113

Uh oh!

Conversation

andi4191 commented Jun 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist:

Uh oh!

narendasan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan Jun 24, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan commented Jun 24, 2020

Uh oh!

narendasan commented Jun 26, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

narendasan commented Jul 7, 2020

Uh oh!

narendasan commented Jul 7, 2020

Uh oh!

andi4191 commented Jul 8, 2020

Uh oh!

Uh oh!

andi4191 commented Jun 24, 2020 •

edited

Loading