Combination: 16 commits with aten improvements

refactor: Moving elementwise and unary core to impl

Signed-off-by: Naren Dasan <[email protected]>

	new file:   ../converters/impl/unary/base.py

Moving elementwise core to impl - rsqrt (FX Converter Refactor [9/N]) <Target: converter_reorg_elementwise> (#1905)

Converter reorg fmod

Converter reorg and rsub

Rsub error fixes and linting error fixed

Rsub test case to include different inputs

Converter reorg batch norm

batch norm error fix and linting issue error fix

layer_norm converter

Layer norm linting correction

ops file correction

fixing lint

Acc_ops layer_norm correction

Converter reorg and softmax operation

softmax linting error fix

Converter reorg and gelu

Linting error

Converter reorg and squeeze operator

Correcting squeeze operator implementation, linting error and acc squeeze test

Adding the condition to convert dim to int and removing the comment

Converter reorg and select operation

select operation correction and linting changes

converter reorg and slice

converter reorg slice op

Correcting linting error and slice changes

Correcting the slice operation

converter reorg and matmul

Matmul issue fixes and lint error check

moving matmul to individual file

Converter reorg and where operator

adding where aten op

aten::where correction and linting error changes

aten::unsqueeze impl refactor

Signed-off-by: Boris Fomitchev <[email protected]>

Moved clamp to impl

Signed-off-by: Boris Fomitchev <[email protected]>

fixed method name

Signed-off-by: Boris Fomitchev <[email protected]>

fix: Add automatic type promotion for FX ops

- Implement functionality to cast tensors to alternative types
- Add functionality to elementwise ops to promote types and perform
necessary casts
- Address issues in FX ops where mixed-precision computations can
cause errors
- Add test cases to validate fix

Revert all changes to py/torch_tensorrt/fx

Revert "fix: Add automatic type promotion for FX ops"

This reverts commit f1f3716.

Revert "Moved clamp to impl"

This reverts commit df401dd.

Revert "aten::unsqueeze impl refactor"

This reverts commit b424735.

Revert "Converter reorg and where operator"

This reverts commit b4da15e.

Revert "converter reorg and matmul"

This reverts commit 7551eee.

Revert "converter reorg and slice"

This reverts commit 9bbdc9e.

Revert "Converter reorg and select operation"

This reverts commit fb70253.

Revert "Converter reorg and squeeze operator"

This reverts commit 294545c.

Revert "Converter reorg and gelu"

This reverts commit 37d1168.

Revert "Converter reorg and softmax operation"

This reverts commit 1ba6d13.

Revert "layer_norm converter"

This reverts commit e0b34b1.

Revert "Converter reorg batch norm"

This reverts commit 59354e5.

Revert "Converter reorg and rsub"

This reverts commit db15d27.

Revert "Converter reorg fmod"

This reverts commit ce3fa67.

Revert "Moving elementwise core to impl - rsqrt (FX Converter Refactor [9/N]) <Target: converter_reorg_elementwise> (#1905)"

This reverts commit 7158ca5.

Revert "refactor: Moving elementwise and unary core to impl"

This reverts commit 45e43ca.

fix: Replay all FX changes in Dynamo

- Add multiple fixes to make FX changes appear in Dynamo directory,
using Dynamo registry
- All converters with open PRs are linked and shown
- Update references, imports, code, merges, rebases accordingly
- Add new test cases to Dynamo for converters

Temporarily removing rsub pending fix

Fixing clamp to not use Torch

Signed-off-by: Boris Fomitchev <[email protected]>

Fixing select to not use torch

fix: Reorganize folders in latest implementation

- Update test references and imports accordingly

Embedding operator in dynamo

reciprocal lowering pass

fix: Fix for Dynamic Shape Tests + Input class

feat: Add permute operation implementation

chore: Move converter registry, update imports

Author: narendasan

.circleci/config.yml

@@ -780,6 +780,21 @@ commands:
       - store_artifacts:
           path: /tmp/testlogs
 
+  test-dynamo-converters:
+    description: "Test the Dynamo aten converters"
+    steps:
+      - run:
+          name: Run Dynamo converter tests
+          command: |
+            cd tests/py/dynamo/converters
+            TESTS_TO_RUN=$(circleci tests glob "test_*.py" | circleci tests split --split-by=timings)
+            pytest --junitxml=/tmp/artifacts/test_results/dynamo/converters/test_results.xml $TESTS_TO_RUN
+
+      - store_test_results:
+          path: /tmp/artifacts
+      - store_artifacts:
+          path: /tmp/testlogs
+
 # =================== Dynamo tests end ======================== #
 
 # Define a job to be invoked later in a workflow.
@@ -1036,6 +1051,7 @@ jobs:
           command: pip3 install --pre /tmp/dist/x86_64-linux/*cp39-cp39*.whl
       # We install torch after torch-trt because pip automatically enforces the version constraint otherwise
       - dump-test-env
+      - test-dynamo-converters
       - test-dynamo-torch_compile
       - test-dynamo-models_torch_compile
       - test-dynamo-models_torch_export

py/torch_tensorrt/_Input.py

@@ -68,6 +68,17 @@ def __init__(self, *args, **kwargs):
             - Input(shape=(1,3,32,32), dtype=torch_tensorrt.dtype.int32, format=torch_tensorrt.TensorFormat.NCHW)
             - Input(min_shape=(1,3,32,32), opt_shape=[2,3,32,32], max_shape=(3,3,32,32)) #Implicitly dtype=torch_tensorrt.dtype.float32, format=torch_tensorrt.TensorFormat.NCHW
         """
+        # Compatibility code for switching over from InputTensorSpec
+        if "shape" in kwargs and "shape_ranges" in kwargs:
+            assert (
+                len(kwargs["shape_ranges"]) == 1 and len(kwargs["shape_ranges"][0]) == 3
+            )
+            del kwargs["shape"]
+
+            kwargs["min_shape"] = kwargs["shape_ranges"][0][0]
+            kwargs["opt_shape"] = kwargs["shape_ranges"][0][1]
+            kwargs["max_shape"] = kwargs["shape_ranges"][0][2]
+
         if len(args) == 1:
             if not Input._supported_input_size_type(args[0]):
                 raise TypeError(

py/torch_tensorrt/dynamo/__init__.py

@@ -3,8 +3,9 @@
 
 if version.parse(sanitized_torch_version()) >= version.parse("2.1.dev"):
     from ._settings import *
+    from .conversion import *
     from .aten_tracer import trace
-    from .converter_registry import (
+    from .conversion.converter_registry import (
         DYNAMO_CONVERTERS,
         dynamo_tensorrt_converter,
     )

py/torch_tensorrt/dynamo/conversion/SourceIR.py

@@ -0,0 +1,24 @@
+from enum import Enum, auto
+
+
+class SourceIR(Enum):
+    NN = auto()
+    ACC = auto()
+    ATEN = auto()
+    PRIM = auto()
+    TORCHTRT_LOWERED = auto()
+    UNKNOWN = auto()
+
+    def __str__(self):
+        if self == SourceIR.NN:
+            return "nn"
+        elif self == SourceIR.ACC:
+            return "acc"
+        elif self == SourceIR.ATEN:
+            return "aten"
+        elif self == SourceIR.PRIM:
+            return "prim"
+        elif self == SourceIR.TORCHTRT_LOWERED:
+            return "torchtrt_lowered"
+        else:
+            return "unknown_ir"

py/torch_tensorrt/dynamo/conversion/__init__.py

@@ -1,3 +1,5 @@
+from .SourceIR import SourceIR
+from .aten_ops_converters import *
 from .trt_interpreter import *
 from .conversion import *
 from .truncate_long_and_double import repair_long_or_double_inputs