You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
inputs=[torch.randn((1, 3, 224, 224)).cuda()] # your inputs go here
75
75
76
-
# fix me
77
-
optimized_model = load_model
78
-
optimized_model(x)
76
+
# You can run this in a new python session!
77
+
model = torch.export.load("trt.ep").module()
78
+
# model = torch_tensorrt.load("trt.ep").module() # this also works
79
+
model(*inputs)
79
80
```
80
81
81
82
##### Deployment in C++:
@@ -87,7 +88,8 @@ optimized_model(x)
87
88
```
88
89
89
90
## Further resources
90
-
-[Optimize models from Hugging Face with Torch-TensorRT]()\[coming soon\]
91
+
-[Up to 50% faster Stable Diffusion inference with one line of code](https://pytorch.org/TensorRT/tutorials/_rendered_examples/dynamo/torch_compile_stable_diffusion.html#sphx-glr-tutorials-rendered-examples-dynamo-torch-compile-stable-diffusion-py)
92
+
-[Optimize LLMs from Hugging Face with Torch-TensorRT]()\[coming soon\]
91
93
-[Run your model in FP8 with Torch-TensorRT]()\[coming soon\]
92
94
-[Tools to resolve graph breaks and boost performance]()\[coming soon\]
0 commit comments