Skip to content

Add SSD - TRTorch demo #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Aug 26, 2020
Merged

Add SSD - TRTorch demo #165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0b94908
Add a notebook with the example
narendasan May 27, 2020
478c42f
Add TRTorch-SSD300 demo notebook
shashank3959 Aug 4, 2020
a95df2e
Rename SSD demo notebook
shashank3959 Aug 5, 2020
ea10df5
Update notebooks/README
shashank3959 Aug 5, 2020
e07c121
Update index.rst to auto-render ssd demo
shashank3959 Aug 5, 2020
7c86ef0
Increase iterations in benchmark util
shashank3959 Aug 6, 2020
2e239f6
Fix module batch size for benchmark
shashank3959 Aug 7, 2020
ff211b1
Fix compile batch-size and script model benchmark
shashank3959 Aug 7, 2020
f2b521c
Display detection results before/after
shashank3959 Aug 13, 2020
ddef6f8
Fix minor typo
shashank3959 Aug 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docsrc/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Notebooks
:hidden:

_notebooks/lenet-getting-started
_notebooks/ssd-object-detection-demo


Python API Documenation
Expand Down
3 changes: 2 additions & 1 deletion notebooks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,5 @@ Within the container, the notebooks themselves are located at `/workspace/TRTorc

## 2. Notebook list

- [LeNet-example.ipynb](LeNet-example.ipynb): simple example on a LeNet network.
- [lenet-getting-started.ipynb](lenet-getting-started.ipynb): simple example on a LeNet network.
- [ssd-object-detection-demo.ipynb](ssd-object-detection-demo.ipynb): demo for compiling a pretrained SSD model using TRTorch.
179 changes: 88 additions & 91 deletions notebooks/lenet-getting-started.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -183,7 +183,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 4,
"metadata": {},
"outputs": [
{
Expand All @@ -202,7 +202,7 @@
")"
]
},
"execution_count": 10,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -214,28 +214,30 @@
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"execution_count": 5,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Warm up ...\n",
"Start timing ...\n",
"Iteration 1000/10000, ave batch time 0.93 ms\n",
"Iteration 2000/10000, ave batch time 0.93 ms\n",
"Iteration 3000/10000, ave batch time 0.93 ms\n",
"Iteration 4000/10000, ave batch time 0.93 ms\n",
"Iteration 5000/10000, ave batch time 0.93 ms\n",
"Iteration 6000/10000, ave batch time 0.93 ms\n",
"Iteration 7000/10000, ave batch time 0.93 ms\n",
"Iteration 8000/10000, ave batch time 0.93 ms\n",
"Iteration 9000/10000, ave batch time 0.93 ms\n",
"Iteration 10000/10000, ave batch time 0.93 ms\n",
"Iteration 1000/10000, ave batch time 0.58 ms\n",
"Iteration 2000/10000, ave batch time 0.58 ms\n",
"Iteration 3000/10000, ave batch time 0.58 ms\n",
"Iteration 4000/10000, ave batch time 0.58 ms\n",
"Iteration 5000/10000, ave batch time 0.58 ms\n",
"Iteration 6000/10000, ave batch time 0.58 ms\n",
"Iteration 7000/10000, ave batch time 0.58 ms\n",
"Iteration 8000/10000, ave batch time 0.58 ms\n",
"Iteration 9000/10000, ave batch time 0.58 ms\n",
"Iteration 10000/10000, ave batch time 0.58 ms\n",
"Input shape: torch.Size([1024, 1, 32, 32])\n",
"Output features size: torch.Size([1024, 10])\n",
"Average batch time: 0.93 ms\n"
"Average batch time: 0.58 ms\n"
]
}
],
Expand All @@ -256,7 +258,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 6,
"metadata": {},
"outputs": [
{
Expand All @@ -278,7 +280,7 @@
")"
]
},
"execution_count": 12,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -290,7 +292,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 7,
"metadata": {},
"outputs": [
{
Expand All @@ -299,19 +301,19 @@
"text": [
"Warm up ...\n",
"Start timing ...\n",
"Iteration 1000/10000, ave batch time 0.68 ms\n",
"Iteration 2000/10000, ave batch time 0.68 ms\n",
"Iteration 3000/10000, ave batch time 0.68 ms\n",
"Iteration 4000/10000, ave batch time 0.68 ms\n",
"Iteration 5000/10000, ave batch time 0.68 ms\n",
"Iteration 6000/10000, ave batch time 0.68 ms\n",
"Iteration 7000/10000, ave batch time 0.68 ms\n",
"Iteration 8000/10000, ave batch time 0.68 ms\n",
"Iteration 9000/10000, ave batch time 0.68 ms\n",
"Iteration 10000/10000, ave batch time 0.68 ms\n",
"Iteration 1000/10000, ave batch time 0.58 ms\n",
"Iteration 2000/10000, ave batch time 0.59 ms\n",
"Iteration 3000/10000, ave batch time 0.59 ms\n",
"Iteration 4000/10000, ave batch time 0.59 ms\n",
"Iteration 5000/10000, ave batch time 0.59 ms\n",
"Iteration 6000/10000, ave batch time 0.59 ms\n",
"Iteration 7000/10000, ave batch time 0.59 ms\n",
"Iteration 8000/10000, ave batch time 0.59 ms\n",
"Iteration 9000/10000, ave batch time 0.59 ms\n",
"Iteration 10000/10000, ave batch time 0.59 ms\n",
"Input shape: torch.Size([1024, 1, 32, 32])\n",
"Output features size: torch.Size([1024, 10])\n",
"Average batch time: 0.68 ms\n"
"Average batch time: 0.59 ms\n"
]
}
],
Expand All @@ -330,7 +332,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -340,7 +342,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 9,
"metadata": {},
"outputs": [
{
Expand All @@ -362,7 +364,7 @@
")"
]
},
"execution_count": 15,
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -373,7 +375,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 10,
"metadata": {},
"outputs": [
{
Expand All @@ -382,19 +384,19 @@
"text": [
"Warm up ...\n",
"Start timing ...\n",
"Iteration 1000/10000, ave batch time 0.68 ms\n",
"Iteration 2000/10000, ave batch time 0.68 ms\n",
"Iteration 3000/10000, ave batch time 0.68 ms\n",
"Iteration 4000/10000, ave batch time 0.68 ms\n",
"Iteration 5000/10000, ave batch time 0.68 ms\n",
"Iteration 6000/10000, ave batch time 0.68 ms\n",
"Iteration 7000/10000, ave batch time 0.68 ms\n",
"Iteration 8000/10000, ave batch time 0.68 ms\n",
"Iteration 9000/10000, ave batch time 0.68 ms\n",
"Iteration 10000/10000, ave batch time 0.68 ms\n",
"Iteration 1000/10000, ave batch time 0.59 ms\n",
"Iteration 2000/10000, ave batch time 0.59 ms\n",
"Iteration 3000/10000, ave batch time 0.59 ms\n",
"Iteration 4000/10000, ave batch time 0.59 ms\n",
"Iteration 5000/10000, ave batch time 0.59 ms\n",
"Iteration 6000/10000, ave batch time 0.59 ms\n",
"Iteration 7000/10000, ave batch time 0.59 ms\n",
"Iteration 8000/10000, ave batch time 0.59 ms\n",
"Iteration 9000/10000, ave batch time 0.59 ms\n",
"Iteration 10000/10000, ave batch time 0.59 ms\n",
"Input shape: torch.Size([1024, 1, 32, 32])\n",
"Output features size: torch.Size([1024, 10])\n",
"Average batch time: 0.68 ms\n"
"Average batch time: 0.59 ms\n"
]
}
],
Expand All @@ -421,26 +423,27 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"import trtorch\n",
"\n",
"# We use a batch-size of 1024, and half precision\n",
"compile_settings = {\n",
" \"input_shapes\": [\n",
" {\n",
" \"min\" : [1, 1, 32, 32],\n",
" \"opt\" : [1, 1, 33, 33],\n",
" \"max\" : [1, 1, 34, 34],\n",
" \"min\" : [1024, 1, 32, 32],\n",
" \"opt\" : [1024, 1, 33, 33],\n",
" \"max\" : [1024, 1, 34, 34],\n",
" }\n",
" ],\n",
" \"op_precision\": torch.half # Run with FP16\n",
"}\n",
"\n",
"trt_ts_module = trtorch.compile(traced_model, compile_settings)\n",
"\n",
"input_data = torch.randn((1, 1, 32, 32))\n",
"input_data = torch.randn((1024, 1, 32, 32))\n",
"input_data = input_data.half().to(\"cuda\")\n",
"\n",
"input_data = input_data.half()\n",
Expand All @@ -450,7 +453,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 12,
"metadata": {},
"outputs": [
{
Expand All @@ -459,24 +462,24 @@
"text": [
"Warm up ...\n",
"Start timing ...\n",
"Iteration 1000/10000, ave batch time 0.24 ms\n",
"Iteration 2000/10000, ave batch time 0.24 ms\n",
"Iteration 3000/10000, ave batch time 0.24 ms\n",
"Iteration 4000/10000, ave batch time 0.24 ms\n",
"Iteration 5000/10000, ave batch time 0.24 ms\n",
"Iteration 6000/10000, ave batch time 0.24 ms\n",
"Iteration 7000/10000, ave batch time 0.23 ms\n",
"Iteration 8000/10000, ave batch time 0.23 ms\n",
"Iteration 9000/10000, ave batch time 0.23 ms\n",
"Iteration 10000/10000, ave batch time 0.23 ms\n",
"Iteration 1000/10000, ave batch time 0.41 ms\n",
"Iteration 2000/10000, ave batch time 0.41 ms\n",
"Iteration 3000/10000, ave batch time 0.41 ms\n",
"Iteration 4000/10000, ave batch time 0.40 ms\n",
"Iteration 5000/10000, ave batch time 0.40 ms\n",
"Iteration 6000/10000, ave batch time 0.40 ms\n",
"Iteration 7000/10000, ave batch time 0.40 ms\n",
"Iteration 8000/10000, ave batch time 0.40 ms\n",
"Iteration 9000/10000, ave batch time 0.40 ms\n",
"Iteration 10000/10000, ave batch time 0.40 ms\n",
"Input shape: torch.Size([1024, 1, 32, 32])\n",
"Output features size: torch.Size([1, 10])\n",
"Average batch time: 0.23 ms\n"
"Output features size: torch.Size([1024, 10])\n",
"Average batch time: 0.40 ms\n"
]
}
],
"source": [
"benchmark(trt_ts_module, dtype=\"fp16\")"
"benchmark(trt_ts_module, input_shape=(1024, 1, 32, 32), dtype=\"fp16\")"
]
},
{
Expand All @@ -490,26 +493,27 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"import trtorch\n",
"\n",
"# We use a batch-size of 1024, and half precision\n",
"compile_settings = {\n",
" \"input_shapes\": [\n",
" {\n",
" \"min\" : [1, 1, 32, 32],\n",
" \"opt\" : [1, 1, 33, 33],\n",
" \"max\" : [1, 1, 34, 34],\n",
" \"min\" : [1024, 1, 32, 32],\n",
" \"opt\" : [1024, 1, 33, 33],\n",
" \"max\" : [1024, 1, 34, 34],\n",
" }\n",
" ],\n",
" \"op_precision\": torch.half # Run with FP16\n",
"}\n",
"\n",
"trt_script_module = trtorch.compile(script_model, compile_settings)\n",
"\n",
"input_data = torch.randn((1, 1, 32, 32))\n",
"input_data = torch.randn((1024, 1, 32, 32))\n",
"input_data = input_data.half().to(\"cuda\")\n",
"\n",
"input_data = input_data.half()\n",
Expand All @@ -519,7 +523,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 15,
"metadata": {},
"outputs": [
{
Expand All @@ -528,24 +532,24 @@
"text": [
"Warm up ...\n",
"Start timing ...\n",
"Iteration 1000/10000, ave batch time 0.20 ms\n",
"Iteration 2000/10000, ave batch time 0.20 ms\n",
"Iteration 3000/10000, ave batch time 0.20 ms\n",
"Iteration 4000/10000, ave batch time 0.21 ms\n",
"Iteration 5000/10000, ave batch time 0.21 ms\n",
"Iteration 6000/10000, ave batch time 0.21 ms\n",
"Iteration 7000/10000, ave batch time 0.21 ms\n",
"Iteration 8000/10000, ave batch time 0.21 ms\n",
"Iteration 9000/10000, ave batch time 0.21 ms\n",
"Iteration 10000/10000, ave batch time 0.21 ms\n",
"Iteration 1000/10000, ave batch time 0.40 ms\n",
"Iteration 2000/10000, ave batch time 0.40 ms\n",
"Iteration 3000/10000, ave batch time 0.40 ms\n",
"Iteration 4000/10000, ave batch time 0.40 ms\n",
"Iteration 5000/10000, ave batch time 0.40 ms\n",
"Iteration 6000/10000, ave batch time 0.40 ms\n",
"Iteration 7000/10000, ave batch time 0.40 ms\n",
"Iteration 8000/10000, ave batch time 0.40 ms\n",
"Iteration 9000/10000, ave batch time 0.40 ms\n",
"Iteration 10000/10000, ave batch time 0.40 ms\n",
"Input shape: torch.Size([1024, 1, 32, 32])\n",
"Output features size: torch.Size([1, 10])\n",
"Average batch time: 0.21 ms\n"
"Output features size: torch.Size([1024, 10])\n",
"Average batch time: 0.40 ms\n"
]
}
],
"source": [
"benchmark(trt_ts_module, dtype=\"fp16\")"
"benchmark(trt_script_module, input_shape=(1024, 1, 32, 32), dtype=\"fp16\")"
]
},
{
Expand All @@ -559,13 +563,6 @@
"### What's next\n",
"Now it's time to try TRTorch on your own model. Fill out issues at https://github.com/NVIDIA/TRTorch. Your involvement will help future development of TRTorch.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -589,4 +586,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}
Loading