Skip to content

Commit 48da44d

Browse files
dongyang0122dongyDong Yangpre-commit-ci[bot]
authored
Add a visualization tutorial for 3D detection boxes (#766)
* init vis folder Signed-off-by: dongy <[email protected]> * update repo Signed-off-by: Dong Yang <[email protected]> * update repo Signed-off-by: Dong Yang <[email protected]> * update repo Signed-off-by: Dong Yang <[email protected]> * update repo Signed-off-by: Dong Yang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: dongy <[email protected]> Co-authored-by: Dong Yang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 4d4c4a6 commit 48da44d

File tree

8 files changed

+308
-0
lines changed

8 files changed

+308
-0
lines changed

‎detection/luna16_visualization/README.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Tutorial to visualize 3D (detection) boxes using 3D Slicer
2+
3+
Visualizing box prediction/annotation in 3D medical image detection is not straightforward. To better understand model predictions, we provide a way to further visualize detection results/ground truth using the free and open source software [3D Slicer](https://www.slicer.org/). 3D Slicer enables visualization of 3D images, 3D segmentation masks, and 3D meshes with 3D orthographic views. And we convert 3D box annotations/predictions into meshes that can be visualized by cross-sectional curves in three 2D planes.
4+
5+
## Prerequisite
6+
7+
- The Version of 3D Slicer should be **4.11.20210226** or later.
8+
- Box information should be stored in a ".json" file. The "data\_sample.json" file is an example. The information of *N* boxes is stored under the key "box" as *N* lists. The six values are the *X*/*Y*/*Z* coordinates of the box center and the box length in the *X*/*Y*/*Z* axes. All the coordinate values are in the world coordinate system.
9+
10+
```
11+
"box": [
12+
[
13+
-100.312255859375,
14+
67.529541015625,
15+
-231.97265625,
16+
6.328125,
17+
6.328125,
18+
6.4453125
19+
],
20+
[
21+
-19.013427734375,
22+
10.576416015625,
23+
-48.28125,
24+
5.09765625,
25+
5.09765625,
26+
5.3125
27+
]
28+
]
29+
```
30+
31+
## Steps
32+
33+
### 1. Create ".obj" file for predictions/annotation using the "save\_obj.sh" script.
34+
35+
```
36+
#!/bin/bash
37+
38+
INPUT_DATASET_JSON="./data_sample.json"
39+
OUTPUT_DIR="./out"
40+
41+
python save_obj.py --input_dataset_json ${INPUT_DATASET_JSON} \
42+
--output_dir ${OUTPUT_DIR}
43+
```
44+
45+
### 2. Load the original 3D image and the resulting ".obj" file into 3D Slicer.
46+
47+
Image is visualized in 3 2D planes, while boxes are visualized in 3D space.
48+
49+
![Step 2](figures/step-1.png)
50+
51+
### 3. Opens the "Data" tab for box property editing.
52+
53+
![Step 3](figures/step-2.png)
54+
55+
### 4. Change the box color by double-clicking the color tab (for better visualization).
56+
57+
![Step 3](figures/step-3.png)
58+
59+
### 5. Enable in-plane visibility of the box by selecting the "Visible" option.
60+
61+
The following example shows predicted boxed (red) and ground truth boxes (green).
62+
63+
![Step 4](figures/step-4.png)
64+
65+
After completing these steps, the user can analyze the boxes using 3D Slicer's basic visualization capabilities (e.g., zoom in, zoom out).

‎detection/luna16_visualization/data_sample.json

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
{
2+
"validation": [
3+
{
4+
"label": [
5+
0,
6+
0,
7+
0,
8+
0
9+
],
10+
"box": [
11+
[
12+
-100.312255859375,
13+
67.529541015625,
14+
-231.97265625,
15+
6.328125,
16+
6.328125,
17+
6.4453125
18+
],
19+
[
20+
-19.013427734375,
21+
10.576416015625,
22+
-48.28125,
23+
5.09765625,
24+
5.09765625,
25+
5.3125
26+
],
27+
[
28+
-53.027099609375,
29+
-79.863037109375,
30+
-221.640625,
31+
5.2734375,
32+
5.2734375,
33+
5.3125
34+
],
35+
[
36+
-19.980224609375,
37+
9.785400390625,
38+
-45.3125,
39+
7.03125,
40+
6.6796875,
41+
6.875
42+
]
43+
],
44+
"score": [
45+
0.99658203125,
46+
0.50146484375,
47+
0.05975341796875,
48+
0.0325927734375
49+
],
50+
"image": "/datasets/1.3.6.1.4.1.14519.5.2.1.6279.6001.108197895896446896160048741492/1.3.6.1.4.1.14519.5.2.1.6279.6001.108197895896446896160048741492.nii.gz"
51+
},
52+
{
53+
"label": [
54+
0,
55+
0,
56+
0,
57+
0,
58+
0,
59+
0,
60+
0
61+
],
62+
"box": [
63+
[
64+
46.05921936035156,
65+
48.53294372558594,
66+
-108.6875,
67+
13.5791015625,
68+
13.623046875,
69+
13.75
70+
],
71+
[
72+
36.47914123535156,
73+
76.83372497558594,
74+
-123.3359375,
75+
4.5263671875,
76+
4.482421875,
77+
4.375
78+
],
79+
[
80+
37.86341857910156,
81+
0.5886077880859375,
82+
-106.265625,
83+
4.21875,
84+
4.21875,
85+
3.90625
86+
],
87+
[
88+
20.351211547851562,
89+
52.53196716308594,
90+
-28.765625,
91+
4.7900390625,
92+
4.74609375,
93+
4.84375
94+
],
95+
[
96+
-102.71763610839844,
97+
0.7643890380859375,
98+
-36.9296875,
99+
6.328125,
100+
6.328125,
101+
6.484375
102+
],
103+
[
104+
-114.14341735839844,
105+
42.86399841308594,
106+
-116.83203125,
107+
4.5703125,
108+
4.5703125,
109+
4.3359375
110+
],
111+
[
112+
-95.59849548339844,
113+
39.69993591308594,
114+
-50.171875,
115+
4.74609375,
116+
4.921875,
117+
4.84375
118+
]
119+
],
120+
"score": [
121+
0.9990234375,
122+
0.92578125,
123+
0.89208984375,
124+
0.8876953125,
125+
0.68994140625,
126+
0.08538818359375,
127+
0.023101806640625
128+
],
129+
"image": "/datasets/1.3.6.1.4.1.14519.5.2.1.6279.6001.109002525524522225658609808059/1.3.6.1.4.1.14519.5.2.1.6279.6001.109002525524522225658609808059.nii.gz"
130+
}
131+
]
132+
}

‎detection/luna16_visualization/figures/step-1.png

228 KB
Loading

‎detection/luna16_visualization/figures/step-2.png

315 KB
Loading

‎detection/luna16_visualization/figures/step-3.png

302 KB
Loading

‎detection/luna16_visualization/figures/step-4.png

298 KB
Loading

‎detection/luna16_visualization/save_obj.py

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2021 - 2022 MONAI Consortium
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
import argparse
15+
import csv
16+
import json
17+
import nibabel as nib
18+
import numpy as np
19+
import os
20+
21+
22+
def save_obj(vertices, faces, filename):
23+
24+
with open(filename, "w") as f:
25+
26+
for v in vertices:
27+
f.write("v {} {} {}\n".format(*np.array(v)))
28+
29+
for t in faces:
30+
f.write("f {} {} {} {}\n".format(*(np.array(t) + 1)))
31+
32+
33+
def main():
34+
parser = argparse.ArgumentParser()
35+
parser.add_argument(
36+
"--input_dataset_json",
37+
action="store",
38+
type=str,
39+
required=True,
40+
)
41+
parser.add_argument(
42+
"--output_dir",
43+
action="store",
44+
type=str,
45+
required=True,
46+
)
47+
args = parser.parse_args()
48+
49+
if not os.path.exists(args.output_dir):
50+
os.makedirs(args.output_dir)
51+
52+
with open(os.path.join(args.input_dataset_json)) as f:
53+
input_dataset = json.load(f)
54+
55+
for key in input_dataset.keys():
56+
section = input_dataset[key]
57+
58+
for _k in range(len(section)):
59+
box_data = section[_k]["box"]
60+
box_filename = section[_k]["image"]
61+
box_filename = box_filename.split(os.sep)[-1]
62+
print("-- {0:d}th case name:".format(_k + 1), box_filename)
63+
64+
vertices = []
65+
faces = []
66+
_i = 0
67+
for vec in box_data:
68+
xmin = vec[0] - 0.5 * vec[3]
69+
ymin = vec[1] - 0.5 * vec[4]
70+
zmin = vec[2] - 0.5 * vec[5]
71+
72+
xmax = vec[0] + 0.5 * vec[3]
73+
ymax = vec[1] + 0.5 * vec[4]
74+
zmax = vec[2] + 0.5 * vec[5]
75+
76+
vertices += [
77+
(xmax, ymax, zmin),
78+
(xmax, ymin, zmin),
79+
(xmin, ymin, zmin),
80+
(xmin, ymax, zmin),
81+
(xmax, ymax, zmax),
82+
(xmax, ymin, zmax),
83+
(xmin, ymin, zmax),
84+
(xmin, ymax, zmax),
85+
]
86+
87+
faces += [
88+
(0 + 8 * _i, 1 + 8 * _i, 2 + 8 * _i, 3 + 8 * _i),
89+
(4 + 8 * _i, 7 + 8 * _i, 6 + 8 * _i, 5 + 8 * _i),
90+
(0 + 8 * _i, 4 + 8 * _i, 5 + 8 * _i, 1 + 8 * _i),
91+
(1 + 8 * _i, 5 + 8 * _i, 6 + 8 * _i, 2 + 8 * _i),
92+
(2 + 8 * _i, 6 + 8 * _i, 7 + 8 * _i, 3 + 8 * _i),
93+
(4 + 8 * _i, 0 + 8 * _i, 3 + 8 * _i, 7 + 8 * _i),
94+
]
95+
96+
_i += 1
97+
98+
save_obj(vertices, faces, os.path.join(args.output_dir, box_filename + ".obj"))
99+
100+
return
101+
102+
103+
if __name__ == "__main__":
104+
main()

‎detection/luna16_visualization/save_obj.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/bash
2+
3+
INPUT_DATASET_JSON="./data_sample.json"
4+
OUTPUT_DIR="./out"
5+
6+
python save_obj.py --input_dataset_json ${INPUT_DATASET_JSON} \
7+
--output_dir ${OUTPUT_DIR}

0 commit comments

Comments
 (0)