Skip to content

Various fixes to PIN integration example #368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 19, 2022
Merged

Various fixes to PIN integration example #368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
16bb2d1
Various fixes to PIN integration example
Oct 13, 2022
fc4bc47
Fix style errors
Oct 14, 2022
b3e3a8d
Fix f-strings
Oct 14, 2022
3416261
Tweak to README
Oct 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 10 additions & 12 deletions integrations/nuance_pin/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ with minimal code changes.

## Prerequisites

Before setting up and running the example MONAI spleen segmentation app to run as a Nuance PIN App, the user will need to install/download the following libraries.
Before setting up and running the example MONAI lung nodule detection app to run as a Nuance PIN App, the user will need to install/download the following libraries.
It is optional to use a GPU for the example app, however, it is recommended that a GPU is used for inference as it is very computationally intensive.

Minimum software requirements:
Expand All @@ -30,9 +30,8 @@ cd integrations/nuance_pin
In this folder you will see the following directory structure
```bash
nuance_pin
├── app # directory with MONAI app code
├── lib # directory where we will place Nuance PIN wheels
├── model # directory where we will place the model used by our MONAI app
├── app/ # directory with MONAI app code ├── lib/ # you should create this directory where we will place Nuance PIN wheels
├── model/ # directory where we will place the model used by our MONAI app
├── app_wrapper.py # Nuance PIN wrapper code
├── docker-compose.yml # docker compose runtime script
├── Dockerfile # container image build script
Expand All @@ -48,7 +47,7 @@ To download the test data you may follow the instructions in the [Lund Nodule De

### Download Nuance PIN SDK

Place the Nuance PIN `ai_service` wheel in the `nuance_pin/lib` folder. This can be obtained in the link provided in step 3 of of the [prerequisites](#prerequisites).
Place the Nuance PIN `ai_service` wheel in the `nuance_pin/lib` folder. This can be obtained in the link provided in step 4 of of the [prerequisites](#prerequisites).

### Running the Example App in the Container

Expand All @@ -57,7 +56,7 @@ Now we are ready to build and start the container that runs our MONAI app as a N
docker-compose up --build
```

If the build is successful the a service will start on `localhost:5000`. We can verify the service is running
If the build is successful the service will start on `localhost:5000`. We can verify the service is running
by issuing a "live" request such as
```bash
curl -v http://localhost:5000/aiservice/2/live && echo ""
Expand Down Expand Up @@ -136,12 +135,12 @@ python -m AiSvcTest -i ~/Downloads/dcm -o ~/Downloads/dcm/out -s http://localhos

### Bring Your Own MONAI App

This example integration may be modified to fit any existing MONAI app, however, there may be caveats.
This example integration may be modified to fit any existing MONAI app by tailoring the files within the `app/` directory, however, there may be caveats.

Nuance PIN requires all artifacts present in the output folder to be also added into the `resultManifest.json` output file
to consider the run successful. To see what this means in practical terms, check the `resultManifest.json` output from the
example app we ran the in previous sections. You will notice an entry in `resultManifest.json` that corresponds to the DICOM
SEG output generated by the underlying MONAI app
GSPS output generated by the underlying MONAI app
```json
"study": {
"uid": "1.2.826.0.1.3680043.2.1125.1.67295333199898911264201812221946213",
Expand All @@ -153,17 +152,16 @@ SEG output generated by the underlying MONAI app
{
"documentType": "application/dicom",
"groupCode": "default",
"name": "dicom_seg-DICOMSEG.dcm",
"name": "gsps.dcm",
"trackingUids": []
}
]
}
]
},
```
This entry is generated by `app_wrapper.py`, which takes care of adding any DICOM present in the output folder in the `resultManifest.json`
to ensure that existing MONAI apps complete successfully when deployed in Nuance. In general, however, the developer may need to tailor some
of the code in `app_wrapper.py` to provide more insight to Nuance's network, such as adding findings, conclusions, etc. and generating more insight
This entry is generated automatically by Nuance's `ai_service` library as a result of uploading the DICOM GSPS object in `app/post_inference_ops.py`.
In general, however, the developer may need to tailor some of the code in `app_wrapper.py` to provide more insight to Nuance's network, such as adding findings, conclusions, etc. and generating more insight
using SNOMED codes. All of this is handled within the Nuance PIN SDK libraries - for more information please consult Nuance PIN [documentation](https://www.nuance.com/healthcare/diagnostics-solutions/precision-imaging-network.html).

In simpler cases, the developer will need to place their code and model under `nuance_pin`. Placing the model under `model` is optional as the model may be placed
Expand Down
2 changes: 1 addition & 1 deletion integrations/nuance_pin/app/inference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ def pre_process(self, img_reader) -> Compose:
keys=[image_key, f"{image_key}_meta_dict"],
names=[orig_image_key, f"{orig_image_key}_meta_dict"],
),
ToDeviced(keys=image_key, device="cuda"),
ToDeviced(keys=image_key, device=self.device),
EnsureChannelFirstd(keys=image_key),
Spacingd(keys=image_key, pixdim=(0.703125, 0.703125, 1.25)),
Orientationd(
Expand Down
90 changes: 51 additions & 39 deletions integrations/nuance_pin/app/post_inference_ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,23 @@ def compute(self, op_input: InputContext, op_output: OutputContext, context: Exe
series_uid = hd.UID()
series_number = randint(1, 100000)

# One graphic layer to contain all detections
layer = hd.pr.GraphicLayer(
layer_name="LUNG_NODULES",
order=1,
description="Lung Nodule Detections",
)

annotations = []

all_ref_images = [ins.get_native_sop_instance() for ins in selected_series.series.get_sop_instances()]
accession = all_ref_images[0].AccessionNumber

for inst_num, (box_data, box_score) in enumerate(zip(detection_result.box_data, detection_result.score_data)):

tracking_id = f"{accession}_nodule_{inst_num}" # site-specific ID
tracking_uid = hd.UID()

polyline = hd.pr.GraphicObject(
graphic_type=hd.pr.GraphicTypeValues.POLYLINE,
graphic_data=np.array(
Expand All @@ -60,8 +75,8 @@ def compute(self, op_input: InputContext, op_output: OutputContext, context: Exe
]
), # coordinates of polyline vertices
units=hd.pr.AnnotationUnitsValues.PIXEL, # units for graphic data
tracking_id="lung_nodule_MONAI", # site-specific ID
tracking_uid=hd.UID(), # highdicom will generate a unique ID
tracking_id=tracking_id,
tracking_uid=tracking_uid,
)

self.logger.info(f"Box: {[box_data[0], box_data[1], box_data[3], box_data[4]]}")
Expand All @@ -70,14 +85,8 @@ def compute(self, op_input: InputContext, op_output: OutputContext, context: Exe
text_value=f"{box_score:.2f}",
bounding_box=(box_data[0], box_data[1], box_data[3], box_data[4]), # left, top, right, bottom
units=hd.pr.AnnotationUnitsValues.PIXEL, # units for bounding box
tracking_id="LungNoduleMONAI", # site-specific ID
tracking_uid=hd.UID(), # highdicom will generate a unique ID
)

layer = hd.pr.GraphicLayer(
layer_name="LUNG_NODULE",
order=1,
description="Lung Nodule Detection",
tracking_id=tracking_id,
tracking_uid=tracking_uid,
)

affected_slice_idx = [
Expand All @@ -103,37 +112,40 @@ def compute(self, op_input: InputContext, op_output: OutputContext, context: Exe
graphic_objects=[polyline],
)

# Assemble the components into a GSPS object
gsps = hd.pr.GrayscaleSoftcopyPresentationState(
referenced_images=ref_images,
series_instance_uid=series_uid,
series_number=series_number,
sop_instance_uid=hd.UID(),
instance_number=inst_num + 1,
manufacturer="MONAI",
manufacturer_model_name="lung_nodule_ct_detection",
software_versions="v0.2.0",
device_serial_number="",
content_label="ANNOTATIONS",
graphic_layers=[layer],
graphic_annotations=[annotation],
institution_name="MONAI",
institutional_department_name="Deploy",
voi_lut_transformations=[
hd.pr.SoftcopyVOILUTTransformation(
window_center=-550.0,
window_width=1350.0,
)
],
)
annotations.append(annotation)

# Assemble the components into a GSPS object
gsps = hd.pr.GrayscaleSoftcopyPresentationState(
referenced_images=all_ref_images,
series_instance_uid=series_uid,
series_number=series_number,
sop_instance_uid=hd.UID(),
instance_number=1,
manufacturer="MONAI",
manufacturer_model_name="lung_nodule_ct_detection",
software_versions="v0.2.0",
device_serial_number="",
content_label="ANNOTATIONS",
graphic_layers=[layer],
graphic_annotations=annotations,
institution_name="MONAI",
institutional_department_name="Deploy",
voi_lut_transformations=[
hd.pr.SoftcopyVOILUTTransformation(
window_center=-550.0,
window_width=1350.0,
)
],
)

gsps.save_as(os.path.join(output_path, f"gsps-{inst_num:04d}.dcm"))
gsps_filename = os.path.join(output_path, "gsps.dcm")
gsps.save_as(gsps_filename)

self.upload_gsps(
file=os.path.join(output_path, f"gsps-{inst_num:04d}.dcm"),
document_detail="MONAI Lung Nodule Detection v0.2.0",
series_uid=series_uid,
)
self.upload_gsps(
file=gsps_filename,
document_detail="MONAI Lung Nodule Detection v0.2.0",
series_uid=series_uid,
)


@md.input("original_dicom", List[StudySelectedSeries], IOType.IN_MEMORY)
Expand Down
4 changes: 1 addition & 3 deletions integrations/nuance_pin/app_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,7 @@ def initialize_class(cls):
monai_app_class_module = cls.monai_app_module.rsplit(".", 1)[0]
monai_app_class_name = cls.monai_app_module.rsplit(".", 1)[1]
if not cls.monai_app_module:
raise ValueError(
"MONAI App to be run has not been specificed in `MONAI_APP_CLASSPATH` environment variable"
)
raise ValueError("MONAI App to be run has not been specified in `MONAI_APP_CLASSPATH` environment variable")

monai_app_class = getattr(import_module(monai_app_class_module), monai_app_class_name)
if monai_app_class is None:
Expand Down