677 Add hybrid programming example for bundle (Project-MONAI#678)

* [DLMED] add hybrid placeholders

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update doc

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add scripts

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] split to 2 examples

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] rename to bundle

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update hybrid programming

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update bundles to bundle

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] add README

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <[email protected]>

* [DLMED] fix format

Signed-off-by: Nic Ma <[email protected]>

Author: Nic-Ma

README.md

@@ -178,7 +178,7 @@ Demonstrates the use of the `ThreadBuffer` class used to generate data batches d
 Illustrate reading NIfTI files and test speed of different transforms on different devices.
 
 **modules**
-#### [engines](./modules/bundles)
+#### [bundle](./modules/bundle)
 Get started tutorial and concrete training / inference examples for MONAI bundle features.
 #### [engines](./modules/engines)
 Training and evaluation examples of 3D segmentation based on UNet3D and synthetic dataset with MONAI workflows, which contains engines, event-handlers, and post-transforms. And GAN training and evaluation example for a medical image generative adversarial network. Easy run training script uses `GanTrainer` to train a 2D CT scan reconstruction network. Evaluation script generates random samples from a trained network.

modules/bundle/README.md

@@ -0,0 +1,9 @@
+# MONAI bundle
+This folder contains the get_started tutorial and concrete training / inference examples for MONAI bundle features.
+
+### [spleen segmentation](./spleen_segmentation)
+A bundle example for volumetric (3D) segmentation of the spleen from CT image.
+### [customize component](./custom_component)
+Example shows the use cases of bringing customized python components, such as transform, network, and metrics, in a configuration-based workflow.
+### [hybrid programming](./hybrid_programming)
+Example shows how to parse the config files in your own python program, instantiate necessary components with python program and execute the inference.

modules/bundle/custom_component/README.md

@@ -0,0 +1,16 @@
+# Description
+This example mainly shows a typical use case that brings customized python components (such as transform, network, metrics) in a configuration-based workflow.
+
+Please note that this example depends on the `spleen_segmentation` bundle example and executes via overriding the config file of it.
+
+## commands example
+To run the workflow with customized components, `PYTHONPATH` should be revised to include the path to the customized component:
+```
+export PYTHONPATH=$PYTHONPATH:"<path to 'custom_component/scripts'>"
+```
+And please make sure the folder `custom_component/scripts` is a valid python module (it has a `__init__.py` file in the folder).
+
+Override the `train` config with the customized `transform` and execute training:
+```
+python -m monai.bundle run training --meta_file <spleen_configs_path>/metadata.json --config_file "['<spleen_configs_path>/train.json','configs/custom_train.json']" --logging_file <spleen_configs_path>/logging.conf
+```

modules/bundle/custom_component/configs/custom_train.json

@@ -0,0 +1,7 @@
+{
+    "train#preprocessing#transforms#6":
+    {
+        "_target_": "scripts.custom_transforms.PrintEnsureTyped",
+        "keys": ["image", "label"]
+    }
+}

modules/bundle/custom_component/scripts/__init__.py

@@ -0,0 +1,10 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

modules/bundle/custom_component/scripts/custom_transforms.py

@@ -0,0 +1,32 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from monai.config import KeysCollection
+from monai.transforms import EnsureTyped
+
+
+class PrintEnsureTyped(EnsureTyped):
+    """
+    Extend the `EnsureTyped` transform to print the image shape.
+
+     Args:
+        keys: keys of the corresponding items to be transformed.
+
+    """
+
+    def __init__(self, keys: KeysCollection, data_type: str = "tensor") -> None:
+        super().__init__(keys, data_type=data_type)
+
+    def __call__(self, data):
+        d = dict(super().__call__(data=data))
+        for key in self.key_iterator(d):
+            print(f"data shape of {key}: {d[key].shape}")
+        return d

modules/bundles/get_started.ipynb renamed to modules/bundle/get_started.ipynb

@@ -8,7 +8,7 @@
     "\n",
     "A MONAI bundle usually includes the stored weights of a model, TorchScript model, JSON files which include configs and metadata about the model, information for constructing training, inference, and post-processing transform sequences, plain-text description, legal information, and other data the model creator wishes to include.\n",
     "\n",
-    "For more information about MONAI bundles read the description: https://docs.monai.io/en/latest/bundle_intro.html.\n",
+    "For more information about MONAI bundle read the description: https://docs.monai.io/en/latest/bundle_intro.html.\n",
     "\n",
     "This notebook is a step-by-step tutorial to help get started to develop a bundle package, which contains a config file to construct the training pipeline and also has a `metadata.json` file to define the metadata information.\n",
     "\n",
@@ -26,7 +26,7 @@
     "- Override config content at runtime.\n",
     "- Hybrid programming with config and python code.\n",
     "\n",
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Project-MONAI/tutorials/blob/master/modules/bundles/get_started.ipynb)"
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Project-MONAI/tutorials/blob/master/modules/bundle/get_started.ipynb)"
    ]
   },
   {
@@ -143,7 +143,7 @@
    "source": [
     "## Define train config - Set imports and input / output environments\n",
     "\n",
-    "Now let's start to define the config file for a regular training task. MONAI bundles support both `JSON` and `YAML` format, here we use `JSON` as the example.\n",
+    "Now let's start to define the config file for a regular training task. MONAI bundle support both `JSON` and `YAML` format, here we use `JSON` as the example.\n",
     "\n",
     "According to the predefined syntax of MONAI bundle, `$` indicates an expression to evaluate, `@` refers to another object in the config content. For more details about the syntax in bundle config, please check: https://docs.monai.io/en/latest/config_syntax.html.\n",
     "\n",
@@ -463,7 +463,7 @@
     "Usually we need to execute validation for every N epochs during training to verify the model and save the best model.\n",
     "\n",
     "Here we don't define the `validate` section step by step as it's similar to the `train` section. The full config is available:  \n",
-    "https://github.com/Project-MONAI/tutorials/blob/master/modules/bundles/spleen_segmentation/configs/train.json\n",
+    "https://github.com/Project-MONAI/tutorials/blob/master/modules/bundle/spleen_segmentation/configs/train.json\n",
     "\n",
     "Just show an example of `macro text replacement` to simplify the config content and avoid duplicated text. Please note that it's just token text replacement of the config content, not refer to the instantiated python objects."
    ]
@@ -498,7 +498,7 @@
     "We can define a `metadata` file in the bundle, which contains the metadata information relating to the model, including what the shape and format of inputs and outputs are, what the meaning of the outputs are, what type of model is present, and other information. The structure is a dictionary containing a defined set of keys with additional user-specified keys.\n",
     "\n",
     "A typical `metadata` example is available:  \n",
-    "https://github.com/Project-MONAI/tutorials/blob/master/modules/bundles/spleen_segmentation/configs/metadata.json"
+    "https://github.com/Project-MONAI/tutorials/blob/master/modules/bundle/spleen_segmentation/configs/metadata.json"
    ]
   },
   {

modules/bundle/hybrid_programming/README.md

@@ -0,0 +1,10 @@
+# Description
+This example mainly shows a typical use case that parses the config files in your own python program, instantiates necessary components with python program and executes the inference.
+
+## commands example
+
+Parse the config files in the python program and execute inference from the python program:
+
+```
+python -m scripts.inference run --config_file "['configs/data_loading.json','configs/net_inferer.json','configs/post_processing.json']" --ckpt_path <path_to_checkpoint>
+```

modules/bundle/hybrid_programming/configs/data_loading.json

@@ -0,0 +1,52 @@
+{
+    "image_key": "image",
+    "preprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "LoadImaged",
+                "keys": "@image_key"
+            },
+            {
+                "_target_": "EnsureChannelFirstd",
+                "keys": "@image_key"
+            },
+            {
+                "_target_": "Orientationd",
+                "keys": "@image_key",
+                "axcodes": "RAS"
+            },
+            {
+                "_target_": "Spacingd",
+                "keys": "@image_key",
+                "pixdim": [1.5, 1.5, 2.0],
+                "mode": "bilinear"
+            },
+            {
+                "_target_": "ScaleIntensityRanged",
+                "keys": "@image_key",
+                "a_min": -57,
+                "a_max": 164,
+                "b_min": 0,
+                "b_max": 1,
+                "clip": true
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "@image_key"
+            }
+        ]
+    },
+    "dataset": {
+        "_target_": "Dataset",
+        "data": "@input_data",
+        "transform": "@preprocessing"
+    },
+    "dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@dataset",
+        "batch_size": 1,
+        "shuffle": false,
+        "num_workers": 4
+    }
+}

modules/bundle/hybrid_programming/configs/net_inferer.json

@@ -0,0 +1,18 @@
+{
+    "network": {
+        "_target_": "UNet",
+        "spatial_dims": 3,
+        "in_channels": 1,
+        "out_channels": 2,
+        "channels": [16, 32, 64, 128, 256],
+        "strides": [2, 2, 2, 2],
+        "num_res_units": 2,
+        "norm": "batch"
+    },
+    "inferer": {
+        "_target_": "SlidingWindowInferer",
+        "roi_size": [96, 96, 96],
+        "sw_batch_size": 4,
+        "overlap": 0.5
+    }
+}

modules/bundle/hybrid_programming/configs/post_processing.json

@@ -0,0 +1,33 @@
+{
+    "pred_key": "pred",
+    "postprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "Activationsd",
+                "keys": "@pred_key",
+                "softmax": true
+            },
+            {
+                "_target_": "Invertd",
+                "keys": "@pred_key",
+                "transform": "@preprocessing",
+                "orig_keys": "@image_key",
+                "meta_key_postfix": "meta_dict",
+                "nearest_interp": false,
+                "to_tensor": true
+            },
+            {
+                "_target_": "AsDiscreted",
+                "keys": "@pred_key",
+                "argmax": true
+            },
+            {
+                "_target_": "SaveImaged",
+                "keys": "@pred_key",
+                "meta_keys": "pred_meta_dict",
+                "output_dir": "@output_dir"
+            }
+        ]
+    }
+}

modules/bundle/hybrid_programming/scripts/__init__.py

@@ -0,0 +1,10 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

modules/bundle/hybrid_programming/scripts/inference.py

@@ -0,0 +1,56 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import Sequence, Union
+import glob
+
+import torch
+from monai.bundle import ConfigParser
+from monai.data import decollate_batch
+
+
+def run(config_file: Union[str, Sequence[str]], ckpt_path: str):
+    parser = ConfigParser()
+    parser.read_config(config_file)
+    # edit the config content at runtime for input / output information and lazy instantiation
+    datalist = list(sorted(glob.glob("/workspace/data/Task09_Spleen/imagesTs/*.nii.gz")))
+    input_data = [{f"{parser['image_key']}": i} for i in datalist]
+    output_dir = "/workspace/data/tutorials/modules/bundle/hybrid_programming/eval"
+    parser["input_data"] = input_data
+    parser["output_dir"] = output_dir
+    parser["inferer"]["roi_size"] = [160, 160, 160]
+
+    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
+    # instantialize the components
+    model = parser.get_parsed_content("network").to(device)
+    model.load_state_dict(torch.load(ckpt_path))
+
+    dataloader = parser.get_parsed_content("dataloader")
+    if len(dataloader) == 0:
+        raise ValueError("no data found in the dataloader, please ensure the input image paths are accessable.")
+    inferer = parser.get_parsed_content("inferer")
+    postprocessing = parser.get_parsed_content("postprocessing")
+
+    model.eval()
+    with torch.no_grad():
+        for d in dataloader:
+            images = d[parser["image_key"]].to(device)
+            # define sliding window size and batch size for windows inference
+            d[parser["pred_key"]] = inferer(inputs=images, network=model)
+            # decollate the batch data into a list of dictionaries, then execute postprocessing transforms
+            [postprocessing(i) for i in decollate_batch(d)]
+
+
+if __name__ == "__main__":
+    from monai.utils import optional_import
+
+    fire, _ = optional_import("fire")
+    fire.Fire()

modules/bundles/spleen_segmentation/configs/evaluate.json renamed to modules/bundle/spleen_segmentation/configs/evaluate.json

@@ -3,7 +3,7 @@
         "$import glob",
         "$import os"
     ],
-    "bundle_root": "/workspace/data/tutorials/modules/bundles/spleen_segmentation",
+    "bundle_root": "/workspace/data/tutorials/modules/bundle/spleen_segmentation",
     "output_dir": "$@bundle_root + '/eval'",
     "dataset_dir": "/workspace/data/Task09_Spleen",
     "datalist": "$list(sorted(glob.glob(@dataset_dir + '/imagesTs/*.nii.gz')))",

modules/bundles/spleen_segmentation/configs/inference.json renamed to modules/bundle/spleen_segmentation/configs/inference.json

@@ -5,7 +5,7 @@
         "0.1.0": "complete the model package",
         "0.0.1": "initialize the model package structure"
     },
-    "monai_version": "0.8.0",
+    "monai_version": "0.9.0",
     "pytorch_version": "1.10.0",
     "numpy_version": "1.21.2",
     "optional_packages_version": {

modules/bundles/spleen_segmentation/configs/logging.conf renamed to modules/bundle/spleen_segmentation/configs/logging.conf

@@ -4,7 +4,7 @@
         "$import os",
         "$import ignite"
     ],
-    "bundle_root": "/workspace/data/tutorials/modules/bundles/spleen_segmentation",
+    "bundle_root": "/workspace/data/tutorials/modules/bundle/spleen_segmentation",
     "ckpt_dir": "$@bundle_root + '/models'",
     "output_dir": "$@bundle_root + '/eval'",
     "dataset_dir": "/workspace/data/Task09_Spleen",