Setup test on runner.sh file

Signed-off-by: nhhung1810 <[email protected]>

Author: nhhung1810

modules/network_contraints/unet_plusplus.ipynb

@@ -1,21 +1,50 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Copyright (c) MONAI Consortium  \n",
+    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
+    "you may not use this file except in compliance with the License.  \n",
+    "You may obtain a copy of the License at  \n",
+    "    http://www.apache.org/licenses/LICENSE-2.0  \n",
+    "Unless required by applicable law or agreed to in writing, software  \n",
+    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
+    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
+    "See the License for the specific language governing permissions and  \n",
+    "limitations under the License.\n",
+    "\n",
+    "# UNet++ input size constrains\n",
+    "\n",
+    "MONAI provides an enhanced version of UNet (``monai.networks.nets.UNet ``), which not only supports residual units, but also can use more hyperparameters (like ``strides``, ``kernel_size`` and ``up_kernel_size``) than ``monai.networks.nets.BasicUNet``. However, ``UNet`` has some constrains for both network hyperparameters and sizes of input.\n",
+    "\n",
+    "MONAI provides a version of UNET++ (`` monai.networks.nets.BasicUnetPlusPlus ``), with fixed num. of down-scale layer, strides of 2. The configurations you can change are: the number input and output channels, number of hidden channels (6 different layers), norm and activation, bias of convolution, dropout rate, and up-sampling model. As `UNET`, different model configurations can affect the input shape.\n",
+    "\n",
+    "The constrains of hyperparameters can be found in the docstring of the network, and this tutorial is focused on how to determine a reasonable input size."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup enviroments"
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Copyright (c) MONAI Consortium\n",
-    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
-    "# you may not use this file except in compliance with the License.\n",
-    "# You may obtain a copy of the License at\n",
-    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
-    "# Unless required by applicable law or agreed to in writing, software\n",
-    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
-    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
-    "# See the License for the specific language governing permissions and\n",
-    "# limitations under the License."
+    "!python -c \"import monai\" || pip install -q monai-weekly"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup imports"
    ]
   },
   {
@@ -41,10 +70,10 @@
       "scikit-image version: NOT INSTALLED or UNKNOWN VERSION.\n",
       "scipy version: NOT INSTALLED or UNKNOWN VERSION.\n",
       "Pillow version: NOT INSTALLED or UNKNOWN VERSION.\n",
-      "Tensorboard version: NOT INSTALLED or UNKNOWN VERSION.\n",
+      "Tensorboard version: 2.13.0\n",
       "gdown version: NOT INSTALLED or UNKNOWN VERSION.\n",
       "TorchVision version: NOT INSTALLED or UNKNOWN VERSION.\n",
-      "tqdm version: NOT INSTALLED or UNKNOWN VERSION.\n",
+      "tqdm version: 4.65.0\n",
       "lmdb version: NOT INSTALLED or UNKNOWN VERSION.\n",
       "psutil version: 5.9.5\n",
       "pandas version: NOT INSTALLED or UNKNOWN VERSION.\n",
@@ -62,11 +91,18 @@
    "source": [
     "from monai.networks.nets import BasicUnetPlusPlus\n",
     "import monai\n",
-    "import math\n",
     "import torch\n",
     "import torch.nn as nn\n",
+    "from typing import Dict\n",
     "\n",
-    "monai.config.print_config()"
+    "monai.config.print_config()\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Check UNet++ structure"
    ]
   },
   {
@@ -117,7 +153,7 @@
     "print(monai.networks.layers.factories.Norm.factories.keys())\n",
     "for norm_layer in monai.networks.layers.factories.Norm.factories.keys():\n",
     "    try:\n",
-    "        model = BasicUnetPlusPlus(  \n",
+    "        model = BasicUnetPlusPlus(\n",
     "            spatial_dims=3,\n",
     "            in_channels=3,\n",
     "            out_channels=3,\n",
@@ -126,15 +162,15 @@
     "            norm=norm_layer,\n",
     "        )\n",
     "    except Exception as msg:\n",
-    "        print(f\"Exception layer: {msg}\")\n",
-    "    "
+    "        print(f\"Exception layer: {msg}\")\n"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# Normalization\n",
+    "\n",
     "UNET++ use the same `TwoConv`, `Down`, and `UpCat` as UNet. Therefore, you can referred to the `modules/UNet_input_size_constrains.ipynb` for break down analysis. For summary, the constraints for these types of normalization are:\n",
     "\n",
     "- Instance Norm: the product of spatial dimension must > 1 (not include channel and batch)\n",
@@ -144,7 +180,7 @@
     "\n",
     "As for UNET++ have 4 down-sampling blocks with 2x kernel size, with no argument to change this behavior, the smallest edge we can have is `2**4 = 16`, and after the last down-sampling block, the `vector.shape  = [..., ..., 1, 1]` or (`[..., ..., 1, 1, 1]` for 3D), which will cause error for the Normalization layer.\n",
     "\n",
-    "See the test code below for examples of batch norm and instance norm"
+    "See the test code below for examples of batch norm and instance norm\n"
    ]
   },
   {
@@ -187,32 +223,26 @@
     }
    ],
    "source": [
-    "from typing import Dict\n",
-    "\n",
-    "\n",
     "def make_model_with_layer(layer_norm):\n",
     "    return BasicUnetPlusPlus(\n",
-    "        spatial_dims=2,\n",
-    "        in_channels=3,\n",
-    "        out_channels=1,\n",
-    "        features=(32, 32, 64, 128, 256, 32),\n",
-    "        norm=layer_norm\n",
+    "        spatial_dims=2, in_channels=3, out_channels=1, features=(32, 32, 64, 128, 256, 32), norm=layer_norm\n",
     "    )\n",
     "\n",
+    "\n",
     "def test_min_dim():\n",
     "    MIN_EDGE = 16\n",
     "    batch_size, spatial_dim, H, W = 1, 3, MIN_EDGE, MIN_EDGE\n",
     "    MODEL_BY_NORM_LAYER: Dict[str, BasicUnetPlusPlus] = {}\n",
     "    print(\"Prepare model\")\n",
-    "    for norm_layer in ['instance', 'batch']:\n",
+    "    for norm_layer in [\"instance\", \"batch\"]:\n",
     "        MODEL_BY_NORM_LAYER[norm_layer] = make_model_with_layer(norm_layer)\n",
-    "        \n",
+    "\n",
     "    # print(f\"Input dimension {(batch_size, spatial_dim, H, W)} that will cause error\")\n",
-    "    for norm_layer in ['instance', 'batch']:\n",
-    "        print(\"=\"*10 + f\" USING NORM LAYER: {norm_layer.upper()} \" + \"=\"*10)\n",
+    "    for norm_layer in [\"instance\", \"batch\"]:\n",
+    "        print(\"=\" * 10 + f\" USING NORM LAYER: {norm_layer.upper()} \" + \"=\" * 10)\n",
     "        model = MODEL_BY_NORM_LAYER[norm_layer]\n",
     "        print(\"_\" * 10 + \" Changing the H dimension of 2D input \" + \"_\" * 10)\n",
-    "        for _H_temp in [H, H*2]:\n",
+    "        for _H_temp in [H, H * 2]:\n",
     "            try:\n",
     "                x = torch.ones(batch_size, spatial_dim, _H_temp, W)\n",
     "                print(f\">> Using Input.shape={x.shape}\")\n",
@@ -231,6 +261,7 @@
     "                print(f\">> Exception: {msg}\\n\")\n",
     "    pass\n",
     "\n",
+    "\n",
     "with torch.no_grad():\n",
     "    test_min_dim()"
    ]
@@ -271,12 +302,12 @@
     "\n",
     "        return x\n",
     "\n",
-    "```"
+    "```\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 86,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
@@ -296,13 +327,18 @@
    ],
    "source": [
     "# Example about the pooling with odd shape\n",
-    "un_pool = torch.Tensor(\n",
-    "    [[1, 2, 3, 4, 5],\n",
-    "    [1, 2, 3, 4, 5],\n",
-    "    [1, 2, 3, 4, 5],\n",
-    "    [1, 2, 3, 4, 5],\n",
-    "    [1, 2, 3, 4, 5]]\n",
-    ") * torch.Tensor([1, 2, 3, 4, 5])[..., None]\n",
+    "un_pool = (\n",
+    "    torch.Tensor(\n",
+    "        [\n",
+    "            [1, 2, 3, 4, 5],\n",
+    "            [1, 2, 3, 4, 5],\n",
+    "            [1, 2, 3, 4, 5],\n",
+    "            [1, 2, 3, 4, 5],\n",
+    "            [1, 2, 3, 4, 5],\n",
+    "        ]\n",
+    "    )\n",
+    "    * torch.Tensor([1, 2, 3, 4, 5])[..., None]\n",
+    ")\n",
     "un_pool = un_pool[None, None, ...]\n",
     "\n",
     "pooled = nn.MaxPool2d(kernel_size=2)(un_pool)\n",
@@ -342,14 +378,14 @@
     "#         [batch_size, spatial_dim, D, H, W]\n",
     "#     )\n",
     "#     # y = model.forward(x)\n",
-    "#     # print(f\"{x.shape=}\")  \n",
+    "#     # print(f\"{x.shape=}\")\n",
     "#     # print(f\"{[o.shape for o in y]=}\")\n",
-    "    \n",
+    "\n",
     "#     return model, x\n",
     "\n",
     "# # Loss edge info. if input dim is not divisible by 2 when pooling\n",
     "# # Ensure the lowest image dimension\n",
-    "# # As the lowest dim before the norm is [1x1x1] \n",
+    "# # As the lowest dim before the norm is [1x1x1]\n",
     "# # -> instance and batch norm don't allow that, so make it at least\n",
     "# # [2x1x1]\n",
     "# from monai.networks.blocks import ADN\n",
@@ -358,23 +394,23 @@
     "#     print(f\"Input {x.shape=}\")\n",
     "#     model.eval()\n",
     "#     # 2 conv\n",
-    "#     x_0_0 = model.conv_0_0(x)     \n",
+    "#     x_0_0 = model.conv_0_0(x)\n",
     "#     # down conv\n",
-    "#     x_1_0 = model.conv_1_0(x_0_0) \n",
+    "#     x_1_0 = model.conv_1_0(x_0_0)\n",
     "#     print(f\"{x_1_0.shape=}\")\n",
     "#     x_0_1 = model.upcat_0_1(x_1_0, x_0_0)\n",
     "#     print(f\"{x_0_1.shape=}\")\n",
-    "#     # x_2_0 = model.conv_2_0(x_1_0) \n",
+    "#     # x_2_0 = model.conv_2_0(x_1_0)\n",
     "#     # print(f\"{x_2_0.shape=}\")\n",
-    "#     # x_3_0 = model.conv_3_0(x_2_0) \n",
+    "#     # x_3_0 = model.conv_3_0(x_2_0)\n",
     "#     # print(f\"{x_3_0.shape=}\") # 2 x 2 -> 1 x 1\n",
     "#     # pooled = nn.MaxPool3d(kernel_size=2)(x_3_0)\n",
     "#     # print(f\"{pooled.shape=}\") # 2 x 2 -> 1 x 1\n",
     "#     # conved1 = nn.Conv3d(in_channels=128, out_channels=256, kernel_size=3, stride=1, padding=1)(pooled)\n",
     "#     # conved2 = nn.Conv3d(in_channels=256, out_channels=256, kernel_size=3, stride=1, padding=1)(conved1)\n",
     "#     # print(f\"{conved1.shape=}\") # 2 x 2 -> 1 x 1\n",
     "#     # print(f\"{conved2.shape=}\") # 2 x 2 -> 1 x 1\n",
-    "    \n",
+    "\n",
     "#     # normed = ADN(\n",
     "#     #     ordering='NDA',\n",
     "#     #     in_channels=256,\n",
@@ -385,53 +421,13 @@
     "#     # )(conved2)\n",
     "#     # print(f\"{normed.shape=}\") # 2 x 2 -> 1 x 1\n",
     "\n",
-    "#     # x_4_0 = model.conv_4_0(x_3_0) \n",
+    "#     # x_4_0 = model.conv_4_0(x_3_0)\n",
     "#     # print(f\"{x_4_0.shape=}\")\n",
     "\n",
     "#     # Up path\n",
     "#     # x_3_0 = model.conv_3_0(x_2_0)\n",
     "#     # print(f\"{x_1_0.shape=}\")"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 37,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "37"
-      ]
-     },
-     "execution_count": 37,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 29,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "un_pool.shape=torch.Size([1, 1, 5, 5]), pooled.shape=torch.Size([1, 1, 2, 2])\n",
-      "tensor([[[[ 1.,  2.,  3.,  4.,  5.],\n",
-      "          [ 2.,  4.,  6.,  8., 10.],\n",
-      "          [ 3.,  6.,  9., 12., 15.],\n",
-      "          [ 4.,  8., 12., 16., 20.],\n",
-      "          [ 5., 10., 15., 20., 25.]]]])\n",
-      "tensor([[[[ 4.,  8.],\n",
-      "          [ 8., 16.]]]])\n"
-     ]
-    }
-   ],
-   "source": []
   }
  ],
  "metadata": {

runner.sh

@@ -34,6 +34,7 @@ doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" image_dataset.ipynb
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" decollate_batch.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" csv_datasets.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" UNet_input_size_constrains.ipynb)
+doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" unet_plusplus.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" network_api.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" tcia_csv_processing.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" transform_visualization.ipynb)