Added a Nifti data loader for issue #270 #361
Conversation
Signed-off-by: gupta <[email protected]>
|
Kudos, SonarCloud Quality Gate passed!
|
| image_reader.SetFileName(str(nii_path)) | ||
| image = image_reader.Execute() | ||
| image_np = np.transpose(SimpleITK.GetArrayFromImage(image), [2, 1, 0]) | ||
| return image_np |
There was a problem hiding this comment.
So this will only return the ndarray, how would the metadata be preserved? Metadata, e.g. spacing and affine are important and needed for the inference transforms.
In App SDK there is a Image domain class, very primitive with only ndarray and a list of metadata in a dict. We used this object to preserve image ndarray as well as a list of required metadata from the DICOM Series to vol image. The built-in inference operators are expecting Image type as input.
Also, with NIfTI image only, unless the original slices are converted to a DICOM series, the DICOM Seg Writer operator can not be used, simply because the Seg needs to refer back to the original DICOM instances.
There was a problem hiding this comment.
I wrote it before metatensor became a norm in MONAI. I wanted an operator which just reads an images and hands off the numpy array to the next operator for inference. I do not think we can or need to worry about DICOM Seg operator if someone is doing inference on Nifti operators. It is not expected. I will look into this class and ensure that we can output metadata from the operator. I will try to tie in this operator with an example to make a complete use case.
There was a problem hiding this comment.
Unless the input images in NIfTI have already been resampled to the spacing/orientation to fit the model, resampling and orientation are needed in the pre-processing step, granted in many cases and in competitions the training and validation images have been pre-prepared.
The MONAI built-in readers simply returns the ndarray and a metadata dict containing certain entries common/required by the transforms. The LoadImage class then convert both into a MetaTensor object.
|
|
||
| def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext): | ||
| input_path = op_input.get().path | ||
| image_np = self.convert_and_save(input_path) |
There was a problem hiding this comment.
The op_input path is typically a folder, even though file path is also possible depending on how the input is set up. It is important that the path itself is checked for what type it is.
Also, in the case of folder, there has to logic to scan the file(s) for supported types, e.g. nii, nii.gz, mhd, etc. The monai core has the LoadImage class that has a set of supported readers, including nii, and wonder why not simple use it in the application, or make use of it here.
There was a problem hiding this comment.
Yes monai core has the logic for LoadImage. But, it is also very specific in terms of how it wants to handle images. Now it is using metadata, there is this idea of channel first etc. I wanted a very raw example, which someone can look and manipulate more easily if needed. We will talk about it today.
Thanks @MMelQin
There was a problem hiding this comment.
LoadImage shouldn't manipulate the image data at all and leaves that to subsequent transforms. It should preserve the Nifti header in the metadata so you can use it later especially for saving output images back to the same format as the input. It doesn't however take as input single file names so to scan a directory we typically use glob to get a list of files.
There was a problem hiding this comment.
@MMelQin I understand that the op_input path is typically a folder. But that is a valid point for dicom images. The nifti images are just files. In a typical scenario, all the nifti images will be contained in a single folder so it makes sense to use op_input path for a file instead of a folder.
@ericspod and @MMelQin
Yes LoadImage can be used for loading the images. However, in the monai environment the channels are considered first by default. and there might be use cases where that is not the case. In addition, I thought it will be nice to have an example where you can show that monai deploy operators do not necessarily depend on the monai core components and it is possible to play around with other readers and demonstrate the flexibility it offers.
But if you think, it is not particularly useful please go ahead with the changes.
There was a problem hiding this comment.
Channels are added first because that's what Pytorch wants when considering what the dimensions of a tensor represent. What we often do is use Orientation/Orientationd transforms to change the orientation of the image based on what the Nifti header says so that the spatial dimensions always represent the same thing, like here. If that's the kind of situation in which you wouldn't want to add a channel dimension then I'd say the use of these transforms together is the correct solution.
I'm obviously biased but I'd always recommend using MONAI code rather than other libraries for doing the same thing.
Signed-off-by: gupta [email protected]