Dyunet pipeline related questions

[fgbxx]

hi, all
I have two questions discussed here:

1. for the Dyunet pipeline , it use the transforms for the pre-processing such as resample and I think It will be very slow for doing data pre-process runtime especially for the large body volume when training.
2. it did not add rotation DA , I don't know reason which nnUNet added and nnUNet also consider patch size changes if do rotation DA.
   Do anyone have any ideas?

[Nic-Ma]

Hi @yiheng-wang-nv ,

Could you please help take a look at these questions?

Thanks in advance.

[yiheng-wang-nv]

Hi @fgbxx , good question!
As for 1, since PreprocessAnisotropic is a deterministic transform, it can be cached with using CacheDataset and cache_rate is not 0. In addition, you can also try to pre-compute all results and saved in the hard drive if CPU memory is not enough.
As for 2, you can definitely use rotate via adding RandAffined during training, it might be helpful for increasing the performance.

[fgbxx]

Very thanks yiheng for the re-implement nnUNet based on Monai architecture! This is very useful for us to easily recover the nnUNet's performance based on MONAI!

For 1st question, as for my task, it include half human body scan, so I think pre-compute is better solution. But I will try your advise.

For 2nd question, How can I expand the patchsize based on the Monai transform? Because the rotation will expand the object ROI size which nnUNet consider in DataLoader3D and padding that?

[yiheng-wang-nv]

Hi @fgbxx , the spatial size will not be changed after rotation via using RandAffined transform. Interpolation and padding based resample will be used after changing the affine to ensure the same spatial size.
You can also refer to the doc here

[fgbxx]

hi, yiheng
Thank you for response, I want to use dyunet to replace nnUNet at all. I will do carefully checking before that.
In nnUNet code(dataset_loading.py) DataLoader3D function, we can see the parameters: patch_size and final_patch_size, we can see Fanbian's explanations as bellow:
:param patch_size: what patch size will this data loader return? it is common practice to first load larger
patches so that a central crop after data augmentation can be done to reduce border artifacts. If unsure, use
get_patch_size() from data_augmentation.default_data_augmentation
:param final_patch_size: what will the patch finally be cropped to (after data augmentation)? this is the patch
size that goes into your network. We need this here because we will pad patients in here so that patches at the
border of patients are sampled properly
So do you think it is necessary for this step?

[yiheng-wang-nv]

Hi @fgbxx , I think it depends on the task, I'm note sure which one performs better: 1) padding to a large size, augmentation, crop to patch size, 2) crop to patch size, augmentation, or even 3) padding to a large size, augmentation (spatial based), crop to patch size, augmentation (non-spatial based).

For method 3), for example, in the reproduction of a 2d medical kaggle challenge (https://github.com/Project-MONAI/tutorials/blob/master/kaggle/RANZCR/4th_place_solution/configs/cfg_seg_philipp_16_ch_1024_ap_full.py#L40), we pad to a large size first, and then do some affine augmentations and finally crop to the patch size (output size), and then do some intensity based augmentations.

I suggest to try different ways if possible, there may not have rule of thumb here : )

[fgbxx]

Hi @yiheng-wang-nv , thanks for your details response, I will try your suggest, thanks for your help and support!

[fgbxx]

@yiheng-wang-nv hi, Yiheng
I run dynunet for train, but it is very slow, for the same gpu A100, time cost for 1 epoch:
　monai 16min一个 200iterations
　nnunet 150s <3min 250iterations

I have done the preprocessing , which mean I used the nnunet preprocessed numpy data, and not do that in runtime transform.
Transform code as bellow:



def get_task_transforms_nn(mode, task_id, pos_sample_num, neg_sample_num, num_samples):
trans_para = get_DA_paras()


basic_generator_patch_size = get_patch_size(patch_size[task_id],
                                            trans_para['rotation_x'],
                                            trans_para['rotation_y'],
                                            trans_para['rotation_z'],
                                            trans_para['scale_range'])


basic_patch_center = [basic_generator_patch_size[0]//2,basic_generator_patch_size[1]//2,basic_generator_patch_size[2]//2]


if mode != "test":
    keys = ["image", "label"]
else:
    keys = ["image"]
# 1. loading
load_transforms = [
    LoadNumpyd(keys=keys), # fanggang , change to npy data format, not need to load nii
    AddChanneld(keys=keys),
    # EnsureChannelFirstd(keys=keys),
]

"""
here , remove the pre-sampling steps in preproess , so not do resampling runtime, fanggnag
"""
# 2. sampling , as preprocessing as the nnunet do,
# comments: if not use GPU or other speed up method, will slow down trainning steps, need change
# change to the npy or npz like nnunet, but label and data splits
# sample_transforms = [
#     PreprocessAnisotropic(
#         keys=keys,
#         clip_values=clip_values[task_id],
#         pixdim=spacing[task_id],
#         normalize_values=normalize_values[task_id],
#         model_mode=mode,
#     ),
# ]
# 3. spatial transforms
if mode == "train":
    other_transforms = [
        SpatialPadd(keys=["image", "label"], spatial_size=patch_size[task_id]),

        RandCropByPosNegLabeld(
            keys=["image", "label"],
            label_key="label",
            spatial_size=basic_generator_patch_size,
            pos=pos_sample_num,
            neg=neg_sample_num,
            num_samples=num_samples,
            image_key="image",
            image_threshold=0,
        ),
        # add random rotation
        RandRotated(
            keys=keys,
            range_x=trans_para['rotation_x'],
            range_y=trans_para['rotation_y'],
            range_z=trans_para['rotation_z'],
        ),

        RandZoomd(
            keys=["image", "label"],
            min_zoom=0.9,
            max_zoom=1.2,
            mode=("trilinear", "nearest"),
            align_corners=(True, None),
            prob=0.15,
        ),
        # reocover to objective patch size
        SpatialCropd(keys=keys, roi_center=basic_patch_center, roi_size=patch_size[task_id]),
        # keep right size of objective patch size
        SpatialPadd(keys=keys, spatial_size=patch_size[task_id]),

        RandGaussianNoised(keys=["image"], std=0.01, prob=0.15),
        RandGaussianSmoothd(
            keys=["image"],
            sigma_x=(0.5, 1.15),
            sigma_y=(0.5, 1.15),
            sigma_z=(0.5, 1.15),
            prob=0.15,
        ),#GaussianBlurTransform

        # NA BrightnessMultiplicativeTransform
        RandScaleIntensityd(keys=["image"], factors=0.25, prob=0.15), #ContrastAugmentationTransform

        # do gamma
        RandAdjustContrastd(keys=["image"],gamma=trans_para['gamma_range'],prob=0.1),

        #do mirror

        RandFlipd(["image", "label"], spatial_axis=[0], prob=0.5),
        RandFlipd(["image", "label"], spatial_axis=[1], prob=0.5),
        RandFlipd(["image", "label"], spatial_axis=[2], prob=0.5),

        CastToTyped(keys=["image", "label"], dtype=(np.float32, np.uint8)),
        EnsureTyped(keys=["image", "label"]),
    ]
elif mode == "validation":
    other_transforms = [
        CastToTyped(keys=["image", "label"], dtype=(np.float32, np.uint8)),
        EnsureTyped(keys=["image", "label"]),
    ]
else:
    other_transforms = [
        CastToTyped(keys=["image"], dtype=(np.float32)),
        EnsureTyped(keys=["image"]),
    ]

# all_transforms = load_transforms + sample_transforms + other_transforms
all_transforms = load_transforms + other_transforms
return Compose(all_transforms)

[yiheng-wang-nv]

Hi, I'm not familiar with the latest nnUNet pipeline, but I remember that when I tried it in 2020, the speed is not faster than using monai.
I suggest to split the training into:

1. loading and transform 2) model forward 3) backward
   and compare these two pipelines, if 1) is slower, then compare only a part of the transforms to figure out which place lead to the issue.

[fgbxx]

hi yiheng @yiheng-wang-nv
I have tested the pipeline , I found seem the multi-process does not work for dataloader. the workernum set 1 cost same as setting 12 or more. 　Do you have any idea?

     train_ds = Dataset(
        data=datalist,
        transform=transform,
    )
    print(train_ds)
    data_loader = DataLoader(
        train_ds,
        batch_size=batch_size,
        shuffle=True,
        num_workers=args.train_num_workers,
        drop_last=True,
    )

[ibro45]

Hi @yiheng-wang-nv, do you still have the metrics on 1) performance before fine-tuning, i.e. the performance of train.sh, and 2) the per-fold performance prior to ensembling? Thank you!