How to train a 2D Unet with a Resnet based encoder initialized to pre trained image net weights ?

[gadithyaraju21]

Hi,

I would like to create and train Albunet initialized to pre trained image net weights. The problem is that we cannot use a 3D Unet instead a 2d Unet is required. But medical images are 3D volumes. The closest tutorial I found was this: https://github.com/Project-MONAI/tutorials/blob/master/modules/2d_inference_3d_volume.ipynb, can we do something similar like this for training, where 3D volumes are converted to 2D slices?

[Nic-Ma]

Hi @gadithyaraju21 ,

Thanks for your interest here.
May I know what the pre trained weights of UNet you mean? Do you want to download weights from somewhere and apply?
CC @ericspod

Thanks.

[gadithyaraju21]

Hi @Nic-Ma thanks for the quick reply, I should have been much clearer:

If I want to use a Resnet 50 as the encoder for the Unet but instead of training from scratch, I want to initialize it to a pre-trained ResNet 50 on image net. Furthermore, I want to train this 2d Unet on 3D images slice by slice. end to end.

Side Note: This is just a random question as two years back when transfer learning for Unet was a big thing I implemented something like this from scratch but it was a really slow process to train.

[Nic-Ma]

Hi @wyli ,

I remember we have some tickets or examples similar to this 3D -> 2D slices question but I can't find...
Do you know something?

Thanks in advance.

[wyli]

yes, it's still open #130

[gadithyaraju21]

Thank you! This is exactly what I was looking for!