Using sklearn CalibratedClassifierCV function to calibrate a model trained with Monai using dataloaders

[LaiaHumbertVidan]

Hi,

I have trained a DenseNet40 3D CNN using Monai and I need to calibrated it on a new dataset, which consists of 3D images (with corresponding labels).

I am trying to implement the CalibratedClassifierCV function from sklearn but struggling to make it work with Dataloaders.

Here is my code so far.

path = 'path to new dataset of 3D images'

names = sorted(os.listdir(path))
paths = []
for name in names:
path = os.path.join(path, name)
paths.append(path)

data = pd.read_excel("Excel sheet with labels.xlsx",header=0)
labels = data ['labels']

Split the data into training and testing sets

ID_train, ID_test, labels_train, labels_test = train_test_split(paths, labels, test_size=0.2, random_state=42)

Create train and test datasets and dataloaders with OUH data (assume I have the corresponding train and test transforms in a separate part of the code)

train_ds = monai.data.ImageDataset(image_files=ID_train, labels=labels_train, transform=train_transforms, image_only=True)
test_ds = monai.data.ImageDataset(image_files=ID_test, labels=labels_test, transform=test_transforms, image_only=True)
batch_size = 10
train_loader = monai.data.DataLoader(train_ds, batch_size=batch_size, pin_memory=torch.cuda.is_available())
test_loader= monai.data.DataLoader(test_ds, batch_size=batch_size, pin_memory=torch.cuda.is_available())

Initialize the base classifier (3D CNN model)

PATH = 'path to the model trained with the original dataset.pth'
model = nets.DenseNet(spatial_dims=3, in_channels=1, out_channels=2, block_config=(12,12,12), dropout_prob=0.8)
model.load_state_dict(torch.load(PATH))
model.to(device)
model.eval()
base_classifier = model()

--> At this point I get the following error: TypeError: forward() missing 1 required positional argument: 'x'

The rest of the code I think would be as follows, but have not been able to test it as cannot get beyond the previous error:

Initialize the calibrated classifier with the base classifier

calibrated_classifier = CalibratedClassifierCV(base_classifier, cv='prefit')

Fit the calibrated classifier to your training data

calibrated_classifier.fit(train_loader)

Obtain predicted probabilities for each class

y_prob = calibrated_classifier.predict_proba(test_loader)

Many thanks in advance for any suggestions!
Laia

[wyli]

duplicate of Project-MONAI/MONAI#6643