Output for 21k classes ViT

[dinarkino]

Is it possible to get an output for 21k classes (from ImageNet-21k) for ViT models instead of 1k classes for standard ImageNet? I tried many of them, and it seems that they pretrained for the 1k ImageNet classes.

[rwightman]

@dinarkino The models with _in21k suffix are the 21k weights from the official JAX repo, but the authors zero'd the head so it won't produce 21k classification results of any use. You can use them for finetuning though. The only models with working 21k heads are the bitm resnetv2 models.

All of these models are on the master here but not yet on the pypi for pip install.

[dinarkino]

@rwightman thank you for your answer!

That is interesting. Do you know why they zeroed the head? I read the article, and I know that they do this before fine-tuning, but I don't know why they do that for the published models without fine-tuning.

I wanted to do several experiments with an output of visual transformer as an image descriptor. Bigger descriptor (e.g. 21k length) could be a more interesting choice for that.