Training Recipe to Reproduce the MaxViT-Tiny results in Timm

[mmaaz60]

The MaxViT-tiny model (maxvit_tiny_rw_224) provided in Timm achieves 83.51 top-1 IN1K accuracy when evaluated. However, I am interested in training this model in Timm and reproduce the same numbers. Can you share the exact training recipe (ideally config/training command) to reproduce these numbers in timm?

Thanks

[rwightman]

@mmaaz60 not sure if it's 100% exact, but would be close to https://gist.github.com/rwightman/943c0fe59293b44024bbd2d5d23e6303#file-maxvit_tiny_256-yaml or the nano there... although this was for a 256x256, don't think I ever finished it, but would have been based on the 224

[rwightman]

Those are the hparams for maxvit_rmlp_tiny_rw_256.sw_in1k actually, renamed the model afterwards. But either way, would be very similar to the 224 non rmlp maxvit_tiny_rw_224

[hankyul2]

Hi @mmaaz60

Here is my training receipt to train maxvit_tiny_tf_224 with 4 GPUs. This will reach about 83.6~8 top-1 accuracy (in my cases: 83.8% top-1 best accuracy).

torchrun --nproc_per_node=4 --master_port=12345 train.py /path/to/imagenet --model maxvit_tiny_tf_224 --aa rand-m15-mstd0.5-inc1 --mixup .8 --cutmix 1.0 --remode pixel --reprob 0.25 --drop-path .2 --opt adamw --weight-decay .05 --sched cosine --epochs 300 --lr 3e-3 --warmup-lr 1e-6 --warmup-epoch 30 --min-lr 1e-5 -b 64 --grad-accum-steps 16 --smoothing 0.1 --clip-grad 1.0 -j 8 --amp --pin-mem --channels-last 

I also upload the training logs and checkpoints at this repo.

Thank you.

Hankyul