Concern on leaderboard

[cuishuhao]

Hello, we doubt the leaderboard.
We try a lot of model pretrained on ImageNet-1K within 100M parameters，and all of them got acc blow 40%.
But we find that 7 teams upload source only model with acc larger than 40%.
We would like to you to check the models for most of the teams, since the rank is determined by all teams.
Thank you!

[MInner]

Thank you for letting us know, we will look into this. We will like to remind you that the source performance does not affect ranking in part because it's meaning might vary, ie some methods start from scratch and fine-tune on source while also adapting to target, so their source only performance is the init performance. But we will look into the abnormally high source only performance of these submissions, thank you.

[BurhanUlTayyab]

Hi,

I got the first place on the leaderboard. We used a BERT model and fed tokens of images rather than the text. The model was trained on ImageNet-1K and had only 98M parameters. FYI, try using this BeiT (https://arxiv.org/pdf/2106.08254.pdf) that's only trained on ImageNet and check the source accuracy.

Also, I would appreciate if you trust the judgement of organisers and their judgment and stop being salty.

[junbaoZHUO]

Hi,

I got the first place on the leaderboard. We used a BERT model and fed tokens of images rather than the text. The model was trained on ImageNet-1K and had only 98M parameters. FYI, try using this BeiT (https://arxiv.org/pdf/2106.08254.pdf) that's only trained on ImageNet and check the source accuracy.

Also, I would appreciate if you trust the judgement of organisers and their judgment and stop being salty.

Congratulations! Can you provide the url of BeiT pretrained model you used? Thx!

[BurhanUlTayyab]

Hi,

We used this for pre-training on ImageNet-1k and then did immediate finetuning on ImageNet-1k again.

I'll share the weights with you in a bit.

[donggoing]

Hi,

We used this for pre-training on ImageNet-1k and then did immediate finetuning on ImageNet-1k again.

I'll share the weights with you in a bit.

Could you please share the fine-tuned weights with me? I will very appreciate for it.