best fid score?

[yangsenwxy]

Hi,
Thank you so much for your work
How to choose the weight of the best FID score, I used your code to reproduce, none of them reached the performance to the original paper, My model in the FlintstonesSV of FID is 30

[xichenpan]

@yangsenwxy Hi, could you please provide some visual samples? and also the training config like how many epochs, freeze clip/blip/resnet or not.

[xichenpan]

btw, in our experiment, we just use the ckpt in 50 epochs, cause the loss is irrelevant with the fid score.

[yangsenwxy]

can I have you wechat?I tell you by email? ys810137152@gmail.com

[yangsenwxy]

This is my config

[xichenpan]

can I have you wechat?I tell you by email? ys810137152@gmail.com

@yangsenwxy Yeah, I am happy to chat. But I prefer to discuss in this thread because this is a new implementation and we haven't train a model using this repo. If we can figure out the problem, it may help a lot~

[xichenpan]

This is my config

It seems the init_lr is too large, you may try our config of 1e-5. Also, could you provide your sample config, like scheduler and steps?

[yangsenwxy]

I will give you some exsample

[xichenpan]

I will give you some exsample

You may follow the config given in https://github.com/Flash-321/ARLDM/blob/main/config.yaml (which is the config for the report performance.) The ddim scheduler with 50 steps provide a 2-3 higher FID score result. We suggest using ddim-6-250 or pndm-7.5-50 (this is faster and provide a 1-2 higher FID score)

[xichenpan]

I will give you some exsample

Thanks a lot! Only the generated images is enough (caption is not related to FID score)

[yangsenwxy]

链接：https://pan.baidu.com/s/1s4WU9xdS7Qn_XO-O2kvwaA
提取码：n1dl
--来自百度网盘超级会员V5的分享

[yangsenwxy]

here

[xichenpan]

here

Copy! So I think it is a training issue, since the generated visual stories are unacceptable.

[xichenpan]

@yangsenwxy Have you ever try to train the model for 50 epochs using a init_lr of 1e-5?

[yangsenwxy]

I will try it

[xichenpan]

I will try it

@yangsenwxy Thanks! I have tried using a large lr, but it produce a bad performance when training for many epochs. You can also try to train the model for 5 epochs using a lr of 1e-4 and see if it works. I tried this setting in my early experiment and it can also generate reasonable results. Feel free to at me in this thread if there is an update or further issue!