checkpoints.md

Pre-trained Checkpoints

Pre-trained checkpoints of base, medium, and small-scale models can be accessed via Dropbox

Instruction-tuned Checkpoints

We provided the checkpoints of base, medium, and small-scale models that were instruction-tuned using the combination of a (downloadable) subset of llava_med_instruct_10k.json with training sets from SLAKE and PathVQA. These checkpoints can be accessed via Dropbox

Fine-tuned Checkpoints

Fine-tuned checkpoints of base-scale model can be downloaded in the following links. Details regrading inference hyperparameters are described in the paper.

Multi-modal Downstreams

• SLAKE: https://www.dropbox.com/s/xk6asgvdezaqtuy/slake.pt?dl=0
• PathVQA: https://www.dropbox.com/s/0jh3k11s4zj0hr3/pathvqa.pt?dl=0
• VQA-RAD: https://www.dropbox.com/scl/fi/4ywpsoehfotqg5rvxo9n1/vqa_rad_fixed.pt?rlkey=0lmwuafggz2ianz57d80lhk67&dl=0
• IU X-ray: https://www.dropbox.com/s/0llg5e77l6nlb2b/iu_xray.pt?dl=0
• Peir Gross: https://www.dropbox.com/s/fwl0r4qpe8rajas/peir_gross.pt?dl=0
• ROCO: https://www.dropbox.com/s/cgvw6q8ozk9uzim/roco.pt?dl=0
  
  

VQ-GAN Checkpoints

• vqgan_gumbel_f8_8192: https://heibox.uni-heidelberg.de/d/2e5662443a6b4307b470/?p=%2Fconfigs&mode=list