We provide the scripts as well as the trained models on the downstream tasks as follows. Note that before you run the finetuning scripts, you need to download the corresponding pretrained ROSITA model here.
The following script run the finetuning on the train+trainvalsplit of VQAv2.
$ bash scripts/train-vqa-vqav2.shAs the training stage complete, you may run the following script to run evaluation on the test split and generate a result.json file under the output folder. This file can be submitted to the online server to obtain the performance.
$ bash scripts/test-vqa-vqav2.shWe also provide the checkpoint model to reproduce the following results on test-dev and test-std split using the testing script above.
| Name | Test-dev | Test-std | Downloads | ||||||
|---|---|---|---|---|---|---|---|---|---|
| All | Y/N | Num | Other | All | Y/N | Num | Other | ||
| ROSITA-base | 73.91 | 89.91 | 56.07 | 64.29 | 73.97 | 89.76 | 55.81 | 64.39 | model |
The following script run the finetuning on the train split of RefCOCO.
$ bash scripts/train-rec-refcoco.shAs the training stage complete, you may run the following script to run evaluation on the val/testA/testB split of RefCOCO.
bash scripts/test-rec-refcoco.shWe also provide the checkpoint model to reproduce the following results on val and testA/B split using the testing script above.
| Name | RefCOCO | Downloads | ||
|---|---|---|---|---|
| val | testA | testB | ||
| ROSITA-base | 84.79 | 87.99 | 78.28 | model |
The following script run the finetuning on the train split of RefCOCO+.
bash scripts/train-rec-refcoco+.shAs the training stage complete, you may run the following script to run evaluation on the val/testA/testB split of RefCOCO+.
bash scripts/test-rec-refcoco+.shWe also provide the checkpoint model to reproduce the following results on val and testA/B split using the testing script above.
| Name | RefCOCO+ | Downloads | ||
|---|---|---|---|---|
| val | testA | testB | ||
| ROSITA-base | 76.06 | 82.01 | 67.40 | model |
The following script run the finetuning on the train split of RefCOCOg.
bash scripts/train-rec-refcocog.shAs the training stage complete, you may run the following script to run evaluation on the val/test split of RefCOCOg.
bash scripts/test-rec-refcocog.shWe also provide the checkpoint model to reproduce the following results on val and testA/B split using the testing script above.
| Name | RefCOCOg | Downloads | |
|---|---|---|---|
| val | test | ||
| ROSITA-base | 78.23 | 78.25 | model |
The following script run the finetuning on the train split of ITR-COCO.
bash scripts/train-itr-coco.shAs the training stage complete, you may run the following script to run evaluation on the testall split of ITR-COCO.
bash scripts/test-itr-coco.shWe also provide the checkpoint model to reproduce the following results on testall split using the testing script above.
| Name | IR-COCO | TR-COCO | Downloads | ||||
|---|---|---|---|---|---|---|---|
| R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | ||
| ROSITA-base | 54.40 | 80.92 | 88.60 | 71.26 | 91.62 | 95.58 | model |
The following script run the finetuning on the train split of ITR-Flickr.
bash scripts/train-itr-flickr.shAs the training stage complete, you may run the following script to run evaluation on the test split of ITR-Flickr.
bash scripts/test-itr-flickr.shWe also provide the checkpoint model to reproduce the following results on test split using the testing script above.
| Name | IR-Flickr | TR-Flickr | Downloads | ||||
|---|---|---|---|---|---|---|---|
| R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | ||
| ROSITA-base | 74.08 | 92.44 | 96.08 | 88.90 | 98.10 | 99.30 | model |