Evaluation code for TAP-VID benchmark

[JennySeidenschwarz]

Hey everyone! :)

Congrats to your amazing work!! Do you have a ready-to-use file for evaluation on TAP-VID? Or do you have per-sequence performances?

Would be amazing!!

Thanks,
Jenny

[m43]

Hi Jenny, I get the following numbers for the DAVIS subset of TAP-Vid and the spaT_final.pth checkpoint shared by the authors:

tapvid_davis_avg_Jaccard	59.10832733877460
tapvid_davis_avg_delta		75.03060472325210
tapvid_davis_avg_OA		89.15536883348100

My per sequence results are here: tapvid_davis_metrics.csv

This does not match the aj=61.1,delta=76.3,oa=89.5 numbers reported in the paper, it is a bit worse than that. But I assume that this might be expected since it is not clear if the released checkpoint matches the full architecture described in the paper (e.g., with the ARAP embeddings). Maybe someone got similar results to the ones I have shared above?