A small little repo for my paper replication on ViTs. The original paper can be found here:
https://arxiv.org/pdf/2010.11929.pdf
-
Download the cifar-100 dataset and unzip it. The dataset can be downloaded here: https://www.cs.toronto.edu/~kriz/cifar.html
-
In the same folder as your cifar-100, clone the repo and open it in your IDE of choice.
-
Run the demo.ipynb file to see a ViT in action on the cifar-100 dataset.
-
Play around with the hyperparameters to see how it impacts the size of the model, the time it takes to train the model, and the accuracy of the model.
-
If you want to see behind the scenes, feel free to check out model.py.
This repository is not meant for development purposes and is only meant to show a small replica of a ViT.
Depending on the power and capability of your machine, it may take long to train the ViT