Unsupervised Adaptation for Synthetic-to-Real Handwritten Word Recognition

An unsupervised writer adaptation approach that is able to automatically adjust a generic handwritten word recognizer, fully trained with synthetic fonts, towards a new incoming writer.

Unsupervised Adaptation for Synthetic-to-Real Handwritten Word Recognition
Lei Kang, Marçal Rusiñol, Alicia Fornés, Pau Riba, and Mauricio Villegas
2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA, 2020, pp. 3491-3500, doi: 10.1109/WACV45572.2020.9093392.

Software environment:

• Ubuntu 16.04 x64
• Python 3.7
• PyTorch 0.3

Dataset preparation

We are using 60K synthetic word images as source data and popular real handwritten word datasets (GW, IAM, RIMES, Esposalles and CVL) as target data.

Note for the project:

We have shown the results of different experimental settings in the paper. However, in order to make the code clean and easy to understand, the code we show in this repo is the domain adaptation from synthetic word images to IAM word dataset. For the other experiments, it would be easy to reproduce with some minor changes.

Before training:

We use tasas_cer.sh and tasas_wer.sh to calculate the character error rate and word error rate, which have dependency on an external tool. So make sure that this tool has been properly installed and also double check the url to be your correct location inside the two shell scripts.

How to train it?

Once both of the synthetic data and real data are prepared, you need to denote the correct urls in the file loadData6_vgg.py, then you are ready to go by running:

./run_train.sh

Note: Which GPU to use or which epoch you want to start from could be set in this shell script. (Epoch ID corresponds to the weights that you want to load in the folder save_weights)

How to test it?

./run_test.sh

Citation

If you use the code for your research, please cite our paper:

@inproceedings{kang2020unsupervised,
  title={Unsupervised Adaptation for Synthetic-to-Real Handwritten Word Recognition},
  author={Kang, Lei and Rusi{\~n}ol, Mar{\c{c}}al and Forn{\'e}s, Alicia and Riba, Pau and Villegas, Mauricio},
  booktitle={2020 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  pages={3491--3500},
  year={2020},
  organization={IEEE}
}