Attention-Guided GANs (AGGAN) for Unsupervised Image-to-Image Translation

AGGAN Framework

Visualization of Attention Mask and Content Mask

Paper | Project page | Slides

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation.
Hao Tang¹, Dan Xu², Nicu Sebe¹, Yan Yan³.
¹University of Trento, Italy, ²University of Oxford, UK, ³Texas State University, USA
In IJCNN 2019 (Oral)
The repository offers the official implementation of our paper in PyTorch.

License

Copyright (C) 2019 University of Trento, Italy.

All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)

The code is released for academic research use only. For commercial use, please contact [email protected].

Prerequisites

Code is intended to work with Python 3.6.x, it hasn't been tested with previous versions.

PyTorch 0.3.1 & Torchvision

Follow the instructions in pytorch.org for your current setup.

Note: Make sure that your PyTorch's version is 0.3.1. Later I will update it to 0.4 if I have time.

Visdom

To plot loss graphs and draw images in a nice web browser view:

pip install visdom

Training & Testing

1. Setup the dataset

First, you will need to build your datasets by setting up the following directory structure:

.
├── datasets                   
|   ├── <dataset_name>         # i.e., bu3dfe_neutral2happiness
|   |   ├── train              # Training
|   |   |   ├── A              # Contains domain A images (i.e., neutral)
|   |   |   └── B              # Contains domain B images (i.e., happiness)
|   |   └── test               # Testing
|   |   |   ├── A              # Contains domain A images (i.e., neutral)
|   |   |   └── B              # Contains domain B images (i.e., happiness)

Examples of the dataset is shown in this folder.

2. Training

export CUDA_VISIBLE_DEVICES=X; 
python train.py --dataroot datasets/<dataset_name>/ --save_name <save_name> --cuda

This command will start a training session using the images under the dataroot/train directory with the default hyperparameters. You are free to change those hyperparameters, see ./train --help for a description of those.

Both generators and discriminators weights will be saved under the output directory.

If you don't own a GPU remove the --cuda option, although I advise you to get one!

You can also view the training progress as well as live output images by running python -m visdom in another terminal and opening http://localhost:8097/ in your favourite web browser.

3. Testing

python test.py --dataroot datasets/<dataset_name>/ --save_name <save_name> --cuda

This command will take the images under the dataroot/test directory, run them through the generators and save the output under the output/A and output/B directories. As with training, some parameters like the weights to load, can be tweaked, see ./test --help for more information.

Code Structure

• train.py, test.py: the entry point for training and testing.
• models.py: defines the architecture of all models for AttentionGAN
• datasets.py: defines the class for loading images.
• utils.py: consists of several helper functions.

Citation

If you use this code for your research, please cite our papers.

@inproceedings{tang2019attention,
  title={Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation},
  author={Tang, Hao and Xu, Dan and Sebe, Nicu and Yan, Yan},
  booktitle={International Joint Conference on Neural Networks (IJCNN)},
  year={2019}
}

Acknowledgments

This source code borrows heavily from CycleGAN. We acknowledge the National Institute of Standards and Technology Grant 60NANB17D191 for funding this research. We also acknowledge the gift donation from Cisco, Inc for this research.

Related Projects (Image-to-Image Translation)

1. Supervised image-to-image translation methods.

Pix2pix
SelectionGAN
Pix2pixHD
SPADE
BiCycleGAN
Encoder-Decoder

2. Unsupervised image-to-image translation methods.

CycleGAN
DiscoGAN
DualGAN
DistancGAN
ComboGAN
StarGAN

3. Label-, mask- or attention-guided image translation methods.

IcGAN
GANimation
Unsupervised-Attention-guided-Image-to-Image-Translation or PyTorch Version
InstaGAN

Contributions

If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Hao Tang ([email protected]).