Code for EE3-25 Deep Learning Coursework
For setup, if you are on a cloud virtual machine please run:
python starter.py
Please note that root permissions will be required, if you are using your personal machine then please ensure all of the required linux and python packages listed in this script is installed.
The setup script also automatically downloads the required dataset, if you wish to do so manually please run:
wget -O hpatches_data.zip https://imperialcollegelondon.box.com/shared/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip
unzip -q ./hpatches_data.zip
rm ./hpatches_data.zip
The environment where the code was tested to be working fine was:
- cuda 10.0.130
- tensorflow-gpu 1.13.0rc2
- tensorboard 1.12
- keras (via tf) 2.2.4
- keras-contrib 2.0.8
- python 3.7.2
Each experiment has an optimised *.yaml file defined with all hyper-parameter settings. The following table lists down the run commands required to try out the different experiments.
| Model Type | Model Name | Command |
|---|---|---|
| Denoiser | Baseline (Shallow U-Net) | python main.py -c config/denoise_1a.yaml |
| Denoiser | Deep U-Net | python main.py -c config/denoise_1b.yaml |
| Denoiser | Deep U-Net + DSSIM + MAE Loss (Best) | python main.py -c config/denoise_1c.yaml |
| Descriptor | Baseline (L2-Net) | python main.py -c config/descript_1a.yaml |
| Descriptor | L2-Net + Ep. Shuffl. | python main.py -c config/descript_1a+.yaml |
| Descriptor | L2-Net + Ep. Shuffl. + SemiHardNeg Loss + ResBlocks | python main.py -c config/descript_1b.yaml |
All experiments are logged using tensorboard and saved in the logs/ directory. There exists a few experiments to show case training and validation curves along with image generation for evaluating denoiser's performance. To view them in your browser, please run:
tensorboard --logdir logs
For evaluation, HPatches is used. In order to perform evaluation you need the saved weights, typically the best weights in terms of validation loss, for both denoiser and descriptor:
python post_train.py -dnf <PATH_TO_DENOISER_CACHE>/denoiser_best.hdf5 -dsf <PATH_TO_DESCRIPTOR_CACHE>/descriptor_best.hdf5
The best score achieved was 0.839, 0.2499 and 0.5535 (in mAP) for patch verification, image matching and retireval respectively. A cache of the best model for denoiser and descriptor can be found in the cache folder.