-
Notifications
You must be signed in to change notification settings - Fork 175
Other Models
While neural-style-pt comes with 3 default models already, you may wish to use other neural network models. Neural style transfer makes use of pre-trained convolutional neural network models to recreate the content image in the style of one or more style images. Models can be trained on different datasets, and already trained models can be "fine-tuned" with a new dataset while preserving it's existing knowledge.
All of the models listed below in neural-style-pt's model zoo are available for download here: https://drive.google.com/open?id=1OGKfoIehp2MiJL2Iq_8VMTy76L6waGC8
After downloading the models, it is recommended that you place them into your models directory for ease of use.
- This script will automatically download all the available models and put them in your models directory, provided that it is placed in your neural-style-pt directory before being run: https://gist.github.com/ProGamerGov/d597dd6cefcd51a4724d59967f86aafa
Currently, the following models are officially supported by neural-style-pt in addition to the default models: VGG16_SOD_finetune, fcn32s-heavy-pascal, nyud-fcn32s-color-heavy, pascalcontext-fcn32s-heavy, siftflow-fcn32s-heavy, channel_pruning, vgg16_places365, vgg16_hybrid1365, VGG16-Stylized-ImageNet. See below for more in depth information on each model.
To use the newly installed models you'll have to call them with the -model_file parameter using: -model_file models/[modelname]. You may have to use different layers with -content_layers and -style_layers, depending on the model.
The standard VGG_ILSVRC_19_layers caffemodel from the original neural style converted to PyTorch, this one is installed with the included model download script.
Model file: vgg19-d01eb7cb.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu3_4, relu4_1, relu4_2, relu4_3, relu4_4, relu5_1, relu5_2, relu5_3, relu5_4
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/vgg19-d01eb7cb.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
The standard VGG_ILSVRC_16_layers caffemodel from the original neural style converted to PyTorch, this one is installed with the included model download script.
Model file: vgg16-00b39a1b.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/vgg16-00b39a1b.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
The standard nin_imagenet_conv NIN caffemodel from the original neural style converted to PyTorch, this one is installed with the included model download script.
Model file: nin_imagenet.pth
Usable layers: relu0, relu1, relu2, relu3, relu5, relu6, relu7, relu8, relu9, relu10, relu11, relu12
Basic command:
python3 neural_style.lua -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/nin_imagenet.pth -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12
Source: https://github.com/ProGamerGov/pytorch-nin, https://gist.github.com/mavenlin/d802a5849de39225bcc6
Similar to VGG-ILSVRC-16, but tends to create more smooth/clean results. Same resource usage as VGG-16. Released in 2016.
- This model has been found to produce smother/cleaner results when compared to the other VGG-16 models.
Model file: VGG16_SOD_finetune.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/VGG16_SOD_finetune.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://www.cs.bu.edu/groups/ivc/Subitizing/model/VGG16/
VGG16-Stylized-ImageNet from ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
A VGG-16 model trained on a stylized version of the standard ImageNet dataset. The ImageNet dataset used to train the model was stylized with pytorch-AdaIN, which is based on 'Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization' [Huang+, ICCV2017].
- Please note that the
VGG16-Stylized-ImageNetmodel requires around a power of 2 increase to both content and style weights in order to achieve good results.
Model file: VGG16-Stylized-ImageNet.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/VGG16-Stylized-ImageNet.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/rgeirhos/texture-vs-shape, https://github.com/rgeirhos/Stylized-ImageNet, https://arxiv.org/abs/1811.12231
Made for the Places365-Challenge which includes the Places2 Challenge 2016, the ILSVRC and the COCO joint workshop at ECCV 2016. Places365 is the successor to the Places205 model.
Model file: vgg16_places365.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/vgg16_places365.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/CSAILVision/places365
Made for the Places365-Challenge which includes the Places2 Challenge 2016, the ILSVRC and the COCO joint workshop at ECCV 2016. Places365 is the successor to the Places205 model.
Model file: vgg16_hybrid1365.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/vgg16_hybrid1365.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/CSAILVision/places365
Model file: fcn32s-heavy-pascal.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/fcn32s-heavy-pascal.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/shelhamer/fcn.berkeleyvision.org
Model file: nyud-fcn32s-color-heavy.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/nyud-fcn32s-color-heavy.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/shelhamer/fcn.berkeleyvision.org
Model file: pascalcontext-fcn32s-heavy.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/pascalcontext-fcn32s-heavy.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/shelhamer/fcn.berkeleyvision.org
Model file: siftflow-fcn32s-heavy.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/siftflow-fcn32s-heavy.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/shelhamer/fcn.berkeleyvision.org
A pruned version of the standard VGG-16 model. Uses less memory, and is a middle ground between NIN and VGG-16 in terms of quality and memory usage.
Model file: channel_pruning.pth
Usable layers: relu1_1, relu1_2, relu2_1, relu2_2, relu3_1, relu3_2, relu3_3, relu4_1, relu4_2, relu4_3, relu5_1, relu5_2, relu5_3
Basic command:
python3 neural_style.py -style_image [image1] -content_image [image2] -output_image [outimage] -model_file models/channel_pruning.pth -content_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1
Source: https://github.com/yihui-he/channel-pruning/