Skip to content
forked from zomux/deepy

Highly extensible deep learning framework

License

Notifications You must be signed in to change notification settings

robotzheng/deepy

 
 

Repository files navigation

deepy: Highly extensible deep learning framework based on Theano

Build Quality Requirements Status Documentation Status Coverage Status MIT

Dependencies

  • Python 2.7 (Better on Linux)
  • numpy
  • theano
  • scipy for L-BFGS and CG optimization

Tutorials (Work in progress)

http://deepy.readthedocs.org/en/latest/

Clean interface

# MNIST Multi-layer model with dropout.
from deepy.dataset import MnistDataset, MiniBatches
from deepy.networks import NeuralClassifier
from deepy.layers import Dense, Softmax, Dropout
from deepy.trainers import MomentumTrainer, LearningRateAnnealer

model = NeuralClassifier(input_dim=28*28)
model.stack(Dense(256, 'relu'),
            Dropout(0.2),
            Dense(256, 'relu'),
            Dropout(0.2),
            Dense(10, 'linear'),
            Softmax())

trainer = MomentumTrainer(model)

annealer = LearningRateAnnealer(trainer)

mnist = MiniBatches(MnistDataset(), batch_size=20)

trainer.run(mnist, controllers=[annealer])

Examples

Enviroment setting

  • CPU
source bin/cpu_env.sh
  • GPU
source bin/gpu_env.sh

MNIST Handwriting task

  • Simple MLP
python experiments/mnist/mlp.py
  • MLP with dropout
python experiments/mnist/mlp_dropout.py
  • MLP with PReLU and dropout
python experiments/mnist/mlp_prelu_dropout.py
  • Deep convolution
python experiments/mnist/deep_convolution.py
  • Elastic distortion
python experiments/mnist/mlp_elastic_distortion.py
python experiments/attention_models/baseline.py

Language model

Penn Treebank benchmark

  • Baseline RNNLM (Full-output layer)
python experiments/lm/baseline_rnnlm.py
  • Class-based RNNLM
python experiments/lm/class_based_rnnlm.py
  • LSTM based LM (Full-output layer)
python experiments/lm/lstm_rnnlm.py

Char-based language models

  • Char-based LM with LSTM
python experiments/lm/char_lstm.py
  • Char-based LM with Deep RNN
python experiments/lm/char_rnn.py

Deep Q learning

  • Start server
pip install Flask-SocketIO
python experiments/deep_qlearning/server.py
  • Open this address in browser
http://localhost:5003

Auto encoders

  • Recurrent NN based auto-encoder
python experiments/auto_encoders/rnn_auto_encoder.py
  • Recursive auto-encoder
python experiments/auto_encoders/recursive_auto_encoder.py

Train with CG and L-BFGS

  • CG
python experiments/scipy_training/mnist_cg.py
  • L-BFGS
python experiments/scipy_training/mnist_lbfgs.py

Other experiments

DRAW

See https://github.com/uaca/deepy-draw

# Train the model
python mnist_training.py
# Create animation
python animation.py experiments/draw/mnist1.gz

Highway networks

python experiments/highway_networks/mnist_baseline.py
python experiments/highway_networks/mnist_highway.py

Effect of different initialization schemes

python experiments/initialization_schemes/gaussian.py
python experiments/initialization_schemes/uniform.py
python experiments/initialization_schemes/xavier_glorot.py
python experiments/initialization_schemes/kaiming_he.py

Other features

  • Auto gradient correction

Sorry for that deepy is not well documented currently, but the framework is designed in the spirit of simplicity and readability. This will be improved if someone requires.

Raphael Shu, 2015

About

Highly extensible deep learning framework

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 93.5%
  • HTML 6.3%
  • Shell 0.2%