Awesome Deep Learning - System Design

Table of Contents

• Domain Classification Papers

• Augmentation

• Self Supervised Learning Papers

• Time series

• Courses

• Videos and Lectures

Domain Classification Papers

1. Compact Feature Learning for Multi-Domain Image Classification

2. Dynamic Domain Adaptation for Efficient Inference

3. Unsupervised Domain Adaptation through Self-Supervision

4. Unsupervised Domain Adaptation with Similarity Learning

5. Compact Feature Learning for Multi-domain Image Classification

Augmentation

1. A Simple Semi-Supervised Learning Framework for Object Detection

Self Supervised Learning Papers

1. Efficient Self-supervised Vision Transformers for Representation Learning

2. MoCo-v1: Momentum Contrast for Unsupervised Visual Representation Learning

3. MoCo-v2: Improved Baselines with Momentum Contrastive Learning

4. Moco-v3: An Empirical Study of Training Self-Supervised Vision Transformers

5. Sisiam: Exploring Simple Siamese Representation Learning

6. A Framework For Contrastive Self-Supervised Learning And Designing A New Approach

7. Meta Pseudo Labels

8. DINO: Emerging Properties in Self-Supervised Vision Transformers

9. SimCLR-v1: A Simple Framework for Contrastive Learning of Visual Representations

10. SimCLR-v2: Big Self-Supervised Models are Strong Semi-Supervised Learners

11. MoBY: Self-Supervised Learning with Swin Transformers

12. SPICE: Semantic Pseudo-Labeling for Image Clustering

13. Infomin: What Makes for Good Views for Contrastive Learning?

14. Bootstrap Your Own Latent： A New Approach to Self-Supervised

15. SeLa: Self-labelling via simultaneous clustering and representation learning

16. SLADE: A Self-Training Framework For Distance Metric Learning

Time series

1. Google AI - Interpretable Deep Learning for Time Series Forecasting

Reranking

1. Re-ranking for image retrieval and transductive few-shot classification - Facebook AI 2021 Neurips

2. Divide and fuse: a re-ranking approach for person Re-ID

3. Re-ranking via Metric Fusion for Object Retrieval and Person Re-identification

4. Contextual Similarity Aggregation with Self-attention for Visual Re-ranking

5. Re-ranking Person Re-identification with k-reciprocal Encoding

6. Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking

Courses

1. Machine Learning - Stanford by Andrew Ng in Coursera (2010-2014)
2. Machine Learning - Caltech by Yaser Abu-Mostafa (2012-2014)
3. Machine Learning - Carnegie Mellon by Tom Mitchell (Spring 2011)
4. Neural Networks for Machine Learning by Geoffrey Hinton in Coursera (2012)
5. Neural networks class by Hugo Larochelle from Université de Sherbrooke (2013)
6. Deep Learning Course by CILVR lab @ NYU (2014)
7. A.I - Berkeley by Dan Klein and Pieter Abbeel (2013)
8. A.I - MIT by Patrick Henry Winston (2010)
9. Vision and learning - computers and brains by Shimon Ullman, Tomaso Poggio, Ethan Meyers @ MIT (2013)
10. Convolutional Neural Networks for Visual Recognition - Stanford by Fei-Fei Li, Andrej Karpathy (2017)
11. Deep Learning for Natural Language Processing - Stanford
12. Neural Networks - usherbrooke
13. Machine Learning - Oxford (2014-2015)
14. Deep Learning - Nvidia (2015)
15. Graduate Summer School: Deep Learning, Feature Learning by Geoffrey Hinton, Yoshua Bengio, Yann LeCun, Andrew Ng, Nando de Freitas and several others @ IPAM, UCLA (2012)
16. Deep Learning - Udacity/Google by Vincent Vanhoucke and Arpan Chakraborty (2016)
17. Deep Learning - UWaterloo by Prof. Ali Ghodsi at University of Waterloo (2015)
18. Statistical Machine Learning - CMU by Prof. Larry Wasserman
19. Deep Learning Course by Yann LeCun (2016)
20. Designing, Visualizing and Understanding Deep Neural Networks-UC Berkeley
21. UVA Deep Learning Course MSc in Artificial Intelligence for the University of Amsterdam.
22. MIT 6.S094: Deep Learning for Self-Driving Cars
23. MIT 6.S191: Introduction to Deep Learning
24. Berkeley CS 294: Deep Reinforcement Learning
25. Keras in Motion video course
26. Practical Deep Learning For Coders by Jeremy Howard - Fast.ai
27. Introduction to Deep Learning by Prof. Bhiksha Raj (2017)
28. AI for Everyone by Andrew Ng (2019)
29. MIT Intro to Deep Learning 7 day bootcamp - A seven day bootcamp designed in MIT to introduce deep learning methods and applications (2019)
30. Deep Blueberry: Deep Learning - A free five-weekend plan to self-learners to learn the basics of deep-learning architectures like CNNs, LSTMs, RNNs, VAEs, GANs, DQN, A3C and more (2019)
31. Spinning Up in Deep Reinforcement Learning - A free deep reinforcement learning course by OpenAI (2019)
32. Deep Learning Specialization - Coursera - Breaking into AI with the best course from Andrew NG.
33. Deep Learning - UC Berkeley | STAT-157 by Alex Smola and Mu Li (2019)
34. Machine Learning for Mere Mortals video course by Nick Chase
35. Machine Learning Crash Course with TensorFlow APIs -Google AI
36. Deep Learning from the Foundations Jeremy Howard - Fast.ai
37. Deep Reinforcement Learning (nanodegree) - Udacity a 3-6 month Udacity nanodegree, spanning multiple courses (2018)
38. Grokking Deep Learning in Motion by Beau Carnes (2018)
39. Face Detection with Computer Vision and Deep Learning by Hakan Cebeci
40. Deep Learning Online Course list at Classpert List of Deep Learning online courses (some are free) from Classpert Online Course Search
41. AWS Machine Learning Machine Learning and Deep Learning Courses from Amazon's Machine Learning unviersity
42. Intro to Deep Learning with PyTorch - A great introductory course on Deep Learning by Udacity and Facebook AI
43. Deep Learning by Kaggle - Kaggle's free course on Deep Learning

Videos and Lectures

1. How To Create A Mind By Ray Kurzweil
2. Deep Learning, Self-Taught Learning and Unsupervised Feature Learning By Andrew Ng
3. Recent Developments in Deep Learning By Geoff Hinton
4. The Unreasonable Effectiveness of Deep Learning by Yann LeCun
5. Deep Learning of Representations by Yoshua bengio
6. Principles of Hierarchical Temporal Memory by Jeff Hawkins
7. Machine Learning Discussion Group - Deep Learning w/ Stanford AI Lab by Adam Coates
8. Making Sense of the World with Deep Learning By Adam Coates
9. Demystifying Unsupervised Feature Learning By Adam Coates
10. Visual Perception with Deep Learning By Yann LeCun
11. The Next Generation of Neural Networks By Geoffrey Hinton at GoogleTechTalks
12. The wonderful and terrifying implications of computers that can learn By Jeremy Howard at TEDxBrussels
13. Unsupervised Deep Learning - Stanford by Andrew Ng in Stanford (2011)
14. Natural Language Processing By Chris Manning in Stanford
15. A beginners Guide to Deep Neural Networks By Natalie Hammel and Lorraine Yurshansky
16. Deep Learning: Intelligence from Big Data by Steve Jurvetson (and panel) at VLAB in Stanford.
17. Introduction to Artificial Neural Networks and Deep Learning by Leo Isikdogan at Motorola Mobility HQ
18. NIPS 2016 lecture and workshop videos - NIPS 2016
19. Deep Learning Crash Course: a series of mini-lectures by Leo Isikdogan on YouTube (2018)
20. Deep Learning Crash Course By Oliver Zeigermann
21. Deep Learning with R in Motion: a live video course that teaches how to apply deep learning to text and images using the powerful Keras library and its R language interface.
22. Medical Imaging with Deep Learning Tutorial: This tutorial is styled as a graduate lecture about medical imaging with deep learning. This will cover the background of popular medical image domains (chest X-ray and histology) as well as methods to tackle multi-modality/view, segmentation, and counting tasks.
23. Deepmind x UCL Deeplearning: 2020 version
24. Deepmind x UCL Reinforcement Learning: Deep Reinforcement Learning
25. CMU 11-785 Intro to Deep learning Spring 2020 Course: 11-785, Intro to Deep Learning by Bhiksha Raj
26. Machine Learning CS 229 : End part focuses on deep learning By Andrew Ng

Attention Transformers

1. BERT video

2. Illustrated Transformer

3. Chapter 9.7

4. Neural Machine Translation by Jointly Learning to Align and Translate

5. Attention is all you need

---

Contributing

Have anything in mind that you think is awesome and would fit in this list? Feel free to send a pull request.

---

License

This repository is my own thought.