Skip to content

Latest commit

 

History

History
70 lines (46 loc) · 2.17 KB

README.md

File metadata and controls

70 lines (46 loc) · 2.17 KB
Raw

Data Structures, Algorithms and Machine Learning Optimization

This repo is home to the code that accompanies Data Structures, Algorithms and Machine Learning Optimization curriculum, which provides a comprehensive overview of all of the subjects — across Data Structures, Algorithms and Machine Learning Optimization — that underlie contemporary machine learning approaches, including deep learning and other artificial intelligence techniques:

Algorithms & Data Structures Curriculum

  • Segment 1: Introduction to Data Structures and Algorithms

    • A Brief History of Data
    • A Brief History of Algorithms
    • “Big O” Notation for Time and Space Complexity

  • Segment 2: Lists and Dictionaries

    • List-Based Data Structures: Arrays, Linked Lists, Stacks, Queues, and Deques
    • Searching and Sorting: Binary, Bubble, Merge, and Quick
    • Set-Based Data Structures: Maps and Dictionaries
    • Hashing: Hash Tables, Load Factors, and Hash Maps

  • Segment 3: Trees and Graphs

    • Trees: Decision Trees, Random Forests, and Gradient-Boosting (XGBoost)
    • Graphs: Terminology, Directed Acyclic Graphs (DAGs)
    • Resources for Further Study of Data Structures & Algorithms

Machine Learning Optimization Curriculum

  • Segment 1: The Machine Learning Approach to Optimization

    • The Statistical Approach to Regression: Ordinary Least Squares
    • When Statistical Approaches to Optimization Break Down
    • The Machine Learning Solution

  • Segment 2: Gradient Descent

    • Objective Functions
    • Cost / Loss / Error Functions
    • Minimizing Cost with Gradient Descent
    • Learning Rate
    • Critical Points, incl. Saddle Points
    • Gradient Descent from Scratch with PyTorch
    • The Global Minimum and Local Minima
    • Mini-Batches and Stochastic Gradient Descent (SGD)
    • Learning Rate Scheduling
    • Maximizing Reward with Gradient Ascent

  • Segment 3: Fancy Deep Learning Optimizers

    • A Layer of Artificial Neurons in PyTorch
    • Jacobian Matrices
    • Hessian Matrices and Second-Order Optimization
    • Momentum
    • Nesterov Momentum
    • AdaGrad
    • AdaDelta
    • RMSProp
    • Adam
    • Nadam
    • Training a Deep Neural Net
    • Resources for Further Study