hpc-ml

my notes on ML/DL and HPC.

Running environment (JupyterHub at OLCF)

• First login (http://jupyter.olcf.ornl.gov), and launch a terminal:

        # create a spec based on current BASE conda environment
        conda list --explicit > spec-file.txt

        # create a new environment, which will persist
        conda create -p /ccs/proj/gen150/fwang2/mistral --file spec-file.txt 

        # other customization if needed, then activate 
        source activate /ccs/proj/gen150/fwang2/mistral

• Make newly created environment visiable

        python -m ipykernel install --user --name mistral --display-name mistral

Now, when you start a new launcher, you should see the new kernel "mistral" listed.

Regression

• Linear Regression
• Linear Regression using PyTorch
• Logistic Regression using PyTorch

Dimension Reduction

• Single Value Decomposition

Clustering

• Hierarchical Clustering
• K-means Clustering
• DBSCAN

Deep Learning

• Gradient Descent

• Compute Graph, Back Propagation

• Neural Network Concept

• 2-Layer Gradient Descent

• 3-Layer Gradient Descent

• Regularization and dropout

• Activation Functions

  • Look into softmax and cross-entropy in PyTorch context

• CNN

• RNN and LSTM

• Transferred Learning

• Federated Learning

Scalable Learning

RNN and LSTM

• My RNN Notes
• Unreasonable effectiveness of RNN
• Understanding LSTM, Colah's Blog, there is accompanying Krish's explaination.
• Word embedding
• Running Jupyter on Summit

Transformer

• Transformer from scratch

Tools

• numpy, matplotlib
• Customized data loader
• Linear Algebra Review