Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/lightning #35

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Feature/lightning #35

wants to merge 20 commits into from

Commits on Aug 29, 2020

  1. step 1: free the GPT module of config and flatten out the args. WIP, … 

    …breaks notebooks
    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    6110298 View commit details
    Browse the repository at this point in the history

  2. refactor out the learning rate decay class as a callback

    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    08f5b9a View commit details
    Browse the repository at this point in the history

  3. data loaders are passed directly to fit() instead of the datasets

    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    3fa57cd View commit details
    Browse the repository at this point in the history

  4. data loaders are passed directly to fit() instead of the dataset, ver… 

    …sion 2 haha
    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    e88f076 View commit details
    Browse the repository at this point in the history

  5. model is also passed into fit() instead of __init__ ,sure.

    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    c0823ec View commit details
    Browse the repository at this point in the history

  6. and finally get rid of Config object for the Trainer

    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    923b6fc View commit details
    Browse the repository at this point in the history

  7. add training_step to the model and remove DataParallel functionality … 

    …from the base Trainer, will go to Lightning
    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    a5a6d1a View commit details
    Browse the repository at this point in the history

  8. final integration pieces, now runs with both, but it ain't super pret… 

    …ty yet...
    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    990c0c7 View commit details
    Browse the repository at this point in the history

  9. one more refactor, this is better because the equivalence to lightnin… 

    …g is now much cleaner and all of lightning functionality is in one file
    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    81650ae View commit details
    Browse the repository at this point in the history

  10. refactor into a datamodule, attempt number 1

    @karpathy
    karpathy committed Aug 29, 2020
    Configuration menu
    Copy the full SHA
    fb37e03 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2020

  1. use a standard benchmark (text8) and implement train/val/test splits

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    fa10298 View commit details
    Browse the repository at this point in the history

  2. move instantiation of text dataset into the constructor so we don't h… 

    …ave to create it twice
    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    0ed3376 View commit details
    Browse the repository at this point in the history

  3. support fp16/32 precision in bench

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    ebd40f1 View commit details
    Browse the repository at this point in the history

  4. switch to a faster version of zero_grad()

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    1aa67ca View commit details
    Browse the repository at this point in the history

  5. massive refactor yet again. this was all probably a pretty bad idea

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    452a5ab View commit details
    Browse the repository at this point in the history

  6. delete Result structs in favor of dicts

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    9b1e5a4 View commit details
    Browse the repository at this point in the history

  7. testing now works with both lightning and minLightning

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    4817231 View commit details
    Browse the repository at this point in the history

  8. make labels non-blocking transfer to overlap them, but i don't really… 

    … expect this to do too much to latency
    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    d91bb1c View commit details
    Browse the repository at this point in the history

  9. get rid of spurious function for the model

    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    492b79f View commit details
    Browse the repository at this point in the history

  10. reorg the bench code to support multigpu training, have to indent pro… 

    …perly under __main__
    @karpathy
    karpathy committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    a796899 View commit details
    Browse the repository at this point in the history