Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Checkpointing] Save/load trainer #12

Merged
merged 26 commits into from
Sep 15, 2023
Merged

[Checkpointing] Save/load trainer #12

merged 26 commits into from
Sep 15, 2023

Conversation

matteobettini
Copy link
Collaborator

This PR introduces the possibility of saving and loading experiments

  • Saving is done at a given interval at config time
  • Loading is done from a provided file and is compatible with wandb run resuming

cc @vmoens if you want to have a look

@matteobettini
amend
c95d140 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
dd92b25 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
3ea5c06 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
661d37e 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
15f74bf 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
6275a60 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
ippo one optim
2611585 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
revert ippo
fc282f9 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
cfg
4a22897 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
update task config
4f9f139 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
remove callback
43be08c 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
readme loggers
05f775e 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
307456b 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
allow wandb loading
7028a05 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
allow wandb loading
6c436fc 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
4193733 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
add finishing logging
497be80 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
more printing
0ae2ad3 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
more printing
575c2b0 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
fixes
c69f14e 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
fixes
1f3347a 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
fixes
c20e7bb 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
Merge branch 'main' into save_trainer
c7335e2 
# Conflicts:
#	benchmarl/experiment/experiment.py
#	benchmarl/experiment/logger.py
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 14, 2023
@matteobettini
add .pt
60051e6 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
move saving
1d7404a 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini
amend
15ee308 
Signed-off-by: Matteo Bettini <[email protected]>
@matteobettini matteobettini marked this pull request as ready for review September 14, 2023 18:52
@matteobettini matteobettini merged commit 0cff333 into main Sep 15, 2023
4 checks passed
@matteobettini matteobettini deleted the save_trainer branch September 15, 2023 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@matteobettini @facebook-github-bot