Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better MP, multi-gpu, atom graphs, replay, GPS, LS-GFN, and fixes #141

Open
wants to merge 35 commits into
base: trunk
Choose a base branch
from
Open

Better MP, multi-gpu, atom graphs, replay, GPS, LS-GFN, and fixes #141

wants to merge 35 commits into from

Conversation

bengioe
Copy link
Collaborator

@bengioe bengioe commented Oct 8, 2024 •
edited
Loading

Tons of new features:

  • Shared pinned buffers #120 shared pinned buffer for better MP
  • C molecular graphs
  • Local Search GFN
  • Better replay (prio + uniques)
  • Multi-GPU training
  • Graph model can now do GPS
  • New fragment env variant
  • Many fixes

@bengioe
first throw at refactoring SamplingIterator
7dbca12
@bengioe
Merge branch 'trunk' into bengioe-better-iterators
939cb56
@bengioe
changed all iterators to DataSource
dfba1ca
@bengioe
lots of little fixes, tested all tasks, better device management
e5239fb
@bengioe
style
43dfc2b
@bengioe
change batch size hyperparameters + fix nested dataclasses
279ecfc
@bengioe
Merge branch 'trunk' into bengioe-better-iterators
2ba251a
@bengioe
move things around & prevent circular import
282bbfb
@bengioe
tox
c3bc6d0
@bengioe
fix imports
b1c5630
@bengioe
replace device references with get_worker_device
a64a639
@bengioe
little fixes
28bcc59
@bengioe
a few more stragglers
4811e7c
@bengioe
proof of concept of using shared pinned buffers
7d32ac1
@bengioe
32mb buffer
d4a2a7d
@bengioe
add to DataSource
27dfc23
@bengioe
various fixes
e9f1dc1
@bengioe
major simplification by reusing pickling mechanisms
c048e77
@bengioe
memory copy + fixes and doc
acfe070
@bengioe
Merge branch 'trunk' into bengioe-mp-with-batch-buffers
9454da8
@bengioe
Merge branch 'trunk' into bengioe-mp-with-batch-buffers
2b9da70
@bengioe
fix global_cfg + opt_Z when there's no Z
907ffcd
@bengioe
fix entropy when masks are used
60722a7
@bengioe
small fixes
f859640
@bengioe
removing timing prints
d536233
@bengioe
C graphs, DDP, logit scaling
6c3beba
@bengioe
C mol valence fix, mask-backwards sample, MLE in TB, priority replay,…
67f4b62 
… Pad action, GPS architecture, some support for distributed DDP
@bengioe
first (bad) attempt
a1534be
@bengioe
working local serach, cond_info dict pass, allow no log dir, fix Pad …
4491f6b 
…and U PB
@bengioe
lstb file
c5373cf
@bengioe bengioe changed the title Better MP, multi-gpu, atom graphs, replay, GPS, and fixes Better MP, multi-gpu, atom graphs, replay, GPS, LS-GFN, and fixes Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@bengioe