Skip to content

v0.15.0
Compare
Choose a tag to compare
@mvpatel2000 mvpatel2000 released this 20 Jun 19:04
v0.15.0
9983bcd
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

🚀 Composer v0.15.0

What's New

  1. Exact Eval (#2218)

    Composer now supports exact evaluation! Now, evaluation will give the exact same results regardless of the number of GPUs by removing any duplicated samples from the dataloader.

  2. Monolithic Checkpoint Loading (#2288)

    When training large models, loading the model and optimizer on every rank can use up all the system memory. With FSDP, Composer can now load the model and optimizer on only rank 0 and broadcast it to all other ranks. To enable:

    from composer import Trainer

# Construct Trainer
trainer = Trainer(
   ...,
   fsdp_config={
      load_monolith_rank0_only: True
   },
)

# Train!
trainer.fit()

    and ensure the model on rank 0 is on CPU/GPU (as opposed to meta).

  3. Spin Dataloaders

    By default, Composer spins dataloaders back to the current timestamp to ensure deterministic resumption. However, dataloader spinning can be very slow, so Trainer now has a new flag to disable spinning if determinism is not required. To enable:

    from composer import Trainer

# Construct Trainer
trainer = Trainer(
   ...,
   spin_dataloaders=False,
)

# Train!
trainer.fit()

Deprecations

  • HealthChecker is now deprecated and will be removed in v0.17.0

Bug Fixes

What's Changed

New Contributors

Full Changelog: v0.14.1...v0.15.0

Contributors

sashaDoubov, eracah, and 15 other contributors
Assets 2
Loading