Skip to content

v0.2.0 - Gemma 2 2b, Compilation support, Assisted Genertion

Latest
Compare
Choose a tag to compare
@gante gante released this 07 Aug 14:00
· 1 commit to main since this release

This release follows the advent of Gemma 2 2b, the smallest Gemma 2 model available to date. Its small size is perfect for limited hardware resources or as a speed-up companion to the larger variants through assisted generation!

Highlights:

  • 💎 Gemma 2 2b support
  • speed preset (uses torch.compile)
  • 👪 CLI uses the 2b to speed up the larger models, through assisted generation

Thank you to all contributors since the previous release: @sanchit-gandhi @gante @shapito27 @Vaibhavs10