Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BurstAttention and Ulyless all2all support for long sequence training. #203

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

BurstAttention and Ulyless all2all support for long sequence training. #203

wants to merge 11 commits into from

Commits on May 28, 2024

  1. add all2all ops

    @MayDomine
    MayDomine committed May 28, 2024
    Configuration menu
    Copy the full SHA
    7b107cc View commit details
    Browse the repository at this point in the history

  2. benchmark and ops update

    @MayDomine
    MayDomine committed May 28, 2024
    Configuration menu
    Copy the full SHA
    d658b12 View commit details
    Browse the repository at this point in the history

  3. add all2all benchmark

    @MayDomine
    MayDomine committed May 28, 2024
    Configuration menu
    Copy the full SHA
    d9e026c View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. sequence parallel comm init and update topology

    @MayDomine
    MayDomine committed May 29, 2024
    Configuration menu
    Copy the full SHA
    bac71f6 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. provide sequence parallel example

    @MayDomine
    MayDomine committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    eaa3719 View commit details
    Browse the repository at this point in the history

  2. empty commit

    @MayDomine
    MayDomine committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    d4d42f3 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. update ops and example for burst_attn

    @MayDomine
    MayDomine committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    43d1190 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. normal attn test pass

    @MayDomine
    MayDomine committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    1f14319 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. add causal for burst_attn and add test for causal/no_causal

    @MayDomine
    MayDomine committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    5c9eacd View commit details
    Browse the repository at this point in the history

  2. clean unused import

    @MayDomine
    MayDomine committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    1614ad1 View commit details
    Browse the repository at this point in the history

  3. move import statement

    @MayDomine
    MayDomine committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    4e1bc98 View commit details
    Browse the repository at this point in the history