New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support sliding window attention in FA3 #1218

Open

lin-ht opened this issue Sep 10, 2024 · 3 comments

lin-ht commented Sep 10, 2024

Thank you for the great work on FA3. I am wondering if FA3 will support sliding window attention soon as FA2 does?

Contributor

tridao commented Sep 10, 2024

There's a PR on that

Author

lin-ht commented Sep 10, 2024

Wow~ Thanks! I put the PR link here for future reference: #1197

Collaborator

ipiszy commented Sep 16, 2024

Sorry I deleted the original branch by accident. This is the new PR: #1233.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment