Skip to content
This repository has been archived by the owner on Jan 24, 2024. It is now read-only.

[WIP]support matmul_v2_grad #1392

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

zkh2016
Copy link

@zkh2016 zkh2016 commented May 5, 2023

支持matmul_v2_grad,对于scale->gemm->scale这种结构,反向可以省去两次scale操作。当seq_len比较大的时候,attention中q*k的输出矩阵比较大,单独执行scale,耗时也会比较高。

@paddle-bot
Copy link

paddle-bot bot commented May 5, 2023

Thanks for your contribution!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants