请问一下在prefill计算时，ProcessKV和FlattenKV的作用分别是什么？ #2063
Unanswered
sitabulaixizawaluduo
asked this question in
Q&A
Replies: 1 comment 3 replies
-
|
是的 |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
我看到ProcessKV和FlattenKV分别做了一次rope，FlattenKV中把kv cache从block中取出来时为什么还要再做一次rope呢，这个操作是针对什么场景的？ |
Beta Was this translation helpful? Give feedback.
-
|
同样好奇 |
Beta Was this translation helpful? Give feedback.
-
|
这两个地方不会同时生效 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
是不是可以理解ProcessKV是把经过qkv_proj计算的k v存到block里面，FlattenKV是把block kv cache转为整块的kv cache参与后续的attention计算？
Beta Was this translation helpful? Give feedback.
All reactions