You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update base for Update on "[Executorch][llm] Add ring buffer based kv cache and mask calculation to MHA"
Leveraging previous work now we allow MHA to have ring buffer cache. If ring buffer cache is used
then we query the mask from kv cache and use that for sdpa instead of using precalculated mask.
In this process we had to adjsut ring buffer implementation to allow keeping the context of
full sliding window. See code for comment.
Differential Revision: [D73891425](https://our.internmc.facebook.com/intern/diff/D73891425/)
[ghstack-poisoned]
0 commit comments