Skip to content

Commit 8451aba

Browse files
committed
Update on "Transform model to be able to use Attention Sink"
This PR adds necessary functions for transforming the model to be able to use Attention Sink. Differential Revision: [D65571289](https://our.internmc.facebook.com/intern/diff/D65571289/) [ghstack-poisoned]
2 parents e67f42c + 3b2aa49 commit 8451aba

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/models/llama/source_transformation/attention_sink.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,7 @@ def _replace_attention(
266266
for _, child_module in module._modules.items():
267267
if len(list(child_module.children())) > 0: # pyre-ignore [16]
268268
_replace_attention(
269-
module=child_module,
269+
module=child_module, # pyre-ignore [6]
270270
rope_with_attention_sink=rope_with_attention_sink,
271271
sink_size=sink_size,
272272
window_size=window_size,

0 commit comments

Comments
 (0)