Skip to content

Commit bc4e112

Browse files
authored
llama : deci : support ffn-free with attention (#13296)
1 parent 39e73ae commit bc4e112

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-model.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4792,7 +4792,7 @@ struct llm_build_deci : public llm_graph_context {
47924792
}
47934793

47944794
// FFN-free layer of Llama-3_1-Nemotron-Ultra-253B
4795-
if (n_head == 0 && n_ff == 0) {
4795+
if (n_ff == 0) {
47964796
continue;
47974797
}
47984798

0 commit comments

Comments
 (0)