Skip to content

Commit 2f3a46f

Browse files
committed
train : make KQ_pos memory buffer permanent via dummy scale op
1 parent 5420696 commit 2f3a46f

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

examples/train-text-from-scratch/train-text-from-scratch.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -795,6 +795,8 @@ struct ggml_tensor * llama_build_train_graphs(
795795
ggml_build_forward_expand(gb, ggml_scale_inplace(ctx, t36, one));
796796
// input gradient
797797
ggml_build_forward_expand(gb, ggml_scale_inplace(ctx, t36->grad, one));
798+
// KQ_pos
799+
ggml_build_forward_expand(gb, ggml_scale_inplace(ctx, KQ_pos, one));
798800
GGML_ASSERT(t36->grad->data == NULL && !ggml_is_view(t36->grad));
799801
ggml_allocr_alloc(alloc, t36->grad);
800802
// gradient tensors (will be set to zero by ggml_graph_reset)

0 commit comments

Comments
 (0)