Skip to content

Commit 77cfcb4

Browse files
committed
squash! finetune: rename feed-forward tensors (w1/w2/w3)
This commit updates the README.md file to reflect the new names of the feed-forward tensors. Signed-off-by: Daniel Bevenius <[email protected]>
1 parent 039f1ce commit 77cfcb4

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

examples/finetune/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -80,9 +80,9 @@ The LORA rank can be configured for each model tensor type separately with these
8080
--rank-wk N LORA rank for wk tensor (default 4)
8181
--rank-wv N LORA rank for wv tensor (default 4)
8282
--rank-wo N LORA rank for wo tensor (default 4)
83-
--rank-w1 N LORA rank for w1 tensor (default 4)
84-
--rank-w2 N LORA rank for w2 tensor (default 4)
85-
--rank-w3 N LORA rank for w3 tensor (default 4)
83+
--rank-ffn_gate N LORA rank for ffn_gate tensor (default 4)
84+
--rank-ffn_down N LORA rank for ffn_down tensor (default 4)
85+
--rank-ffn_up N LORA rank for ffn_up tensor (default 4)
8686
```
8787

8888
The LORA rank of 'norm' tensors should always be 1.

0 commit comments

Comments
 (0)