Skip to content

Commit 91e5dd2

Browse files
author
Danqing Wang (MPK)
committed
change use_hf_rope to False
1 parent 34c5dee commit 91e5dd2

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
{
22
"dim": 576,
33
"ffn_dim_multiplier": 1,
4-
"hidden_dim": 576,
4+
"hidden_dim": 1536,
55
"n_heads": 9,
66
"n_kv_heads": 3,
77
"n_layers": 30,
88
"norm_eps": 1e-05,
99
"rope_theta": 10000.0,
1010
"use_scaled_rope": false,
1111
"vocab_size": 49152,
12-
"use_hf_rope": true,
12+
"use_hf_rope": false,
1313
"attention_qkv_bias": false
1414
}

0 commit comments

Comments
 (0)