Skip to content

Commit bd5b5f8

Browse files
committed
feat(granite): Add model params for granite-code-8b
Something isn't quite working with this model yet, but the config should be accurate at this point. Branch: GraniteCodeSupport Signed-off-by: Gabe Goodhart <[email protected]>
1 parent 166acc8 commit bd5b5f8

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"block_size": 128000,
3+
"dim": 4096,
4+
"hidden_dim": 14336,
5+
"n_heads": 32,
6+
"n_local_heads": 8,
7+
"n_layers": 36,
8+
"rope_base": 10000000,
9+
"vocab_size": 49152,
10+
"use_hf_tokenizer": true,
11+
"norm_eps": 0.00001,
12+
"rope_scaling": null,
13+
"attention_bias": true,
14+
"feed_forward_bias": true,
15+
"tie_word_embeddings": true
16+
}

0 commit comments

Comments
 (0)