Skip to content

Commit d49e252

Browse files
committed
add OLMo checksum to pre-tokenizer function
1 parent dba3d24 commit d49e252

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

convert-hf-to-gguf.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,9 @@ def get_vocab_base_pre(self, tokenizer) -> str:
314314
if chkhsh == "9c2227e4dd922002fb81bde4fc02b0483ca4f12911410dee2255e4987644e3f8":
315315
# ref: https://huggingface.co/CohereForAI/c4ai-command-r-v01
316316
res = "command-r"
317+
if chkhsh == "b6dc8df998e1cfbdc4eac8243701a65afe638679230920b50d6f17d81c098166":
318+
# ref: https://huggingface.co/allenai/OLMo-1.7-7B-hf
319+
res = "olmo"
317320

318321
if res is None:
319322
logger.warning("\n")

0 commit comments

Comments
 (0)