Skip to content

Commit 08959c8

Browse files
committed
Fix vocab space conversion logic
1 parent f7e61fd commit 08959c8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

convert-llama-ggmlv3-to-gguf.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ def add_vocab(self, gguf_writer):
182182
toktypes = []
183183
for (tokid, (vbytes, vscore)) in enumerate(self.model.vocab.items):
184184
tt = 1
185-
if len(vbytes) > 1 and vbytes[0] == 32:
185+
if len(vbytes) > 0 and vbytes[0] == 32:
186186
vbytes = vbytes.replace(b' ', b'\xe2\x96\x81')
187187
elif len(vbytes) == 0:
188188
tt = 3

0 commit comments

Comments
 (0)