Skip to content

Add tokenizer.ggml.pre to gguf-new-metadata.py #7627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 30, 2024
Merged

Add tokenizer.ggml.pre to gguf-new-metadata.py #7627

merged 2 commits into from
May 30, 2024

Conversation

Galunid
Copy link
Collaborator

@Galunid Galunid commented May 29, 2024

It can be useful to fix older models without need to reconvert

@github-actions github-actions bot added the python python script changes label May 29, 2024
@Galunid Galunid mentioned this pull request May 29, 2024
Merged
@mofosyne mofosyne added the Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix label May 29, 2024
@mofosyne mofosyne added the merge ready indicates that this may be ready to merge soon and is just holding out in case of objections label May 29, 2024
@Galunid Galunid merged commit eb57fee into ggml-org:master May 30, 2024
16 of 17 checks passed
@Galunid Galunid deleted the gguf-new-tokenizer branch May 30, 2024 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mofosyne mofosyne mofosyne approved these changes

Assignees
No one assigned
Labels
merge ready indicates that this may be ready to merge soon and is just holding out in case of objections python python script changes Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Galunid @mofosyne