Skip to content

llava-cli: fix base64 prompt #7248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 13, 2024
Merged

llava-cli: fix base64 prompt #7248

merged 1 commit into from
May 13, 2024

Conversation

Adriankhl
Copy link
Contributor

@Adriankhl Adriankhl commented May 13, 2024

The multiple image PR #6969 breaks base64 prompt processing in llava-cli

Without this change, the main function just iterate through image file paths, ignore base64 code in prompt, you have to add an empty image path --image "" as a workaround

.\bin\llava-cli.exe -m ..\..\..\models\llava-phi-3-mini-int4.gguf --mmproj ..\..\..\models\llava-phi-3-mini-mmproj-f16.gguf -c 4096 -p "<img src=\`"data:image/jpeg;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAARklEQVR4nGLpPPSRgRSw2aaBJPVMJKkmA4xaMGrBqAWjFoxaQA3A+NnMjCQN72pkSVI/9INo1IJRC0YtGLVgRFgACAAA//8j+AbTk2+QtAAAAABJRU5ErkJggg==\`">Provide a full description" --image ""

Now this works

.\bin\llava-cli.exe -m ..\..\..\models\llava-phi-3-mini-int4.gguf --mmproj ..\..\..\models\llava-phi-3-mini-mmproj-f16.gguf -c 4096 -p "<img src=\`"data:image/jpeg;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAARklEQVR4nGLpPPSRgRSw2aaBJPVMJKkmA4xaMGrBqAWjFoxaQA3A+NnMjCQN72pkSVI/9INo1IJRC0YtGLVgRFgACAAA//8j+AbTk2+QtAAAAABJRU5ErkJggg==\`">Provide a full description"

@mofosyne mofosyne added bugfix fixes an issue or bug Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix labels May 13, 2024
@mofosyne mofosyne merged commit 30e7033 into ggml-org:master May 13, 2024
teleprint-me pushed a commit to teleprint-me/llama.cpp that referenced this pull request May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix fixes an issue or bug Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants