scripts: fix compare-llama-bench commit hash logic #11891
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently
compare-llama-bench.py
assumes that all rows have the same commit hash length and simply retrieves this length from the first row. This change seems to have been made in #10714 without further documentation as to why, I assume it's because the llama.cpp commit hash has a variable length. By default it's 8 hexadecimal characters but if there is a hash collision the length ofgit rev-parse --short
is increased to yield a unique prefix. On the master branch this does not seem to be necessary but on a local repository the hash can be longer.compare-llama-bench.py
thus may receive hashes that are longer than 8 characters so the trimming to 8 characters that I originally implemented can fail. However, for my workflows this fix does not work. I have a desktop where I do most of my development and on this machine 9 characters are needed for unique hashes. However, I have other machines that I primarily use for testing/batch jobs where the hash is only 8 characters. Sosqlite
files that contain data from multiple machines (or from the same machine when the hash length changes) may not work correctly. This PR generalizes the logic to cover this use case by shortening the prefixes and making them comparable. It is in principle possible that this will cause incorrect results due to hash collisions but the probability is extremely low. Still, if this is necessary the script prints a warning, recommending that old commits be purged.