use std::random_device{}() for default random seed #6962
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the default random seed is set by
time(NULL)
, which returns an integer number of seconds. If we callllama_sampling_init()
many times within a single second, then all of the sampling contexts get the same random seed. Such behavior can come as an unpleasant surprise. I encountered it recently while attempting to generate many samples via ollama, which makes use of an adapted version of llama.cpp/examples/server/server.cpp.We can avoid the problem by using
std::random_device
, as is already done in utils.hpp.Note that on old versions of libstd++ on vxworks and mingw
std::random_device
was sometimes deterministic, however that was considered a bug and was fixed.I don't believe there should be any performance issues with using
std::random_device{}()
. On my Linux machine, it does not even require a system call, I think because it's using one of the special CPU instructionsRDSEED
orRDRAND
.