use std::chrono::system_clock::now() for random seed #6953
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the default random seed is set by
time(NULL)
, which returns an integer number of seconds. If we callllama_sampling_init()
many times within a single second, then all of the sampling contexts get the same random seed. Such behavior can come as an unpleasant surprise. I encountered it today while attempting to generate many samples via ollama, which makes use of an adapted version of llama.cpp/examples/server/server.cpp.We can avoid the problem by using a higher-precision clock to set the random seed. On my (Linux) system,
std::chrono::system_clock
measures its ticks in nanoseconds. On Windows/Visual Studio, a tick is 100 nanoseconds.static_cast<uint32_t>
then truncates the higher-order bits.An alternative to this clock-based approach would be to use
std::random_device
, as is already done in utils.hpp. I opted against that approach in this PR, however, because, as noted here, on some platformsstd::random_device
is deterministic.