Skip to content

Commit 386ae58

Browse files
committed
[benchmark] Document deterministic hashing requirement
1 parent e73ef1f commit 386ae58

File tree

1 file changed

+24
-1
lines changed

1 file changed

+24
-1
lines changed

benchmark/README.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -199,7 +199,30 @@ benchmarks will be compiled with -Onone!**
199199
* `$ ./Benchmark_O --tags=Dictionary`
200200
* `$ ./Benchmark_O --skip-tags=unstable,skip,validation`
201201

202-
### Note
202+
### Deterministic Hashing Requirement
203+
204+
To run benchmarks, you'll need to disable randomized hash seeding by setting the
205+
`SWIFT_DETERMINISTIC_HASHING` environment variable to `1`. (You only need to do
206+
this when running the benchmark executables directly -- the driver script does
207+
this for you automatically.)
208+
209+
* `$ env SWIFT_DETERMINISTIC_HASHING=1 ./Benchmark_O --num-iters=1 --num-samples=1`
210+
211+
This makes for more stable results, by preventing random hash collision changes
212+
from affecting benchmark measurements. Benchmark measurements start by checking
213+
that deterministic hashing is enabled and they fail with a runtime trap when it
214+
isn't.
215+
216+
If for some reason you want to run the benchmarks using standard randomized
217+
hashing, you can disable this check by passing the
218+
`--allow-nondeterministic-hashing` option to the executable.
219+
220+
* `$ ./Benchmark_O --num-iters=1 --num-samples=1 --allow-nondeterministic-hashing`
221+
222+
This will affect the reliability of measurements, so this is not recommended.
223+
224+
### Benchmarking by Numbers
225+
203226
As a shortcut, you can also refer to benchmarks by their ordinal numbers.
204227
These are printed out together with benchmark names and tags using the
205228
`--list` parameter. For a complete list of all available performance tests run

0 commit comments

Comments
 (0)