Skip to content

Commit d37ddcf

Browse files
committed
Merge pull request #1514 from PatrickPijnappel/patch-1
Clarify usage of N in benchmark README
2 parents 5bc2749 + 4754285 commit d37ddcf

File tree

1 file changed

+8
-4
lines changed

1 file changed

+8
-4
lines changed

benchmark/README.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -173,10 +173,14 @@ To add a new multiple file test:
173173
The generator script looks for functions prefixed with `run_` in order to
174174
populate `utils/main.swift`.
175175

176-
Each iteration of work performed in the for-loop below should run in under a
177-
few milliseconds. The benchmark driver will automatically calculate the
178-
necessary number of iterations to run each benchmark in approximately one
179-
second.
176+
The benchmark driver will measure the time taken for `N = 1` and automatically calculate
177+
the necessary number of iterations `N` to run each benchmark in approximately one second,
178+
so the test should ideally run in a few milliseconds for `N = 1`. If the test contains
179+
any setup code before the loop, ensure the time spent on setup is insignificant compared to
180+
the time spent inside the loop (for `N = 1`) – otherwise the automatic calculation of `N` might be
181+
significantly off and any performance gains/regressions will be masked by the fixed setup time.
182+
If needed you can multiply N by a fixed amount (e.g. `1...100*N`) to achieve this.
183+
180184

181185
**Performance Test Template**
182186

0 commit comments

Comments
 (0)