[Stress Tester] Use new duration measurement capability of stress tester to analyse code completion durations

[ahoppen]

Depends on swiftlang/swift-stress-tester#150

---

Use the new capability of the stress tester to measure the time it took to execute code completion requests, gather that information and report an analysis on how long all code completion requests that were executed during the stress tester run took.

This should allow us to verify changes to code completion don’t introduce significant regressions and will hopefully give us a way to find situations in which code completion is slower than it should be.

---

It looks like CI isn’t able to pick up this change as a related PR to my changes in the main Swift or the stress tester repo, so I can’t run it and see what the final results look like in CI. Instead, here’s a short teaser of what I got from determining the code completion timings locally on the Dollar.swift project from the source compatibility suite.

IMPORTANT:
All measurements printed by this script were done in number of number of
instructions. Measurements in seconds are computed assuming that all machines
execute the same number of instructions per second at all times, which is not
correct. Thus all measurements in seconds should be taken with extreme care.
As they scale linearly with the number of instructions, it is safe to compare
values in seconds output by this script.

---------- All CodeComplete requests ----------

Average: 3e+08 instr (ca. 0.048s)

Histogram:
< 8e+06 instr (ca.  0.001s): ***                                       6.1% (<   6.1%)
< 2e+07 instr (ca.  0.003s): **                                        5.1% (<  11.2%)
< 3e+07 instr (ca.  0.005s):                                           0.8% (<  11.9%)
< 7e+07 instr (ca.  0.010s): *******                                  13.0% (<  24.9%)
< 1e+08 instr (ca.  0.020s):                                           0.0% (<  24.9%)
< 3e+08 instr (ca.  0.040s): **                                        4.7% (<  29.6%)
< 5e+08 instr (ca.  0.080s): **************************************** 68.3% (<  98.0%)
< 1e+09 instr (ca.  0.160s):                                           0.5% (<  98.4%)
< 2e+09 instr (ca.  0.321s):                                           1.5% (<  99.9%)
< 4e+09 instr (ca.  0.642s):                                           0.1% (<  99.9%)
< 9e+09 instr (ca.  1.283s):                                           0.1% (< 100.0%)

---------- 10 slowest files for CodeComplete requests ----------

-- /Users/alex/swift-src/swift-source-compat-suite/project_cache/Dollar/Sources/Dollar.swift --

Average: 4e+08 instr (ca. 0.058s)

Histogram:
< 2e+07 instr (ca.  0.003s): *******                                  11.0% (<  11.0%)
< 3e+07 instr (ca.  0.005s): ****                                      6.2% (<  17.2%)
< 7e+07 instr (ca.  0.010s): ****                                      6.2% (<  23.3%)
< 1e+08 instr (ca.  0.020s):                                           0.0% (<  23.3%)
< 3e+08 instr (ca.  0.040s): *******                                  10.6% (<  33.9%)
< 5e+08 instr (ca.  0.080s): **************************************** 58.6% (<  92.5%)
< 1e+09 instr (ca.  0.160s): **                                        4.0% (<  96.5%)
< 2e+09 instr (ca.  0.321s): *                                         2.6% (<  99.1%)
< 4e+09 instr (ca.  0.642s):                                           0.4% (<  99.6%)
< 9e+09 instr (ca.  1.283s):                                           0.4% (< 100.0%)

-- /Users/alex/swift-src/swift-source-compat-suite/project_cache/Dollar/Sources/AutoBind.swift --

Average: 4e+08 instr (ca. 0.054s)

Histogram:
< 8e+06 instr (ca.  0.001s):                                           1.7% (<   1.7%)
< 2e+07 instr (ca.  0.003s):                                           0.9% (<   2.6%)
< 3e+07 instr (ca.  0.005s):                                           0.0% (<   2.6%)
< 7e+07 instr (ca.  0.010s): ****                                      8.4% (<  11.0%)
< 1e+08 instr (ca.  0.020s):                                           0.0% (<  11.0%)
< 3e+08 instr (ca.  0.040s): ***                                       7.1% (<  18.1%)
< 5e+08 instr (ca.  0.080s): **************************************** 80.4% (<  98.6%)
< 1e+09 instr (ca.  0.160s):                                           0.0% (<  98.6%)
< 2e+09 instr (ca.  0.321s):                                           1.4% (< 100.0%)

-- /Users/alex/swift-src/swift-source-compat-suite/project_cache/Dollar/Sources/AutoCurry.swift --

Average: 3e+08 instr (ca. 0.041s)

Histogram:
< 8e+06 instr (ca.  0.001s): *******                                  11.7% (<  11.7%)
< 2e+07 instr (ca.  0.003s): ****                                      7.2% (<  18.9%)
< 3e+07 instr (ca.  0.005s):                                           0.0% (<  18.9%)
< 7e+07 instr (ca.  0.010s): ************                             18.9% (<  37.8%)
< 1e+08 instr (ca.  0.020s):                                           0.0% (<  37.8%)
< 3e+08 instr (ca.  0.040s):                                           1.1% (<  38.9%)
< 5e+08 instr (ca.  0.080s): **************************************** 60.0% (<  98.8%)
< 1e+09 instr (ca.  0.160s):                                           0.0% (<  98.8%)
< 2e+09 instr (ca.  0.321s):  

[bnbarham]

"emirically" :)

[bnbarham]

Could probably just round it if it doesn't really matter anyway 🤷

[ahoppen]

OK, I’ve removed the decimal digits. Not sure if rounding it any further gives much benefit. Might as well use the most accurate number we have at the moment.

[shahmishal]

LGTM!