RUBY-1250 Run DriverBench benchmarks in Evergreen #2827
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a large PR, primarily new files, though most of the new files are small.
The main files to consider are:
DriverBench::Suite
in profile/driver_bench/suite.rbDriverBench::Base
in profile/driver_bench/base.rbThe benchmarks are run via
DriverBench::Suite
, which accumulates the results, computes the scores, and ultimately emits the results as a JSON file. The JSON file is then sent to the evergreen performance plugin viaperf.send
.Each benchmark is ultimately a subclass of
DriverBench::Base
. The core of the benchmark is implemented in thedo_task
method, whichDriverBench::Base#run_benchmark
invokes. Other lifecycle hooks (setup
,before_task
,after_task
, andteardown
) are used as described in the specification, to manage the state of the benchmark.