[benchmark] Expand ParseInt benchmarks #29988
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@swift-ci Please benchmark |
Performance: -O
Code size: -O
Performance: -Osize
Code size: -Osize
Performance: -Onone
Code size: -swiftlibs
How to read the dataThe tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.If you see any unexpected regressions, you should consider fixing the Noise: Sometimes the performance results (not code size!) contain false Hardware Overview |
|
@swift-ci Please benchmark |
eeckstein
requested changes
Feb 21, 2020
There was a problem hiding this comment.
Is it possible to pick a few (maybe ~ 5) representative benchmarks which cover most of the scenarios?
We should avoid inflating the number of benchmarks too much, because we want to keep the total runtime of benchmarking on an acceptable level. If we add exhaustive benchmarks for each small piece of the library and the language it's just too much.
I suggest to pick a few and add the .skip tag to all others. If someone wants to run all the integer parsing benchmarks, it's still possible with the -skipTags option. See ExistentialPerformance.swift as an example.
Also, we should avoid using gyb, especially in the benchmarks. Eventually we want to convert the benchmarks to a swiftpm package, where we cannot use gyb. Beside that, gyb has other problems (e.g. it complicates debugging).
Performance: -O
Code size: -O
Performance: -Osize
Code size: -Osize
Performance: -Onone
Code size: -swiftlibs
How to read the dataThe tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.If you see any unexpected regressions, you should consider fixing the Noise: Sometimes the performance results (not code size!) contain false Hardware Overview |
|
@eeckstein Thanks, I added Good to know about the gyb usage. For this one do you want to just leave it as is for now, or delete the gyb file? (or another suggestion) |
|
@swift-ci Please benchmark |
Performance: -O
Code size: -O
Performance: -Osize
Code size: -Osize
Performance: -Onone
Code size: -swiftlibs
How to read the dataThe tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.If you see any unexpected regressions, you should consider fixing the Noise: Sometimes the performance results (not code size!) contain false Hardware Overview |
Unfortunately no. But you can run benchmark/scripts/run_smoke_bench locally on your machine. You'll need to modify run_smoke_bench to pass the right options to the benchmark executable.
My preference is to not add the gyb file. |
|
There has been an effort to de-gyb, so it would certainly be preferable to not introduce new gyb based sources. |
PatrickPijnappel
changed the title
|
@eeckstein @compnerd 👍 I've removed the gyb file. Let me know if you think we should try to generalize the test implementation functions as well. |
PatrickPijnappel
force-pushed
the
expand-parse-int-benchmarks
branch
from
fb91091 to
c73b4ee
Compare
|
I'm also now skipping all but five specific tests, that should be representative both of the different code paths and fairly common real-world use cases. |
|
@swift-ci Please smoke test |
|
@swift-ci Please benchmark |
Performance: -O
Code size: -O
Performance: -Osize
Code size: -Osize
Performance: -Onone
Code size: -swiftlibs
How to read the dataThe tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.If you see any unexpected regressions, you should consider fixing the Noise: Sometimes the performance results (not code size!) contain false Hardware Overview |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Expands the integer parsing benchmarks to distinguish between:
These are in preparation for a full rewrite of integer parsing (which I finally found some time to finish up), where performance will clearly vary along these axes.