IRGen, benchmarks: add an option -align-module-to-page-size for benchmarking #18318
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The option aligns all modules to the page size. This help giving more consistent results when doing performance testing with the swift benchmark suite. It solves the problem that benchmarks which compile down to identical code give different runtime data because of different alignment of the code within a page.
…f supported by the compiler To stabilize the benchmark results.
|
@swift-ci smoke benchmark |
|
@swift-ci smoke test |
Build comment file:Optimized (O)Regression (6)
Improvement (6)
No Changes (447)
Hardware Overview |
|
@eeckstein I have also seen weird alignment things with environment variables. I think it is b/c the os puts the env on the stack so all subsequent stack allocations can get their alignment changed if the env variables change... but my memory might be wrong You may want to look at that as well if you are working in this area. |
|
This shouldn't be a problem (and I didn't see this) as long as both benchmark runs have the same environment. And that's usually the case. But thanks, it's a good point. |
|
I am saying that it would be easy to write either a wrapper script that detects bad behavior or some sort of simulation script that standardizes the environment no matter where it runs. Then we can at least have a way of identifying (or eliminating) the cause of such a regression without needing to do anything. That being said, I don't know how important it is in practice. I think I have seen it a couple of times before in other test suites. Seems simple enough that it may be worth defining away. |
|
How did this issue manifest? Can you give me some pointers on how could I reproduce the problem? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The option aligns all modules to the page size. This help giving more consistent results when doing performance testing with the swift benchmark suite.
It solves the problem that benchmarks which compile down to identical code give different runtime data because of different alignment of the code within a page.
Also compile the benchmarks with the new option, if supported by the compiler