Skip to content

Scale test improvements #11892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 13, 2017
Merged

Conversation

graydon
Copy link
Contributor

@graydon graydon commented Sep 13, 2017

Two categories of improvement to scale-test:

  1. Revive the direct linear regression code as a first-pass (not in log-space) to catch cases that best-fit linearly with no higher-degree exponent term. These cases occur in practice pretty often when someone has a process that's linear "once it gets going" with a sub- or super-linear first point; these presently cause the simplex-algorithm to continue to tweak the polynomial model's exponent to find a curve that fits the extra point, rather than just tilting a line, and it winds up with a bad overall fit (fails, requires more data, etc). Checking a line first (as with the constant early-check) is worthwhile. This also lets us tighten bounds a bit on tests.

  2. Complete some work started in 93a832d to make scale-test use the same stats-dir parsing code as process-stats-dir, to use the -stats-output-dir reporting mode of the compiler (not -Xllvm -stats-json) when it's available. This gives us more stats, and also usable scale-testing on non-assert (i.e. end-user) compilers, for the stats we ship in that mode.

@graydon graydon requested review from xedin, rudkx and huonw September 13, 2017 05:46
@graydon
Copy link
Contributor Author

graydon commented Sep 13, 2017

@swift-ci please test and merge

Copy link
Contributor

@xedin xedin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to revert the tests I've modified after changing shrinking, to see if this helps, and will take a look at the code, thanks @graydon !

@swift-ci swift-ci merged commit 33b673e into swiftlang:master Sep 13, 2017
@graydon graydon deleted the scale-test-improvements branch September 13, 2017 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants