Skip to content

[swift-evolve] More source order dependencies in tests #21427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Dec 19, 2018

Conversation

beccadax
Copy link
Contributor

Grab bag of test changes for swift-evolve:

  • Several ParsableInterface tests involving RawRepresentable could fail if that protocol's members were shuffled. Switch those to use CHECK-DAG instead of CHECK-NEXT.
  • Use UNSUPPORTED instead of XFAIL to skip source-order-dependent tests. swift-evolve makes random changes, so most of these tests become flaky rather than failing reliably as XFAIL requires.
  • Mark all SourceKit tests as unsupported. Many of these tests are inherently source-order-dependent and it's not worth triaging the failures at this point.
  • One of the two issues preventing us from running the ABI checker test with swift-evolve has been fixed; delete the comment citing it.

Tagging @jrose-apple to make sure the ParsableInterface tests aren't made useless by this change.

Several tests implicitly depended on the order of declarations in RawRepresentable. This change loosens their requirements.
These tests may not *reliably* fail with shuffled standard libraries, so we can’t use XFAIL, which fails if the test succeeds.
A lot of these are source-order-dependent and there’s no particular reason to think they’re relevant to resilience testing.
One of the two issues preventing us from running test/api-digester/stability-stdlib-abi.swift in swift-evolve tests has been fixed; delete the comment referencing it. There’s another bug that still needs to be fixed, so we still aren’t running this yet.
@beccadax beccadax requested a review from jrose-apple December 19, 2018 00:40
@beccadax
Copy link
Contributor Author

@swift-ci please smoke test

@beccadax
Copy link
Contributor Author

@swift-ci please smoke test and merge

@swift-ci swift-ci merged commit 14dfa75 into swiftlang:master Dec 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants