Skip to content

Enable swift-testing by default. #7591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

grynspan
Copy link
Contributor

This PR sets the default state of --{enable,disable}-experimental-swift-testing to enabled.

This PR is speculative.

This PR sets the default state of `--{enable,disable}-experimental-swift-testing` to
enabled.

This PR is speculative.
@grynspan grynspan added swift test Changes impacting `swift test` tool swift build Changes impacting `swift build` swift package init Package templates command changes command-line interface labels May 24, 2024
@grynspan grynspan self-assigned this May 24, 2024
@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test

1 similar comment
@grynspan
Copy link
Contributor Author

@swift-ci please test

@Jerry-Carter
Copy link

Hoping to see this change made soon. Some general comments, mostly related to this issue.

The current behavior is rather unintuitive. WWDC24 suggests that Swift Testing is now the preferred test framework, but these tests aren't actually executed unless the command line argument --enable-swift-testing is given. The XCTest cases are always executed. When only Swift Testing is used, the build processes ends with 'Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds' which is confusing and false. At a minimum, may I suggest changing

Test run started  → Swift Test execution started
Test Suite 'All tests'  → XCTest Suite 'All tests'
Executed 0 tests, with 0 failures  → XCTest complete.  Executed 0 tests, with 0 failures

so that the stages are a bit clearer.

I understand from the forum discussion that the code is compiled, then the Swift Testing cases are executed (if the flag is specified), and then the XCTest cases are executed with the 'Executed 0 tests' message originating from the later. However, it appears that plugins are re-executed between the Swift Testing and XCTest stages which (1) feels unnecessary and (2) breaks up the results from unit testing.

Finally, I appreciate that the unit tests are being performed in parallel but I wish the output from Suites was grouped somehow rather than interwoven. In my mind, Suites represent the fundamental conceptual units with the underlaying Test functions being supporting details. I'd love to see something like

▢ Test run started.
↳ Testing Library Version: 75.9 (arm64e-apple-macos13.0)
▢ Suite "LongTests" started
      ▢ Test dataDiscombobulation() started.
      ☑️ Test reskewingWidgets() passed after 52.182 seconds.
 ☑️ Suite "ShortTests" passed after 0.390 seconds.
      ☑️ Test checkName() passed after 0.02 seconds.
      ☑️ Test sortOrder() passed after 0.389 seconds.

at runtime, but would also welcome an option to generate a final test report.

@grynspan
Copy link
Contributor Author

grynspan commented Jul 5, 2024

Thank you for your feedback, @Jerry-Carter. The current behaviour on macOS (where you need to specify the flag) is a known issue with Xcode 16's built-in Swift toolchain. We are working to resolve it.

@grynspan
Copy link
Contributor Author

Superseded by #7766.

@grynspan grynspan closed this Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
command-line interface swift build Changes impacting `swift build` swift package init Package templates command changes swift test Changes impacting `swift test` tool
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants