Skip to content

Support a Unified Test Binary #986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

plemarquand
Copy link
Contributor

With the merge of swiftlang/swift-package-manager#7766 a swift build --build-tests produces a single binary that contains both XCTests and swift-testing tests.

Update the build configs when debugging tests to use this new unified test binary if it exists.

To determine if we're running a unified binary on macOS we can check for the presence of the swiftpm-testing-helper, which is used to run the swift-testing tests in the unified binary.

This utility isn't required on Windows and Linux as the test binary can be invoked directly. To determine if we're running a unified binary we check if the latest build produced a .swift-testing binary in the build folder. If it doesn't exist, or if it does exist but its mtime is older than the last build start time, we know its a unified binary. The mtime check is required in case the user updates their toolchain to a version that produces a unified binary but still has a .swift-testing binary in their .build folder left over from an old build.

In the future we can remove these checks once we're confident users on 6.0 and main toolchains have moved on to versions that generate a unified test binary. Tentatively I propose this to be around when Swift 6.0 is finalized and released.

This change only affects debugging tests. All other testing code paths use swift test which lets SwiftPM automatically handle the switch to the new unified binary.

@plemarquand plemarquand marked this pull request as ready for review July 25, 2024 15:59
@plemarquand plemarquand force-pushed the unified-test-executable-support branch 4 times, most recently from f29db87 to 6551f60 Compare July 25, 2024 18:23
@adam-fowler
Copy link
Contributor

Are we certain swiftpm-testing-helper is only used to run swift-testing tests. I'm sure I've seen that process name before the existence of swift-testing. I maybe wrong.

@plemarquand
Copy link
Contributor Author

plemarquand commented Jul 26, 2024

@adam-fowler I consulted @grynspan who mentioned looking for the swiftpm-testing-helper would be an appropriate way to check if SwiftPM had moved to the new unified test executable. Both that tool and the unified binaries landed in approximately the same nightly toolchain.

@grynspan
Copy link

There's a window of maybe two nightly builds where the tool is available but the new build mechanism is not yet implemented. I think that's narrow enough we don't need to worry about it, and the solution for anybody affected is to slightly upgrade or downgrade their nightly toolchains. :)

@grynspan
Copy link

Are we certain swiftpm-testing-helper is only used to run swift-testing tests. I'm sure I've seen that process name before the existence of swift-testing. I maybe wrong.

There's a swiftpm-xctest-helper on macOS that's used by swift test list. Different tool. :)

With the merge of swiftlang/swift-package-manager#7766
a `swift build --build-tests` produces a single binary that contains
both XCTests and swift-testing tests.

Update the build configs when debugging tests to use this new unified
test binary if it exists.

To determine if we're running a unified binary on macOS we can check for
the presence of the swiftpm-testing-helper, which is used to run the
swift-testing tests in the unified binary.

This utility isn't required on Windows and Linux as the test binary can
be invoked directly. To determine if we're running a unified binary we
check if the latest build produced a .swift-testing binary in the build
folder. If it doesn't exist, or if it does exist but its `mtime` is
older than the last build start time, we know its a unified binary. The
`mtime` check is required in case the user updates their toolchain to a
version that produces a unified binary but still has a .swift-testing
binary in their .build folder left over from an old build.

In the future we can remove these checks once we're confident users on
6.0 and main toolchains have moved on to versions that generate a
unified test binary. Tentatively I propose this to be around when
Swift 6.0 is finalized and released.

This change only affects debugging tests. All other testing code paths
use `swift test` which lets SwiftPM automatically handle the switch to
the new unified binary.
@plemarquand plemarquand force-pushed the unified-test-executable-support branch from 6551f60 to 9f92d15 Compare July 26, 2024 14:45
@adam-fowler
Copy link
Contributor

There's a swiftpm-xctest-helper on macOS that's used by swift test list. Different tool. :)

Yeah you’re probably right

@plemarquand plemarquand merged commit d0c543f into swiftlang:main Aug 7, 2024
8 checks passed
@plemarquand plemarquand deleted the unified-test-executable-support branch August 7, 2024 15:30
@grynspan
Copy link

grynspan commented Aug 7, 2024

Woohoo!

@plemarquand plemarquand restored the unified-test-executable-support branch August 7, 2024 19:11
@award999
Copy link
Contributor

award999 commented Aug 7, 2024

Verified

@plemarquand plemarquand deleted the unified-test-executable-support branch August 7, 2024 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants