Skip to content

build-using-self use a self build of swift-test instead #8548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 23, 2025
Merged

build-using-self use a self build of swift-test instead #8548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
03e57b1
build-using-self use a self build of swift-test instead
dschaefer2 Apr 22, 2025
f0de0af
Add missing products from swift build.
dschaefer2 Apr 22, 2025
20458c0
Don't build testing helper on non-Darwin.
dschaefer2 Apr 22, 2025
0df58ba
Change Windows fix to use a smaller count.
dschaefer2 Apr 22, 2025
13c3be0
More pieces missing, just build everything.
dschaefer2 Apr 22, 2025
68efcab
Clean up calls to build products. Try smaller buffer.
dschaefer2 Apr 22, 2025
390b512
Missed an f
dschaefer2 Apr 22, 2025
f1c9dbe
Hang not simply resolved. Turn off Windows tests for now.
dschaefer2 Apr 22, 2025
3a621aa
Fix up the name of the custom build directory.
dschaefer2 Apr 23, 2025
87750c2
Change the Integration tests to use the same env var
dschaefer2 Apr 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion IntegrationTests/Sources/IntegrationTestSupport/Helpers.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ public let lldb: AbsolutePath = {
}()

public let swiftpmBinaryDirectory: AbsolutePath = {
if let environmentPath = ProcessInfo.processInfo.environment["SWIFTPM_BIN_DIR"] {
if let environmentPath = ProcessInfo.processInfo.environment["SWIFTPM_CUSTOM_BIN_DIR"] {
return try! AbsolutePath(validating: environmentPath)
}

Expand Down
39 changes: 18 additions & 21 deletions Utilities/build-using-self
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ def is_on_darwin() -> bool:
def set_environment(*, swiftpm_bin_dir: pathlib.Path,) -> None:
os.environ["SWIFTCI_IS_SELF_HOSTED"] = "1"

# Set the SWIFTPM_BIN_DIR path
os.environ["SWIFTPM_BIN_DIR"] = str(swiftpm_bin_dir)
# Set the SWIFTPM_CUSTOM_BIN_DIR path
os.environ["SWIFTPM_CUSTOM_BIN_DIR"] = str(swiftpm_bin_dir)

# Ensure SDKROOT is configure
if is_on_darwin():
Expand Down Expand Up @@ -133,6 +133,7 @@ def run_bootstrap(swiftpm_bin_dir: pathlib.Path) -> None:

def main() -> None:
args = get_arguments()
ignore = "-Xlinker /ignore:4217" if os.name == "nt" else ""
logging.getLogger().setLevel(logging.DEBUG if args.is_verbose else logging.INFO)
logging.debug("Args: %r", args)

Expand All @@ -144,31 +145,27 @@ def main() -> None:
shlex.split("swift --version"),
)

# call(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: It's always nice seeing commented code being removed!

# shlex.split("swift package reset"),
# )
call(
shlex.split("swift package update"),
)
call(
shlex.split(f"swift build --configuration {args.config}"),
)
swift_testing_arg= "--enable-swift-testing" if args.enable_swift_testing else ""
xctest_arg= "--enable-xctest" if args.enable_swift_testing else ""
call(
shlex.split(f"swift test --configuration {args.config} --parallel {swift_testing_arg} {xctest_arg}"),
shlex.split(f"swift build --configuration {args.config} {ignore}"),
)

with change_directory(REPO_ROOT_PATH / "IntegrationTests"):
call(
shlex.split("swift package update"),
)
call(
shlex.split(
f"{swiftpm_bin_dir / 'swift-test'} --parallel",
posix=(os.name == "posix"), # must be set correctly, otherwhsie shlex.split("C:\\Foo\\bar") become ['CFoobar']
),
)
if os.name != "nt": # turn off for Windows until we get the hang resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know what may be causing the hangs? It's quite coincidental the hangs started to occurs after the self hosted windows pipeline was changes from using the nightly toolchain, to the nightly 6.1 toolchain.

The self-hosted windows toolchain used the official docker image. should we try to see if the released 6.1 image has the same hangs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the 6.1 swiftpm has a bug in it. It's still there in 6.2. We should probably fix it.

swift_testing_arg= "--enable-swift-testing" if args.enable_swift_testing else ""
xctest_arg= "--enable-xctest" if args.enable_swift_testing else ""
call(
shlex.split(f"swift run swift-test --configuration {args.config} --parallel {swift_testing_arg} {xctest_arg} --scratch-path .test {ignore}"),
)

integration_test_dir = REPO_ROOT_PATH / "IntegrationTests"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to make this code more robust to changes, it would be better to change the change_directory(REPO_ROOT_PATH context manager to return the directory, and use that path here instead of REPO_ROOT_PATH.

That is, change with change_directory(REPO_ROOT_PATH): to with change_directory(REPO_ROOT_PATH) as my_path: and then set integration_test_dir = my_path / "IntegrationTests" or just set it to a relative path (ie: integration_test_dir = Path("./IntegrationTests")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll be back here to re-enable them on Windows. We can clean things up then. Let's just focus on re-enabling the Windows CI for now.

call(
shlex.split(f"swift package --package-path {integration_test_dir} update"),
)
call(
shlex.split(f"swift run swift-test --package-path {integration_test_dir} --parallel {ignore}"),
)

if is_on_darwin():
run_bootstrap(swiftpm_bin_dir=swiftpm_bin_dir)
Expand Down
2 changes: 1 addition & 1 deletion Utilities/test-toolchain
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ def get_env(args):
if args.lldb_path:
env['LLDB_PATH'] = args.lldb_path
if args.swiftpm_bin_dir:
env["SWIFTPM_BIN_DIR"] = args.swiftpm_bin_dir
env["SWIFTPM_CUSTOM_BIN_DIR"] = args.swiftpm_bin_dir

return env

Expand Down