Move stress tester and swiftevolve to new swift_build_support build infrastructure #27580

ahoppen · 2019-10-08T20:39:42Z

I’m planning to move SwiftSyntax to the new build infrastructure in swift_build_support and started off by migrating the SourceKit stress tester over.

ahoppen · 2019-10-08T20:40:51Z

@swift-ci Please test
@swift-ci Please Build Toolchain macOS Platform
@swift-ci Please Build Toolchain Linux Platform

ahoppen · 2019-10-08T20:43:03Z

Apparently, I’m not able to start multiple swift-ci commands in one comment. So here we go again.

@swift-ci Please test

ahoppen · 2019-10-08T20:43:08Z

@swift-ci Please Build Toolchain macOS Platform

swift-ci · 2019-10-08T21:29:42Z

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - 25c842ba4b19d18294122e7cc47a891bf31e1833

Install command
tar zxf swift-PR-27580-294-ubuntu16.04.tar.gz
More info

swift-ci · 2019-10-08T21:52:12Z

Build failed
Swift Test Linux Platform
Git Sha - 25c842ba4b19d18294122e7cc47a891bf31e1833

swift-ci · 2019-10-08T21:54:46Z

Build failed
Swift Test OS X Platform
Git Sha - 25c842ba4b19d18294122e7cc47a891bf31e1833

benlangmuir

I think we should couple this with a change to simplify the stress tester's build-script-helper. There shouldn't be any reason to pass all these tools separately.

E.g.

--toolchain

instead of

-- swiftc-exec
--swift-build-exec
--swift-test-exec
--syntax-parser-lib-dir
--sourcekitd-dir
--swiftsyntax-dir

utils/build-script

ahoppen · 2019-10-10T17:24:05Z

@swift-ci Please tests

swift-ci · 2019-10-10T17:26:46Z

Build failed
Swift Test Linux Platform
Git Sha - 25c842ba4b19d18294122e7cc47a891bf31e1833

swift-ci · 2019-10-10T17:28:36Z

Build failed
Swift Test OS X Platform
Git Sha - 25c842ba4b19d18294122e7cc47a891bf31e1833

utils/swift_build_support/swift_build_support/products/skstresstester.py

shahmishal · 2019-10-14T17:42:22Z

@swift-ci Please Build Toolchain macOS Platform

ahoppen · 2019-10-18T23:45:41Z

@swift-ci Please test

ahoppen · 2019-10-18T23:46:22Z

preset=buildbot_skstresstester_macos
@swift-ci Please test with preset macOS Platform

swift-ci · 2019-10-18T23:47:58Z

Build failed
Swift Test Linux Platform
Git Sha - 0c8e1c7a738309815494ba6b856b4ac99e3027a5

swift-ci · 2019-10-18T23:50:05Z

Build failed
Swift Test OS X Platform
Git Sha - 0c8e1c7a738309815494ba6b856b4ac99e3027a5

ahoppen · 2019-10-19T04:42:34Z

swiftlang/swift-stress-tester#88

preset=buildbot_skstresstester_macos
@swift-ci Please test with preset macOS Platform

ahoppen · 2019-10-20T02:40:19Z

@swift-ci Please test

swift-ci · 2019-10-20T02:42:39Z

Build failed
Swift Test Linux Platform
Git Sha - 64c3435cb1ac0ec4f3cada698f08bb88f2a12ae1

swift-ci · 2019-10-20T02:45:14Z

Build failed
Swift Test OS X Platform
Git Sha - 64c3435cb1ac0ec4f3cada698f08bb88f2a12ae1

ahoppen · 2019-10-20T03:20:21Z

swiftlang/swift-stress-tester#88

preset=buildbot_skstresstester_macos
@swift-ci Please test with preset macOS Platform

ahoppen · 2019-10-20T15:52:40Z

swiftlang/swift-stress-tester#88

preset=buildbot_swiftevolve_macos
@swift-ci Please test with preset macOS Platform

ahoppen · 2019-10-20T15:53:57Z

@swift-ci Please build toolchain macOS

ahoppen · 2019-10-21T00:45:43Z

swiftlang/swift-stress-tester#88

@swift-ci Please build toolchain macOS

swift-ci · 2019-10-21T02:29:32Z

macOS Toolchain
Download Toolchain
Git Sha - 92a0e20

Install command
tar -zxf swift-PR-27580-402-osx.tar.gz --directory ~/

ahoppen · 2019-10-21T03:55:30Z

Manually verified that the files in the toolchain match the ones in the open source toolchains.

benlangmuir

To clarify, this is only for macOS still right? I assume Linux support is for a later change?

benlangmuir · 2019-10-21T17:06:05Z

utils/build_swift/driver_arguments.py

@@ -1026,6 +1036,17 @@ def create_argument_parser():
    option('--skip-test-optimize-none-with-implicit-dynamic', unsupported)
    option('--skip-test-optimized', unsupported)

+    # -------------------------------------------------------------------------
+    in_group('Build-script-impl arguments (for disambiguation)')
+    # We need to list --skip-build-swift explicitly because otherwise argparse


Should this be "skip-test-swift" ?

Yeah it should be. I’ll fix it in a follow-up PR.

ahoppen · 2019-10-21T17:30:41Z

Yes, the PR only tackles macOS support. I’ll look into Linux support later.

CodaFi · 2019-10-21T18:19:32Z

It looks like this broke the macOS bot. Could you run python-lint and commit the result?

rintaro · 2019-10-21T18:41:25Z

Fixing python-lint issue #27807

ahoppen requested review from shahmishal and benlangmuir October 8, 2019 20:39

benlangmuir reviewed Oct 10, 2019

View reviewed changes

utils/build-script Outdated Show resolved Hide resolved

ahoppen force-pushed the skstresstester-new-build-infrastructure branch from 25c842b to 0c8e1c7 Compare October 10, 2019 17:15

benlangmuir reviewed Oct 10, 2019

View reviewed changes

utils/swift_build_support/swift_build_support/products/skstresstester.py Outdated Show resolved Hide resolved

benlangmuir approved these changes Oct 10, 2019

View reviewed changes

aciidgh mentioned this pull request Oct 12, 2019

Move swiftpm to swift_build_support infra #27636

Closed

ahoppen force-pushed the skstresstester-new-build-infrastructure branch 4 times, most recently from d97ce8d to a7953e3 Compare October 18, 2019 18:02

ahoppen changed the title ~~Move stress tester to new swift_build_support build infrastructure~~ Move stress tester and swiftevolve to new swift_build_support build infrastructure Oct 18, 2019

ahoppen mentioned this pull request Oct 18, 2019

Refactor the build script swiftlang/swift-stress-tester#88

Merged

ahoppen added 2 commits October 19, 2019 19:10

[build-script] Move building skstresstester to swift_build_support

adbd96b

[build-script] Move building swiftevolve to swift_build_support

1926da3

[build-script] Add preset to test the stress tester and swiftevolve

92a0e20

ahoppen force-pushed the skstresstester-new-build-infrastructure branch from 64c3435 to 92a0e20 Compare October 20, 2019 02:40

benlangmuir reviewed Oct 21, 2019

View reviewed changes

ahoppen merged commit cfae1a3 into swiftlang:master Oct 21, 2019

ahoppen mentioned this pull request Oct 21, 2019

[build-script] Fix typo skip-build-swift -> skip-test-swift #27809

Merged

ahoppen deleted the skstresstester-new-build-infrastructure branch October 21, 2019 22:05

Move stress tester and swiftevolve to new swift_build_support build infrastructure #27580

Move stress tester and swiftevolve to new swift_build_support build infrastructure #27580

Uh oh!

Conversation

ahoppen commented Oct 8, 2019

Uh oh!

ahoppen commented Oct 8, 2019

Uh oh!

ahoppen commented Oct 8, 2019

Uh oh!

ahoppen commented Oct 8, 2019

Uh oh!

swift-ci commented Oct 8, 2019

Uh oh!

swift-ci commented Oct 8, 2019

Uh oh!

swift-ci commented Oct 8, 2019

Uh oh!

benlangmuir left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ahoppen commented Oct 10, 2019

Uh oh!

swift-ci commented Oct 10, 2019

Uh oh!

swift-ci commented Oct 10, 2019

Uh oh!

Uh oh!

shahmishal commented Oct 14, 2019

Uh oh!

ahoppen commented Oct 18, 2019

Uh oh!

ahoppen commented Oct 18, 2019

Uh oh!

swift-ci commented Oct 18, 2019

Uh oh!

swift-ci commented Oct 18, 2019

Uh oh!

ahoppen commented Oct 19, 2019

Uh oh!

ahoppen commented Oct 20, 2019

Uh oh!

swift-ci commented Oct 20, 2019

Uh oh!

swift-ci commented Oct 20, 2019

Uh oh!

ahoppen commented Oct 20, 2019

Uh oh!

ahoppen commented Oct 20, 2019

Uh oh!

ahoppen commented Oct 20, 2019

Uh oh!

ahoppen commented Oct 21, 2019

Uh oh!

swift-ci commented Oct 21, 2019

Uh oh!

ahoppen commented Oct 21, 2019

Uh oh!

benlangmuir left a comment

Choose a reason for hiding this comment

Uh oh!

benlangmuir Oct 21, 2019

Choose a reason for hiding this comment

Uh oh!

ahoppen Oct 21, 2019

Choose a reason for hiding this comment

Uh oh!

ahoppen commented Oct 21, 2019

Uh oh!

CodaFi commented Oct 21, 2019

Uh oh!

rintaro commented Oct 21, 2019

Uh oh!

Uh oh!

benlangmuir left a comment •

edited

Loading