build: migrate playground support to post-build artifact #29443

compnerd · 2020-01-24T23:05:02Z

This migrates the playground support out of the build-script-impl and
into the python based build system. This makes it build more similarly
to the Swift Package Manager and SourceKit-LSP. More importantly, it
reduces the dependency on build-script-impl.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

compnerd · 2020-01-24T23:05:11Z

@swift-ci please clean test

compnerd · 2020-01-24T23:05:30Z

CC: @Rostepher @edymtt @shahmishal @gottesmm @xiaobai

Rostepher · 2020-01-24T23:11:32Z

utils/swift_build_support/swift_build_support/products/playgroundsupport.py

+import os
+import re
+from . import product
+from .. import shell


If possible I would like for us to use the new build_swift.shell module. It should have all the functionality you want and is Python 2/3 compatible.

Actually we can hold-off for now and do follow-up PRs to migrate to the new shell module.

utils/swift_build_support/swift_build_support/products/playgroundsupport.py

Rostepher · 2020-01-24T23:12:22Z

utils/swift_build_support/swift_build_support/products/playgroundsupport.py

+                "xcodebuild",
+                "-configuration", self.args.build_variant,
+                "-workspace", "swift-xcode-playground-support.xcworkspace",
+                "-scheme", "BuildScript-{}".format({


I feel like we should have a helper function of variable for this rather than compute it multiple times in-line.

It is only computed twice (once in build, once in install). It is a dictionary lookup, which is pretty fast. I think that trying to extract this and reduce the duplication would actually hurt the readability of the build process.

swift-ci · 2020-01-25T00:19:49Z

Build failed
Swift Test Linux Platform
Git Sha - 85351fefd3793dec44c2b41c8858e07b99181543

swift-ci · 2020-01-25T00:31:05Z

Build failed
Swift Test OS X Platform
Git Sha - 85351fefd3793dec44c2b41c8858e07b99181543

Rostepher · 2020-01-25T01:02:13Z

Looks like the build_swift test suite failed. Likely you need to add defaults to utils/build_swift/tests/expected_options.py.

compnerd · 2020-01-26T18:04:11Z

@swift-ci please clean test

swift-ci · 2020-01-26T18:04:14Z

Build failed
Swift Test Linux Platform
Git Sha - 85351fefd3793dec44c2b41c8858e07b99181543

swift-ci · 2020-01-26T19:28:43Z

Build failed
Swift Test OS X Platform
Git Sha - ddc0f308c70cf3bd2f74b3fe06b029bf9db486d6

swift-ci · 2020-01-26T19:50:40Z

Build failed
Swift Test Linux Platform
Git Sha - ddc0f308c70cf3bd2f74b3fe06b029bf9db486d6

compnerd · 2020-01-26T19:50:48Z

@swift-ci please clean test

swift-ci · 2020-01-26T21:18:12Z

Build failed
Swift Test OS X Platform
Git Sha - d2be5284a0c302a8adba1464d16603f27ff31b61

compnerd · 2020-01-27T00:25:45Z

@swift-ci please clean test

swift-ci · 2020-01-27T00:57:39Z

Build failed
Swift Test Linux Platform
Git Sha - 939a3a9f143f3f6140108cc88ce359b7406011f1

swift-ci · 2020-01-27T00:58:29Z

Build failed
Swift Test OS X Platform
Git Sha - 939a3a9f143f3f6140108cc88ce359b7406011f1

compnerd · 2020-02-03T23:22:18Z

@swift-ci please clean test

swift-ci · 2020-02-04T00:37:27Z

Build failed
Swift Test Linux Platform
Git Sha - 5c01040e07f7ac13c5fbfd334a3fd9d47062185a

swift-ci · 2020-02-04T00:58:38Z

Build failed
Swift Test OS X Platform
Git Sha - 5c01040e07f7ac13c5fbfd334a3fd9d47062185a

compnerd · 2020-02-04T17:41:20Z

@swift-ci please test

swift-ci · 2020-02-04T17:41:52Z

Build failed
Swift Test OS X Platform
Git Sha - 5c01040e07f7ac13c5fbfd334a3fd9d47062185a

swift-ci · 2020-02-04T17:43:30Z

Build failed
Swift Test Linux Platform
Git Sha - 5c01040e07f7ac13c5fbfd334a3fd9d47062185a

compnerd · 2020-02-07T16:58:22Z

Thanks for the hints @Rostepher, that was extremely helpful.

compnerd · 2020-02-07T16:58:58Z

@swift-ci please test

swift-ci · 2020-02-07T17:00:56Z

Build failed
Swift Test Linux Platform
Git Sha - 6ea9c09e05fa887f0a29f6136929513e3948b99c

swift-ci · 2020-02-07T17:02:23Z

Build failed
Swift Test OS X Platform
Git Sha - 6ea9c09e05fa887f0a29f6136929513e3948b99c

compnerd · 2020-02-07T18:44:02Z

@swift-ci please test

swift-ci · 2020-02-07T18:46:15Z

Build failed
Swift Test OS X Platform
Git Sha - 317cb1116ed35cb94cbeb23aea05bfd6b0ffef37

swift-ci · 2020-02-07T18:46:38Z

Build failed
Swift Test Linux Platform
Git Sha - 317cb1116ed35cb94cbeb23aea05bfd6b0ffef37

utils/build_swift/build_swift/driver_arguments.py

compnerd · 2020-02-07T21:05:56Z

@swift-ci please test

swift-ci · 2020-02-07T21:06:31Z

Build failed
Swift Test OS X Platform
Git Sha - 1c1a8b24fb76f7910c5b3eb6e876e319f2ac3cd3

swift-ci · 2020-02-07T21:08:25Z

Build failed
Swift Test Linux Platform
Git Sha - 1c1a8b24fb76f7910c5b3eb6e876e319f2ac3cd3

compnerd · 2020-02-07T21:56:52Z

CC: @cwakamo

cwakamo · 2020-02-07T22:51:36Z

@swift-ci Please Build Toolchain macOS Platform

cwakamo

The change here generally looks good! Had one comment, but otherwise everything makes sense. I kicked off a toolchain build so I can double-check that all the right stuff ends up there with these changes; I'd like to see the result of that before approving.

utils/swift_build_support/swift_build_support/products/playgroundsupport.py

Rostepher · 2020-02-07T22:55:40Z

@swift-ci please test macOS platform

This migrates the playground support out of the build-script-impl and into the python based build system. This makes it build more similarly to the Swift Package Manager and SourceKit-LSP. More importantly, it reduces the dependency on build-script-impl.

compnerd · 2020-02-08T22:01:09Z

@swift-ci please test

swift-ci · 2020-02-08T22:01:47Z

Build failed
Swift Test OS X Platform
Git Sha - c13d2efd41a91dfd59b783e82d5b2bd6676c2491

compnerd · 2020-02-08T22:02:54Z

@swift-ci please build toolchain macOS platform

swift-ci · 2020-02-08T22:03:45Z

Build failed
Swift Test Linux Platform
Git Sha - c13d2efd41a91dfd59b783e82d5b2bd6676c2491

compnerd · 2020-02-09T01:08:00Z

https://ci.swift.org/job/swift-PR-toolchain-osx/484/artifact/branch-master/swift-PR-29443-484-osx.tar.gz @cwakamo seems that the link isn't being post into the PR

compnerd · 2020-02-12T18:21:50Z

@cwakamo ping

cwakamo

Took a look at the generated toolchain and compared it with the latest master toolchain on swift.org; looks good to me! (After a long hiatus, the macOS PlaygroundLogger/PlaygroundSupport/XCPlayground have actually returned to the toolchain; I'll need to investigate at some point what needs to be done to get the iOS Simulator and tvOS Simulator versions of those frameworks back as well.)

compnerd requested a review from Rostepher January 24, 2020 23:05

Rostepher reviewed Jan 24, 2020

View reviewed changes

utils/swift_build_support/swift_build_support/products/playgroundsupport.py Show resolved Hide resolved

Rostepher reviewed Jan 24, 2020

View reviewed changes

compnerd force-pushed the unsupported-playground-support branch from 85351fe to ddc0f30 Compare January 26, 2020 18:03

compnerd force-pushed the unsupported-playground-support branch from ddc0f30 to d2be528 Compare January 26, 2020 19:49

shahmishal requested a review from cwakamo January 26, 2020 20:42

compnerd force-pushed the unsupported-playground-support branch from d2be528 to 939a3a9 Compare January 27, 2020 00:25

compnerd force-pushed the unsupported-playground-support branch from 939a3a9 to 5c01040 Compare February 3, 2020 23:22

compnerd force-pushed the unsupported-playground-support branch from 5c01040 to cd511ec Compare February 4, 2020 17:41

compnerd force-pushed the unsupported-playground-support branch from 6ea9c09 to 317cb11 Compare February 7, 2020 16:57

compnerd force-pushed the unsupported-playground-support branch from 317cb11 to 1c1a8b2 Compare February 7, 2020 18:43

Rostepher reviewed Feb 7, 2020

View reviewed changes

utils/build_swift/build_swift/driver_arguments.py Outdated Show resolved Hide resolved

Rostepher reviewed Feb 7, 2020

View reviewed changes

utils/build_swift/build_swift/driver_arguments.py Show resolved Hide resolved

compnerd force-pushed the unsupported-playground-support branch from 1c1a8b2 to c13d2ef Compare February 7, 2020 21:05

cwakamo reviewed Feb 7, 2020

View reviewed changes

utils/swift_build_support/swift_build_support/products/playgroundsupport.py Show resolved Hide resolved

compnerd force-pushed the unsupported-playground-support branch from c13d2ef to f4086d8 Compare February 8, 2020 22:00

cwakamo approved these changes Feb 12, 2020

View reviewed changes

compnerd merged commit 8d32ee3 into swiftlang:master Feb 12, 2020

compnerd deleted the unsupported-playground-support branch February 12, 2020 19:28

build: migrate playground support to post-build artifact #29443

build: migrate playground support to post-build artifact #29443

Uh oh!

Conversation

compnerd commented Jan 24, 2020

Uh oh!

compnerd commented Jan 24, 2020

Uh oh!

compnerd commented Jan 24, 2020

Uh oh!

Rostepher Jan 24, 2020

Choose a reason for hiding this comment

Uh oh!

Rostepher Jan 24, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Rostepher Jan 24, 2020

Choose a reason for hiding this comment

Uh oh!

compnerd Jan 26, 2020

Choose a reason for hiding this comment

Uh oh!

swift-ci commented Jan 25, 2020

Uh oh!

swift-ci commented Jan 25, 2020

Uh oh!

Rostepher commented Jan 25, 2020

Uh oh!

compnerd commented Jan 26, 2020

Uh oh!

swift-ci commented Jan 26, 2020

Uh oh!

swift-ci commented Jan 26, 2020

Uh oh!

swift-ci commented Jan 26, 2020

Uh oh!

compnerd commented Jan 26, 2020

Uh oh!

swift-ci commented Jan 26, 2020

Uh oh!

compnerd commented Jan 27, 2020

Uh oh!

swift-ci commented Jan 27, 2020

Uh oh!

swift-ci commented Jan 27, 2020

Uh oh!

compnerd commented Feb 3, 2020

Uh oh!

swift-ci commented Feb 4, 2020

Uh oh!

swift-ci commented Feb 4, 2020

Uh oh!

compnerd commented Feb 4, 2020

Uh oh!

swift-ci commented Feb 4, 2020

Uh oh!

swift-ci commented Feb 4, 2020

Uh oh!

compnerd commented Feb 7, 2020

Uh oh!

compnerd commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020

Uh oh!

compnerd commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020

Uh oh!

Uh oh!

Uh oh!

compnerd commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020

Uh oh!

swift-ci commented Feb 7, 2020