[build] Don’t mark toolchain snapshots as requiring a dev Swift runtime #33178
Conversation
The development toolchains we provide on swift.org set the SWIFT_USE_DEVELOPMENT_TOOLCHAIN_RUNTIME flag in their Info.plist, which makes Xcode 1) refuse to allow running apps compiled with them on iOS (and other embedded) devices. 2) set DYLD_LIBRARY_PATH on macOS such that executables compiled with the toolchain load the toolchain’s custom stdlib/overlay dylibs instead of using the OS-provided one. I believe neither of these behaviors are desirable. We ought to be able to test prerelease compilers by trying to compile & run iOS applications with them, and the stdlib that comes from a random toolchain snapshot won’t necessarily function well (or at all) as a replacement for the one that was built for the specific OS version it’s running on. Rather, these toolchains should compile apps using a Swift Standard Library module that matches the compiler, but at runtime they should load the OS-built libraries; i.e., the resulting binaries should be running in a backward- (or forward-) deployment mode. Resolves rdar://66252035.
|
@swift-ci smoke test |
|
Thanks for hunting this down, Karoy! This makes sense for iOS. Does this leave a way for users to test out the toolchain standard library on macOS? |
|
Can you add an option to build-toolchain that controls this behavior? |
|
The way to do that is to add the toolchain's library path to DYLD_LIBRARY_PATH at the time the executable is launched. I don't think this should be forced on by any toolchain, on any (ABI-stable) platform. I'm okay with adding an option for leaving this setting on, but it should never be used. Loading the stdlib/overlay dylibs from a downloadable toolchain assumes that they are able to interoperate with the rest of the libraries/frameworks in the system, which is a shaky assumption. It may or may not work depending on the running OS and how good we are at publishing/maintaining non-ABI stable Note that this is different from the in-tree tests for the stdlib -- those aren't supposed to import anything beyond Foundation. |
|
I agree, using DYLD_LIBRARY_PATH seems like a reasonable approach. Are there issues with SIP or other security measures that might prevent users from doing that though? |
|
I don't expect DYLD_LIBRARY_PATH will work when launching some system executables, but it does work for things people build themselves while testing! (We ourselves use it within this repo to run our lit tests on the just-built stdlib dylibs.) For simulator processes spawn through simctl, the variable name needs to be prefixed with SYMCTL_CHILD_ as usual. lit.cfg has the details. The stdlib dylibs we install in the toolchain have their install_name set to /usr/lib/swift, so I think manually forcing them onto the dyld path at launch time is the most practical way of testing them. |
|
@lorentey the reason why I want this is sometimes it is useful to be able to do this especially when working on macOS and giving prototypes to people to try out that require stdlib changes. |
|
@gottesmm Setting this flag in the toolchain's Info.plist only affects people who are building & running a test app within Xcode. If someone tries to compile a simple test snippet with the toolchain's Are you sure we want to add a new option to Note that stdlib API changes will still get picked up through the module file, which gets loaded from the toolchain, not the SDK. |
|
@lorentey Yes. It is important to be able to provide prototype toolchains to people who use Xcode with new features. This is something we have relied on in the past for people to play with. |
|
@gottesmm People will be able to test new compiler features/fixes on more platforms with these changes. Documenting the use of DYLD_LIBRARY_PATH will help people try out new stdlib features/fixes independent of whether they use xcodebuild or SwiftPM or any of the million other ways to build binaries. This latter use case will forever be limited by the fact that custom-built stdlib binaries don't necessarily work as a drop-in replacement for the OS-provided ones. |
This effectively reverts #31183 -- we need to match the install name convention of the other stdlib libraries. From the review feedback: > The right way to load the stdlib & runtime libraries from a custom toolchain is to set `DYLD_LIBRARY_PATH` when executing the generated binary. This is how we run tests against the just-built libraries and this is how downloadable toolchain snapshots are currently configured in Xcode -- see #33178
|
@lorentey I added an option to build-toolchain that allows for one to build toolchains against the os runtime. That being said, given that the only difference is in the plist file, I wonder if we can just produce both snapshots. |
The development toolchains we provide on swift.org set the SWIFT_USE_DEVELOPMENT_TOOLCHAIN_RUNTIME flag in their Info.plist, which makes Xcode
I believe neither of these behaviors are desirable. We ought to be able to test prerelease compilers by trying to compile & run iOS applications with them, and the stdlib that comes from a random toolchain snapshot won’t necessarily function well (or at all) as a replacement for the one that was built for the specific OS version it’s running on.
Rather, these toolchains should compile apps using a Swift Standard Library module that matches the compiler, but at runtime they should load the OS-built libraries; i.e., the resulting binaries should be running in a backward- (or forward-) deployment mode.
Resolves rdar://66252035.