[CMake] Allow only static stdlib to be built #2431

modocache · 2016-05-06T21:19:09Z

What's in this pull request?

Prior to these changes, it was only possible to build static libraries for the standard library and SDK overlays if also building dynamic libraries. This is problematic if, for example, you wanted to defer the linking of the standard library build products until later.

These changes allow you to build only static libraries, using something like the following command:

$ utils/build-script -R --skip-build-benchmarks \
    -- --build-swift-static-stdlib=1 --build-swift-dynamic-stdlib=0

Resolved bug number: None

Before merging this pull request to apple/swift repository:

Test pull request on Swift continuous integration.

Triggering Swift CI

The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:

Smoke Testing

Platform	Comment
All supported platforms	@swift-ci Please smoke test
OS X platform	@swift-ci Please smoke test OS X platform
Linux platform	@swift-ci Please smoke test Linux platform

Validation Testing

Platform	Comment
All supported platforms	@swift-ci Please test
OS X platform	@swift-ci Please test OS X platform
Linux platform	@swift-ci Please test Linux platform

Note: Only members of the Apple organization can trigger swift-ci.

tkremenek · 2016-05-07T00:56:16Z

@swift-ci test

tkremenek · 2016-05-07T06:33:48Z

@modocache On Linux we get this failure:

...swift/stdlib/public/runtime/swift_sections.S:20:2: error: "Define SWIFT_BEGIN or SWIFT_END to compile this file."
#error "Define SWIFT_BEGIN or SWIFT_END to compile this file."
 ^

modocache · 2016-05-09T15:38:13Z

@swift-ci please test

gribozavr · 2016-05-09T15:44:43Z

CMakeLists.txt

-option(SWIFT_BUILD_STDLIB
-    "Build the Swift standard library (independent of the SDK headers)"
-    TRUE)
-


Sorry, the names are confusing here. This option controls the build of the core standard library (stdlib/public/CMakeLists.txt):

if(SWIFT_BUILD_STDLIB) # These must be kept in dependency order so that any referenced targets # exist at the time we look for them in add_swift_*. add_subdirectory(SwiftShims) add_subdirectory(runtime) add_subdirectory(stubs) add_subdirectory(core) add_subdirectory(SwiftOnoneSupport) add_subdirectory(Reflection) add_subdirectory(SwiftRemoteMirror) endif()

The variables allow building just the compiler, or the compiler + core library, or just the SDK overlay using an existing compiler.

jrose-apple · 2016-05-09T23:28:21Z

Shouldn't the SDK overlay also come in both static and non-static variants?

modocache · 2016-05-10T13:50:14Z

Aha, I see... I'll take another look at this. I hadn't looked too closely at the SDK overlay parts. I'll try to address:

Building only the SDK overlay. I'm not certain these changes affect that build, but I'll try it out and see.
Making sure the dynamic/static distinction makes sense with the SDK overlay taken into account. Perhaps this means adding dynamic/static variants to the SDK overlay as well?

Does that sound reasonable?

gribozavr · 2016-05-10T16:07:21Z

@modocache Absolutely! I think it would be best if we had orthogonal options for:

building the standard library (build-swift-stdlib);
building the SDK overlay (build-swift-sdk-overlay);
building dynamic variants of the target libraries (build-swift-dynamic-stdlib);
building static variants of the target libraries (build-swift-static-stdlib).

modocache · 2016-05-13T00:00:31Z

Sorry for the lack of updates here. I'm still working on this, but things get tricky since different sets of options work or fail based on whether it's a "clean" build or not.

On master:

On master, a clean build with --build-swift-stdlib=0 --build-swift-sdk-overlay=0: ✅
On master, a clean build with --build-swift-stdlib=1 --build-swift-sdk-overlay=0: ✅
On master, a clean build with --build-swift-stdlib=0 --build-swift-sdk-overlay=1: ❌ (output)
On master, a clean build with --build-swift-stdlib=1 --build-swift-sdk-overlay=1: ✅
On master, after having build the Swift stdlib, a build with --build-swift-stdlib=0 --build-swift-sdk-overlay=1: ✅

With this pull request:

With this pull request, a clean build with --build-swift-{dynamic,static}-stdlib=0 --build-swift-{dynamic,static}-sdk-overlay=0: ✅
With this pull request, a clean build with --build-swift-{dynamic,static}-stdlib=1 --build-swift-{dynamic,static}-sdk-overlay=0: ✅
With this pull request, a clean build with --build-swift-{dynamic,static}-stdlib=0 --build-swift-{dynamic,static}-sdk-overlay=1: ❌
With this pull request, a clean build with --build-swift-{dynamic,static}-stdlib=1 --build-swift-{dynamic,static}-sdk-overlay=1: ✅
With this pull request, after having build the Swift {dynamic,static} stdlib, a clean build with --build-swift-{dynamic,static}-stdlib=0 --build-swift-{dynamic,static}-sdk-overlay=1: ❌

The major difference being that these changes don't allow building just the SDK overlay, whereas on master one is able to build just the SDK overlay (provided that the stdlib has already been built). This is because this pull request no longer adds targets for stdlib/SDK libraries that won't be built. For example, the swiftSwiftRemoteMirror target will not be added unless -build-swift-{dynamic,static}-stdlib is specified.

I understand that it's desirable to build just the SDK overlay, so I'm working on that. (I'm still trying to figure out how master works.)

gribozavr · 2016-05-13T00:29:54Z

On master, a clean build with --build-swift-stdlib=0 --build-swift-sdk-overlay=1: ❌

Sorry, I didn't give you the full information. The way we use this mode is by disabling the compiler build as well, and using the compiler that was built during the time when the standard library was built. Basically, we build in two steps, first is compiler+stdlib, and then SDK overlay that uses the build products from the previous step that are already installed into Xcode.app. For the second step we have a preset that goes like this:

ios-all
tvos-all
watchos-all

dash-dash

install-swift=1
build-llvm=0
build-swift-tools=0
build-swift-stdlib=0
build-swift-sdk-overlay=1
native-llvm-tools-path=%(native_llvm_tools_path)s
native-clang-tools-path=%(native_clang_tools_path)s
native-swift-tools-path=%(native_swift_tools_path)s

swift-install-components=sdk-overlay

Prior to these changes, it was only possible to build static libraries for the standard library and SDK overlays if *also* building dynamic libraries. This is problematic if, for example, you wanted to defer the linking of the standard library build products until later. These changes allow you to build *only* static libraries, using something like the following command: ``` $ utils/build-script -R --skip-build-benchmarks \ -- --build-swift-static-stdlib=1 --build-swift-dynamic-stdlib=0 ```

gribozavr · 2016-06-01T00:52:30Z

cmake/modules/AddSwift.cmake

+  set(all_library_targets)
+
+  # First up, we conditionally add a dynamic library target.
+  if(SWIFTLIB_IS_STDLIB OR SWIFTLIB_IS_SDK_OVERLAY)


Should this be SWIFTLIB_SINGLE_IS_STDLIB OR SWIFTLIB_SINGLE_IS_SDK_OVERLAY?

gribozavr · 2016-06-01T09:15:17Z

@modocache Just wanted to say that I'm interested in getting this feature 👍

While reviewing the patch, I found some idiosyncrasies in our handling of shared/static libraries, which I have started to fix in #2809. I'm also thinking about changing the add_swift_library function to get an explicit annotation about the library: whether it is only shared, only static, or defers to the general stdlib policy. For example, we want libswiftReflection to always be static, and libswiftRemoteMirror to always be shared, so that libswiftRemoteMirror.dylib does not have other runtime dependencies.

modocache · 2016-06-08T14:55:40Z

Just wanted to say that I'm interested in getting this feature

Oh good! Thanks, this is always good to hear. :)

I'm also thinking about changing the add_swift_library function to get an explicit annotation about the library: whether it is only shared, only static, or defers to the general stdlib policy.

Do you think it'd be simpler to update this pull request after this change is made? You wouldn't happen to have this work on a branch somewhere?

gribozavr · 2016-06-08T16:34:05Z

I'm also thinking about changing the add_swift_library function to get an explicit annotation about the library: whether it is only shared, only static, or defers to the general stdlib policy.

Do you think it'd be simpler to update this pull request after this change is made? You wouldn't happen to have this work on a branch somewhere?

Yes, I think it would be cleaner. The primary issue is that currently the CMake code is not set up around static/shared libraries; it builds a "primary" one (which is usually static for compiler components and shared for the standard library), and then it can build a secondary one, which is static for the standard library.

I have started that work with #2809 but we're not done yet. I'm currently swamped with other things unfortunately, so if you want to pick up the refactoring to clean up shared/static/global default library marking, go for it! Just let me know so that we are not duplicating work.

modocache · 2016-06-13T14:10:47Z

@gribozavr I think I'll try my hand at this beginning today. I created https://bugs.swift.org/browse/SR-1738 to track progress. If you have any tips, I'd really appreciate them!

modocache · 2016-06-16T19:33:08Z

Closing this in favor of #3020, #3027, and #3043. Will probably link back to this discussion often, though -- thanks for the feedback!

[pull] swiftwasm from main

modocache force-pushed the build-just-static-stdlib branch 2 times, most recently from 06b8047 to b2f7d5f Compare May 6, 2016 23:36

tkremenek assigned modocache May 7, 2016

modocache force-pushed the build-just-static-stdlib branch from b2f7d5f to aea3c01 Compare May 9, 2016 15:37

gribozavr reviewed May 9, 2016
View reviewed changes

modocache force-pushed the build-just-static-stdlib branch from aea3c01 to 2827d1c Compare May 12, 2016 21:45

modocache force-pushed the build-just-static-stdlib branch from 2827d1c to 3919f10 Compare May 16, 2016 03:26

modocache force-pushed the build-just-static-stdlib branch from 3919f10 to 6a1c981 Compare May 23, 2016 17:41

gribozavr reviewed Jun 1, 2016
View reviewed changes

modocache mentioned this pull request Jun 14, 2016

[SR-1738] add_swift_library takes SHARED/STATIC arg #3020

Merged

1 task

modocache closed this Jun 16, 2016

MaxDesiatov pushed a commit that referenced this pull request Apr 19, 2021

Merge pull request #2431 from swiftwasm/main

f88d8e2

[pull] swiftwasm from main

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CMake] Allow only static stdlib to be built #2431

[CMake] Allow only static stdlib to be built #2431

Uh oh!

modocache commented May 6, 2016 •

edited

Loading

Uh oh!

tkremenek commented May 7, 2016

Uh oh!

tkremenek commented May 7, 2016

Uh oh!

modocache commented May 9, 2016

Uh oh!

gribozavr May 9, 2016 •

edited

Loading

Uh oh!

jrose-apple commented May 9, 2016

Uh oh!

modocache commented May 10, 2016

Uh oh!

gribozavr commented May 10, 2016

Uh oh!

modocache commented May 13, 2016 •

edited

Loading

Uh oh!

gribozavr commented May 13, 2016

Uh oh!

gribozavr Jun 1, 2016

Uh oh!

gribozavr commented Jun 1, 2016

Uh oh!

modocache commented Jun 8, 2016

Uh oh!

gribozavr commented Jun 8, 2016 •

edited

Loading

Uh oh!

modocache commented Jun 13, 2016

Uh oh!

modocache commented Jun 16, 2016

Uh oh!

Uh oh!

[CMake] Allow *only* static stdlib to be built #2431

[CMake] Allow *only* static stdlib to be built #2431

Uh oh!

Conversation

modocache commented May 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's in this pull request?

Resolved bug number: None

Uh oh!

tkremenek commented May 7, 2016

Uh oh!

tkremenek commented May 7, 2016

Uh oh!

modocache commented May 9, 2016

Uh oh!

gribozavr May 9, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jrose-apple commented May 9, 2016

Uh oh!

modocache commented May 10, 2016

Uh oh!

gribozavr commented May 10, 2016

Uh oh!

modocache commented May 13, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gribozavr commented May 13, 2016

Uh oh!

gribozavr Jun 1, 2016

Choose a reason for hiding this comment

Uh oh!

gribozavr commented Jun 1, 2016

Uh oh!

modocache commented Jun 8, 2016

Uh oh!

gribozavr commented Jun 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

modocache commented Jun 13, 2016

Uh oh!

modocache commented Jun 16, 2016

Uh oh!

Uh oh!

[CMake] Allow only static stdlib to be built #2431

[CMake] Allow only static stdlib to be built #2431

modocache commented May 6, 2016 •

edited

Loading

gribozavr May 9, 2016 •

edited

Loading

modocache commented May 13, 2016 •

edited

Loading

gribozavr commented Jun 8, 2016 •

edited

Loading