Skip to content

[Runtimes][CMake] Set up supplemental super-build #81006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1665,13 +1665,14 @@ if(SWIFT_ENABLE_NEW_RUNTIME_BUILD)

ExternalProject_Get_Property("${stdlib_target}-core" INSTALL_DIR)

ExternalProject_Add("${stdlib_target}-StringProcessing"
SOURCE_DIR
"${CMAKE_CURRENT_SOURCE_DIR}/Runtimes/Supplemental/StringProcessing"
ExternalProject_Add("${stdlib_target}-Supplemental"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is a stronger reason to setup a proper build in Supplemental. We are now doing two levels of external project :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are good reasons to not turning Supplemental into one large project. Not every distribution wants/needs all of the libraries and we have a history of not being great about tracking what can be masked out of the build together resulting in broken builds, for instance. The split projects mean that each library is responsible for using find_package to pull in the other bits that they depend on or they'll fail to link, ensuring that the dependencies don't get out of sync and we can continue to build/configure each piece independently. The current structure also ensures that each library can be built independently if you're using a build system that allows splitting the projects across machines. Much like the LLVM_ENABLE_RUNTIMES variable, the super build (and configuring the runtime build through the same CMake invocation as the compiler) is a convenience.

SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/Runtimes/Supplemental"
DEPENDS "${stdlib_target}-core"
INSTALL_DIR "${INSTALL_DIR}"
INSTALL_COMMAND "" # No install story set up yet
INSTALL_COMMAND ""
LIST_SEPARATOR "|"
CMAKE_ARGS
-DSwift_ENABLE_RUNTIMES=StringProcessing
-DBUILD_SHARED_LIBS=YES
-DCMAKE_Swift_COMPILER_WORKS:BOOLEAN=YES
-DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}
Expand Down
44 changes: 44 additions & 0 deletions Runtimes/Supplemental/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
cmake_minimum_required(VERSION 3.29)

project(SwiftRuntime LANGUAGES Swift C CXX)

include(ExternalProject)

set(SwiftRuntime_SWIFTC_SOURCE_DIR "${PROJECT_SOURCE_DIR}/../../")

foreach(lib ${Swift_ENABLE_RUNTIMES})
string(TOLOWER ${lib} name)
set(SwiftRuntime_ENABLE_${name} YES)
endforeach()

if(SwiftCore_DIR)
set(SwiftCore_DIR_FLAG "-DSwiftCore_DIR=${SwiftCore_DIR}")
Comment on lines +14 to +15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use SwiftCore_ROOT instead? If I'm following the documentation properly, <PackageName>_DIR is supposed to be an env variable from an user perspective, and the matching cache variable is mostly managed by CMake itself (which can change it if invalid)

Suggested change
if(SwiftCore_DIR)
set(SwiftCore_DIR_FLAG "-DSwiftCore_DIR=${SwiftCore_DIR}")
if(SwiftCore_ROOT)
set(SwiftCore_ROOT_FLAG "-DSwiftCore_ROOT=${SwiftCore_ROOT}")

endif()

if(CMAKE_MAKE_PROGRAM)
set(MAKE_PROGRAM_FLAG "-DCMAKE_MAKE_PROGRAM=${CMAKE_MAKE_PROGRAM}")
endif()

set(COMMON_OPTIONS
-DBUILD_SHARED_LIBS=${BUILD_SHARED_LIBS}
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added

  -DCMAKE_INSTALL_LIBDIR=${CMAKE_INSTALL_LIBDIR}
  -DCMAKE_INSTALL_NAME_DIR=${CMAKE_INSTALL_NAME_DIR}
  -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=${CMAKE_BUILD_WITH_INSTALL_NAME_DIR}

In my synchronization PR so the supplemental super build could pass those through (assuming each uses the same install destination). Not sure if that's the best approach but it's similar to install_prefix

-DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
-DCMAKE_COLOR_DIAGNOSTICS=${CMAKE_COLOR_DIAGNOSTICS}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_Swift_COMPILER=${CMAKE_Swift_COMPILER}
-DCMAKE_C_COMPILER_TARGET=${CMAKE_C_COMPILER_TARGET}
-DCMAKE_CXX_COMPILER_TARGET=${CMAKE_CXX_COMPILER_TARGET}
-DCMAKE_Swift_COMPILER_TARGET=${CMAKE_Swift_COMPILER_TARGET}
${SwiftCore_DIR_FLAG}
${MAKE_PROGRAM_FLAG})

# StringProcessing
if(SwiftRuntime_ENABLE_stringprocessing)
ExternalProject_Add(StringProcessing
SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/StringProcessing"
INSTALL_DIR "${CMAKE_INSTALL_PREFIX}"
INSTALL_COMMAND ""
CMAKE_ARGS
${COMMON_OPTIONS})
endif()
52 changes: 52 additions & 0 deletions Runtimes/Supplemental/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Swift Supplemental Libraries

The supplemental libraries are all libraries that are not one of the Core or
overlay libraries. Each supplemental library builds as an independent project.

The supplemental libraries are:
- CxxInterop
- Differentiation
- Distributed
- Observation
- StringProcessing
- SwiftRuntime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use the user visible name, not the one we need to specify in the CMake code

Suggested change
- SwiftRuntime
- Runtime

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that SwiftRuntime is actually Backtracing? If so, I think that we should clarify this as there is a SwiftRuntime in core as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @al45tair was planning on exposing more of the lower-level runtime code to Swift through this module, not just backtracing. This is referring to the RuntimeModule portion of the stdlib though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are other things that I (and others) would like to put into the Runtime module, which is why it is not just called Backtracing.

- Synchronization

The top-level Supplemental CMakeLists supplies a super-build pattern for
configuring and compiling each of the supplemental library projects through a
single CMake invocation. The `Swift_ENABLE_RUNTIMES` CMake option enables the
specified supplemental libraries. All libraries configured this way are built
with the same compilers, against the same sysroot, with the same target triple
and installed into the same location.

## Super-Build

Configuring each project independently is tedious. The Supplemental directory
contains a Super-Build CMakeLists that invokes the build of each of the
supplemental libraries in the appropriate order, simplifying the process of
building each library.

Important configuration variables:
- `Swift_ENABLE_RUNTIMES`: Used to configure which runtime libraries are built.

The super-build forwards the following variables to each sub-project
unconditionally:
- `BUILD_SHARED_LIBS`
- `CMAKE_BUILD_TYPE`
- `CMAKE_INSTALL_PREFIX`
- `CMAKE_COLOR_DIAGNOSTICS`
- `CMAKE_C_COMPILER`
- `CMAKE_C_COMPILER_TARGET`
- `CMAKE_CXX_COMPILER`
- `CMAKE_CXX_COMPILER_TARGET`
- `CMAKE_Swift_COMPILER`
- `CMAKE_Swift_COMPILER_TARGET`

If set, the super-build forwards the following values to each sub-project:

- `SwiftCore_DIR`: Path to the SwiftCore build directory
- `CMAKE_MAKE_PROGRAM`: Path to `ninja`

The super-build is for convenience. If more fine-grained control is desired for
configuring a specific runtime library, you may configure that library
independently.