Skip to content

Force enabling SwiftCompilerSources in the build #73417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 7, 2024
Merged

Force enabling SwiftCompilerSources in the build #73417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
db50797
cmake: set default value of BOOTSTRAPPING_MODE to HOSTTOOLS
eeckstein May 6, 2024
410b81b
Force enabling SwiftCompilerSources in the build
eeckstein May 6, 2024
eac23ed
tests: deprecate the `swift_in_compiler` lit feature
eeckstein May 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 6 additions & 15 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,9 +341,8 @@ option(SWIFT_TOOLS_LD64_LTO_CODEGEN_ONLY_FOR_SUPPORTING_TARGETS
debugging Swift)"
FALSE)

option(BOOTSTRAPPING_MODE [=[
set(BOOTSTRAPPING_MODE HOSTTOOLS CACHE STRING [=[
How to build the swift compiler modules. Possible values are
OFF: build without swift modules
HOSTTOOLS: build with a pre-installed toolchain
BOOTSTRAPPING: build with a 2-stage bootstrapping process
BOOTSTRAPPING-WITH-HOSTLIBS: build with a 2-stage bootstrapping process,
Expand All @@ -352,7 +351,7 @@ How to build the swift compiler modules. Possible values are
`SWIFT_NATIVE_SWIFT_TOOLS_PATH` (non-Darwin only)
CROSSCOMPILE-WITH-HOSTLIBS: build with a bootstrapping-with-hostlibs compiled
compiler, provided in `SWIFT_NATIVE_SWIFT_TOOLS_PATH`
]=] OFF)
]=])

option(BRIDGING_MODE [=[
How swift-C++ bridging code is compiled:
Expand Down Expand Up @@ -736,18 +735,6 @@ option(SWIFT_ENABLE_BACKTRACING
set(SWIFT_DARWIN_DEPLOYMENT_VERSION_MACCATALYST "14.5" CACHE STRING
"Minimum deployment target version for macCatalyst")

# A tempoarary hack: force enabling HOSTTOOLS mode on Windows.
# Right now, SwiftCompilerSources cannot be enabled for lldb because on Windows
# swift and lldb are built in a unified build and there is a missing dependency
# on swiftrt.
# Swift and lldb are configured with the same cmake invocation and therefore
# enabling bootstrapping for swift and disabling it for lldb only works by
# hardcoding the bootstrapping mode in the cmake file.
# https://github.com/apple/swift/issues/73322
if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
set(BOOTSTRAPPING_MODE "HOSTTOOLS")
endif()

#
# End of user-configurable options.
#
Expand Down Expand Up @@ -910,6 +897,10 @@ set(SWIFT_MAIN_INCLUDE_DIR "${SWIFT_SOURCE_DIR}/include")
set(SWIFT_SHIMS_INCLUDE_DIR "${SWIFT_SOURCE_DIR}/stdlib/public/SwiftShims")
set(SWIFT_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/include")

if (NOT BOOTSTRAPPING_MODE)
Copy link
Member

@etcwilde etcwilde May 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to be able to have bootstrapping mode off until we can reliably cross-compile the compiler. Otherwise there is no way to bring up new platforms. (Unless you're okay restricting features to what is available and works with the 5.10 compiler.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is anybody checking that building with bootstrapping off then using that to build a full Swift compiler still works? For example, the regular bootstrapping modes are no longer checked on CI, and have been broken for months on linux.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been checking. It takes two build-script invocations to get the complete toolchain built. The first stage 0 compiler doesn't have macros or SwiftCompilerSources and it won't pass the test suite. You use that one as the host-tools compiler to bring up a stage 1 compiler with macros and SwiftCompilerSources support. That one will (or should) pass its tests. Finally, I like to do one more stage 2 build with the stage 1 compiler, which compounds issues in either of the other two. Then I run the tests, validation tests, stress tests, and long tests on that stage 2 compiler.

Unfortunately, Swift 6 doesn't build at the moment since there are some unguarded macros in the stdlib, so you can't get through the stage-0 build, but that should be reversible at the moment. You're right though, we really need some level of testing. Either that a stage-0 bootstrapping=off build works, or that we can build the 5.10 compiler, and then build the latest compiler with the 5.10 compiler or we won't be able to bring up Swift on new platforms (at least until we can cross-compile reliably).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been cross-compiling the Swift compiler to run natively on Android since Swift 5.2, termux/termux-packages#5134, so works fine for me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, @eeckstein, if Evan is still using the compiler with bootstrapping off, can't we keep that mode around until his cross-compilation changes are in? I don't use it and I'm all for your first change in this patch of changing the default mode to HOSTTOOLS, but why disable being able to turn it off right now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still possible to turn if off by not providing a host compiler

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't understand your comment below before, you simply turned it into an undocumented internal option.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly

message(FATAL_ERROR "turning off bootstrapping is not supported anymore")
endif()

set(SWIFT_RUNTIME_OUTPUT_INTDIR "${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/bin")
set(SWIFT_LIBRARY_OUTPUT_INTDIR "${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib")
if("${SWIFT_NATIVE_SWIFT_TOOLS_PATH}" STREQUAL "")
Expand Down
4 changes: 4 additions & 0 deletions test/lit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -901,6 +901,10 @@ if 'swift_evolve' in lit_config.params:
if not 'swift_driver' in lit_config.params:
config.available_features.add("cplusplus_driver")

# SwiftCompilerSources are now always enabled:
# TODO: remove all the `REQUIRES: swift_in_compiler` in tests
config.available_features.add('swift_in_compiler')

# Check if we need to run lit tests using the legacy driver or the new driver
# The default for existing test runs is to use the legacy driver.
# The new driver is tested separately.
Expand Down
10 changes: 0 additions & 10 deletions test/lit.site.cfg.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,16 +166,6 @@ config.swift_stdlib_enable_objc_interop = "@SWIFT_STDLIB_ENABLE_OBJC_INTEROP@" =
# Configured in DarwinSDKs.cmake
config.freestanding_sdk_name = "@SWIFT_SDK_FREESTANDING_LIB_SUBDIR@"

if "@BOOTSTRAPPING_MODE@" != "OFF":
config.available_features.add('swift_in_compiler')

if "@BOOTSTRAPPING_MODE@" == 'HOSTTOOLS':
config.available_features.add('hosttools_mode')
elif "@BOOTSTRAPPING_MODE@" == 'BOOTSTRAPPING':
config.available_features.add('bootstrapping_mode')
elif "@BOOTSTRAPPING_MODE@" == 'BOOTSTRAPPING-WITH-HOSTLIBS':
config.available_features.add('bootstrapping_with_hostlibs_mode')

if '@SWIFT_BUILD_SWIFT_SYNTAX@' == 'TRUE':
config.available_features.add('swift_swift_parser')

Expand Down
9 changes: 0 additions & 9 deletions utils/build-presets.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -952,15 +952,6 @@ mixin-preset=
buildbot_linux
mixin_buildbot_linux,no_test

[preset: buildbot_linux,no_test,bootstrapping_off]
mixin-preset=
buildbot_linux,no_test

skip-early-swift-driver
skip-early-swiftsyntax

bootstrapping=off

[preset: buildbot_linux,no_assertions,no_test]
mixin-preset=
buildbot_linux,no_assertions
Expand Down
26 changes: 7 additions & 19 deletions utils/build-script-impl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,24 +379,6 @@ function true_false() {
esac
}

function to_bootstrapping_mode() {
case "$1" in
false | FALSE | 0)
echo "OFF"
;;
true | TRUE | 1 | "")
if [[ "$(uname -s)" == "Darwin" ]] ; then
echo "BOOTSTRAPPING-WITH-HOSTLIBS"
else
echo "BOOTSTRAPPING"
fi
;;
*)
echo `toupper $1`
;;
esac
}

function to_varname() {
# Uses `tr` because it is orders of magnitude faster than ${1//-/_} on long
# strings, which happens when translating KNOWN_SETTINGS.
Expand Down Expand Up @@ -1628,6 +1610,13 @@ for host in "${ALL_HOSTS[@]}"; do
)
fi

if [[ "${BOOTSTRAPPING}" ]] ; then
swift_cmake_options=(
"${swift_cmake_options[@]}"
-DBOOTSTRAPPING_MODE:STRING=`toupper ${BOOTSTRAPPING}`
)
fi

if [[ "${NATIVE_CLANG_TOOLS_PATH}" ]] ; then
CLANG_BIN="${NATIVE_CLANG_TOOLS_PATH}"
if [[ ! -f "${CLANG_BIN}/clang" ]] ; then
Expand Down Expand Up @@ -1785,7 +1774,6 @@ for host in "${ALL_HOSTS[@]}"; do
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-O2 -DNDEBUG"
-DCMAKE_BUILD_TYPE:STRING="${SWIFT_BUILD_TYPE}"
-DLLVM_ENABLE_ASSERTIONS:BOOL=$(true_false "${SWIFT_ENABLE_ASSERTIONS}")
-DBOOTSTRAPPING_MODE:STRING=$(to_bootstrapping_mode "${BOOTSTRAPPING}")
-DSWIFT_ANALYZE_CODE_COVERAGE:STRING=$(toupper "${SWIFT_ANALYZE_CODE_COVERAGE}")
-DSWIFT_STDLIB_BUILD_TYPE:STRING="${SWIFT_STDLIB_BUILD_TYPE}"
-DSWIFT_STDLIB_ASSERTIONS:BOOL=$(true_false "${SWIFT_STDLIB_ENABLE_ASSERTIONS}")
Expand Down
4 changes: 2 additions & 2 deletions utils/build_swift/build_swift/driver_arguments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,9 +622,9 @@ def create_argument_parser():
help='A semi-colon split list of llvm components to install')

option('--bootstrapping', store('bootstrapping_mode'),
choices=['off', 'hosttools', 'bootstrapping', 'bootstrapping-with-hostlibs'],
choices=['hosttools', 'bootstrapping', 'bootstrapping-with-hostlibs'],
help='The bootstrapping build mode for swift compiler modules. '
'Available modes: `off`, `hosttools`, `bootstrapping`, '
'Available modes: `hosttools`, `bootstrapping`, '
'`bootstrapping-with-hostlibs`, `crosscompile`, and '
'`crosscompile-with-hostlibs`')

Expand Down
2 changes: 1 addition & 1 deletion utils/build_swift/tests/expected_options.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -838,7 +838,7 @@ class BuildScriptImplOption(_BaseOption):

SetOption('--bootstrapping', dest='bootstrapping_mode'),
ChoicesOption('--bootstrapping', dest='bootstrapping_mode',
choices=['off', 'hosttools', 'bootstrapping',
choices=['hosttools', 'bootstrapping',
'bootstrapping-with-hostlibs']),

# NOTE: We'll need to manually test the behavior of these since they
Expand Down
3 changes: 0 additions & 3 deletions validation-test/lit.site.cfg.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,6 @@ config.swift_stdlib_enable_objc_interop = "@SWIFT_STDLIB_ENABLE_OBJC_INTEROP@" =
# Configured in DarwinSDKs.cmake
config.freestanding_sdk_name = "@SWIFT_SDK_FREESTANDING_LIB_SUBDIR@"

if "@BOOTSTRAPPING_MODE@" != "OFF":
config.available_features.add('swift_in_compiler')

# Let the main config do the real work.
config.test_exec_root = os.path.dirname(os.path.realpath(__file__))
lit_config.load_config(config, "@SWIFT_SOURCE_DIR@/validation-test/lit.cfg")