Skip to content

[cxx-interop] Include Cxx and CxxStdlib modules in no-stdlib builds #65055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 12, 2023

Conversation

egorzhdan
Copy link
Contributor

@egorzhdan egorzhdan commented Apr 11, 2023

These modules are shipped with the toolchain, while the stdlib might be built and shipped separately.

rdar://107840627

These modules are shipped with the toolchain, while the stdlib might be built and shipped separately.

rdar://107780733
@egorzhdan egorzhdan added the c++ interop Feature: Interoperability with C++ label Apr 11, 2023
@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test

@egorzhdan
Copy link
Contributor Author

@swift-ci please test

@egorzhdan egorzhdan requested review from hyp and zoecarver April 11, 2023 01:04
@egorzhdan
Copy link
Contributor Author

@swift-ci please test macOS

Cxx & CxxStdlib modules are Swift-only, they do not require invoking clang directly.

When building with `SWIFT_INCLUDE_TOOLS=NO`, Clang is not available as a CMake target (see `swift_common_standalone_build_config`).

rdar://107780733
@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan
Copy link
Contributor Author

The test failure on macOS is unrelated and should be fixed by #65069

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

5 similar comments
@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan
Copy link
Contributor Author

@swift-ci please smoke test macOS

@egorzhdan egorzhdan merged commit 66cc676 into main Apr 12, 2023
@egorzhdan egorzhdan deleted the egorzhdan/cmake-cxx-toolchain branch April 12, 2023 10:16
@compnerd
Copy link
Member

compnerd commented Apr 12, 2023

This adversely effects the Windows toolchain builds :-(. Building in stages where the runtime is not built, this results in failures as this categorises Cxx as "extra toolchain content" except that does not depend on a Swift compiler or runtime.

"cd /D S:\b\1\tools\swift\stdlib\public\Cxx && "C:\Program Files\Python311\python.exe" S:/SourceCache/swift-project/swift/utils/line-directive @S:/b/1/tools/swift/stdlib/public/Cxx/d4b0ce566b2ceb69b905c4c871ae3c46452383ac.txt -- S:/b/0/bin/swiftc.exe -c -target x86_64-unknown-windows-msvc -resource-dir S:/b/1/./lib/swift -O -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY -D SWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED -D SWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING -D SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING -D SWIFT_ENABLE_EXPERIMENTAL_OBSERVATION -D SWIFT_RUNTIME_OS_VERSIONING -D SWIFT_STDLIB_ENABLE_UNICODE_DATA -D SWIFT_STDLIB_ENABLE_VECTOR_TYPES -D SWIFT_STDLIB_HAS_COMMANDLINE -D SWIFT_STDLIB_HAS_STDIN -D SWIFT_STDLIB_HAS_ENVIRON -Xcc -DSWIFT_STDLIB_HAS_ENVIRON -D SWIFT_THREADING_WIN32 -tools-directory S:/b/1/bin -module-cache-path S:/b/1/./module-cache -no-link-objc-runtime -enable-library-evolution -library-level api -Xfrontend -require-explicit-availability=ignore -Xfrontend -enforce-exclusivity=unchecked -D SWIFT_ENABLE_REFLECTION -module-name Cxx -swift-version 5 -runtime-compatibility-version none -disable-autolinking-runtime-compatibility-dynamic-replacements -Xfrontend -disable-autolinking-runtime-compatibility-concurrency -Xfrontend -disable-objc-interop -Xfrontend -enable-experimental-cxx-interop -Xcc -nostdinc++ -warn-implicit-overrides -Xfrontend -enable-ossa-modules -Xfrontend -enable-lexical-lifetimes=false -Xfrontend -disable-implicit-concurrency-module-import -Xfrontend -disable-implicit-string-processing-module-import -D_WINDLL -vfsoverlay "" -Xcc -isystem -Xcc "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\/include" -Xcc -isystem -Xcc "C:\Program Files (x86)\Windows Kits\10\/Include/10.0.22621.0/ucrt" -Xcc -isystem -Xcc "C:\Program Files (x86)\Windows Kits\10\/Include/10.0.22621.0/shared" -Xcc -isystem -Xcc "C:\Program Files (x86)\Windows Kits\10\/Include/10.0.22621.0/um" -libc MultiThreadedDLL -Xfrontend -define-availability -Xfrontend "SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4" -Xfrontend -define-availability -Xfrontend "SwiftStdlib 5.9:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" -Xfrontend -target-min-inlining-version -Xfrontend min -whole-module-optimization -color-diagnostics -parse-as-library -resource-dir S:/b/1/./lib/swift -I S:/b/1/./lib/swift/windows -o S:/b/1/tools/swift/stdlib/public/Cxx//WINDOWS/x86_64/Cxx.obj @S:/b/1/tools/swift/stdlib/public/Cxx/d4b0ce566b2ceb69b905c4c871ae3c46452383ac.txt"
Traceback (most recent call last):
  File "S:\SourceCache\swift-project\swift\utils\line-directive", line 738, in <module>
    run()
  File "S:\SourceCache\swift-project\swift\utils\line-directive", line 686, in run
    command_args = expand_response_files(sys.argv[dashes + 1:])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "S:\SourceCache\swift-project\swift\utils\line-directive", line 222, in expand_response_files
    if file_path[0] == '@':
       ~~~~~~~~~^^^

Would you be opposed to introducing an ENABLE_EXPERIMENTAL_CXX_SUPPORT or something along those lines?

drodriguez added a commit to drodriguez/swift that referenced this pull request Apr 14, 2023
With the changes introduced in swiftlang#65055, the Cxx module will build even
when not building the stdlib as long as
`SWIFT_BUILD_STDLIB_EXTRA_TOOLCHAIN_CONTENT` was set. With the further changes
in swiftlang#65122 the previous flag is not necessary, and only
`SWIFT_BUILD_STDLIB_CXX_MODULE` (which defaults to `TRUE`) is necessary.

However, `stdlib/public/Cxx` did rely on `StdlibOptions.cmake` to be
include before it, or the functions that build the target libraries will
find a half configured state. `StdlibOptions.cmake` are included in
`stdlib/toolchain`, in `stdlib/` and `StandaloneOverlay.cmake`. Before swiftlang#65122
it was working just because `stdlib/toolchain` was added just before.

The second changes adds dependencies on the legacy layouts and the clang
headers. The legacy layouts are a dependency only added to the libraries of
the stdlib core (IS_STDLIB_CORE flag). Since enabling that flag also enables
a bunch other stuff, and I am not sure that Cxx should be classified as "core"
anyway, I preferred to add the dependency. The clang headers are another
dependency of the swiftCore, which otherwise causes problems to not find
the compiler headers when building Cxx. This is necessary for builds
that use a previously built compiler to compile the stdlib.
drodriguez added a commit to drodriguez/swift that referenced this pull request Apr 14, 2023
With the changes introduced in swiftlang#65055, the Cxx module will build even
when not building the stdlib as long as
`SWIFT_BUILD_STDLIB_EXTRA_TOOLCHAIN_CONTENT` was set. With the further changes
in swiftlang#65122 the previous flag is not necessary, and only
`SWIFT_BUILD_STDLIB_CXX_MODULE` (which defaults to `TRUE`) is necessary.

However, `stdlib/public/Cxx` did rely on `StdlibOptions.cmake` to be
include before it, or the functions that build the target libraries will
find a half configured state. `StdlibOptions.cmake` are included in
`stdlib/toolchain`, in `stdlib/` and `StandaloneOverlay.cmake`. Before swiftlang#65122
it was working just because `stdlib/toolchain` was added just before.

The second changes adds dependencies on the legacy layouts and the clang
headers. The legacy layouts are a dependency only added to the libraries of
the stdlib core (IS_STDLIB_CORE flag). Since enabling that flag also enables
a bunch other stuff, and I am not sure that Cxx should be classified as "core"
anyway, I preferred to add the dependency. The clang headers are another
dependency of the swiftCore, which otherwise causes problems to not find
the compiler headers when building Cxx. This is necessary for builds
that use a previously built compiler to compile the stdlib.
drodriguez added a commit that referenced this pull request Apr 14, 2023
With the changes introduced in #65055, the Cxx module will build even
when not building the stdlib as long as
`SWIFT_BUILD_STDLIB_EXTRA_TOOLCHAIN_CONTENT` was set. With the further changes
in #65122 the previous flag is not necessary, and only
`SWIFT_BUILD_STDLIB_CXX_MODULE` (which defaults to `TRUE`) is necessary.

However, `stdlib/public/Cxx` did rely on `StdlibOptions.cmake` to be
include before it, or the functions that build the target libraries will
find a half configured state. `StdlibOptions.cmake` are included in
`stdlib/toolchain`, in `stdlib/` and `StandaloneOverlay.cmake`. Before #65122
it was working just because `stdlib/toolchain` was added just before.

The second changes adds dependencies on the legacy layouts and the clang
headers. The legacy layouts are a dependency only added to the libraries of
the stdlib core (IS_STDLIB_CORE flag). Since enabling that flag also enables
a bunch other stuff, and I am not sure that Cxx should be classified as "core"
anyway, I preferred to add the dependency. The clang headers are another
dependency of the swiftCore, which otherwise causes problems to not find
the compiler headers when building Cxx. This is necessary for builds
that use a previously built compiler to compile the stdlib.
drodriguez added a commit that referenced this pull request Apr 17, 2023
In #65172 I tried to fix a problem when the Cxx module is enabled, but
we are not building the stdlib. The fix work for `swiftCxx`, but failed
to to realize that `swiftCxxStdlib` overlay is also built, which needs
parts of the stdlib.

The original #65055 only built the Cxx module when
`EXTRA_TOOLCHAIN_CONTENT` was set, but #65122 changed so it was two
independent checks. A more faithful fix will had been to nest the `if`
as I am trying to do with this PR. Parts of #65172 are still necessary
to build in every case, though: the `swiftCxx` target is dependent on
the Clang headers and the legacy layouts. Those two pieces are normally
in place because of other targets in upstream builds, but fail to
materialize when the stdlib is trying to be built with the host
compiler.

This one and #65172 will need to be cherry-picked to 5.9 because #65055
and #65122 where cherry-picked, so 5.9 is having the same problems as
`main`.
drodriguez added a commit to drodriguez/swift that referenced this pull request Apr 17, 2023
…65172)

With the changes introduced in swiftlang#65055, the Cxx module will build even
when not building the stdlib as long as
`SWIFT_BUILD_STDLIB_EXTRA_TOOLCHAIN_CONTENT` was set. With the further changes
in swiftlang#65122 the previous flag is not necessary, and only
`SWIFT_BUILD_STDLIB_CXX_MODULE` (which defaults to `TRUE`) is necessary.

However, `stdlib/public/Cxx` did rely on `StdlibOptions.cmake` to be
include before it, or the functions that build the target libraries will
find a half configured state. `StdlibOptions.cmake` are included in
`stdlib/toolchain`, in `stdlib/` and `StandaloneOverlay.cmake`. Before swiftlang#65122
it was working just because `stdlib/toolchain` was added just before.

The second changes adds dependencies on the legacy layouts and the clang
headers. The legacy layouts are a dependency only added to the libraries of
the stdlib core (IS_STDLIB_CORE flag). Since enabling that flag also enables
a bunch other stuff, and I am not sure that Cxx should be classified as "core"
anyway, I preferred to add the dependency. The clang headers are another
dependency of the swiftCore, which otherwise causes problems to not find
the compiler headers when building Cxx. This is necessary for builds
that use a previously built compiler to compile the stdlib.

(cherry picked from commit 539fc69)
drodriguez added a commit to drodriguez/swift that referenced this pull request Apr 17, 2023
In swiftlang#65172 I tried to fix a problem when the Cxx module is enabled, but
we are not building the stdlib. The fix work for `swiftCxx`, but failed
to to realize that `swiftCxxStdlib` overlay is also built, which needs
parts of the stdlib.

The original swiftlang#65055 only built the Cxx module when
`EXTRA_TOOLCHAIN_CONTENT` was set, but swiftlang#65122 changed so it was two
independent checks. A more faithful fix will had been to nest the `if`
as I am trying to do with this PR. Parts of swiftlang#65172 are still necessary
to build in every case, though: the `swiftCxx` target is dependent on
the Clang headers and the legacy layouts. Those two pieces are normally
in place because of other targets in upstream builds, but fail to
materialize when the stdlib is trying to be built with the host
compiler.

This one and swiftlang#65172 will need to be cherry-picked to 5.9 because swiftlang#65055
and swiftlang#65122 where cherry-picked, so 5.9 is having the same problems as
`main`.

(cherry picked from commit ccf57da)
meg-gupta pushed a commit to meg-gupta/swift that referenced this pull request Apr 18, 2023
In swiftlang#65172 I tried to fix a problem when the Cxx module is enabled, but
we are not building the stdlib. The fix work for `swiftCxx`, but failed
to to realize that `swiftCxxStdlib` overlay is also built, which needs
parts of the stdlib.

The original swiftlang#65055 only built the Cxx module when
`EXTRA_TOOLCHAIN_CONTENT` was set, but swiftlang#65122 changed so it was two
independent checks. A more faithful fix will had been to nest the `if`
as I am trying to do with this PR. Parts of swiftlang#65172 are still necessary
to build in every case, though: the `swiftCxx` target is dependent on
the Clang headers and the legacy layouts. Those two pieces are normally
in place because of other targets in upstream builds, but fail to
materialize when the stdlib is trying to be built with the host
compiler.

This one and swiftlang#65172 will need to be cherry-picked to 5.9 because swiftlang#65055
and swiftlang#65122 where cherry-picked, so 5.9 is having the same problems as
`main`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ interop Feature: Interoperability with C++
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants