Add support to llvm and cmark to cross-compile using an external sysroot

[xtremekforever]

This goes towards #78960...

When cross-compiling, we typically do not want to build all of LLVM so we just add --build-llvm=0 to the build script parameters. However, the cmake configuration stage for llvm still needs to be able to verify that it "can" cross-compile to the target arch, so that is what this PR does. This also applies to cmark if it is enabled.

Changes

• Return cross_compile_deps_path from get_linux_sysroot if it is set to allow using an external sysroot.
• Use lld when cross compiling to a Linux architecture, since lld is better at finding cross-libraries in a variety of sysroots.
• Enable cross-compiling to any linux-<arch> in the build scripts, including linux-x86_64 or linux-aarch64, which could be a useful tool when running on the opposite platform.
• Add tests for all functionality added.

Testing Locally

To test this, I created a swift:jammy docker container like this:

• docker run -it --name swift-armv7-builder -v $(pwd):/src swift:jammy /bin/bash

Then, I installed the needed/missing packages:

• apt update && apt install cmake ninja-build libsqlite3-dev ncurses-dev git python3 libstdc++-11-dev-armhf-cross

Then, to just test that llvm and cmark configure correctly without building anything else, I ran the following command from the workspace inside the container:

• ./swift/utils/build-script --cross-compile-hosts=linux-armv7 --cross-compile-deps-path=/ --build-llvm=0 --skip-local-build --skip-build-swift

This finds dependencies for armhf in the root "/", which were installed with the libstdc++-11-dev-armhf-cross package. However, an external sysroot can also be supplied:

• ./swift/utils/build-script --cross-compile-hosts=linux-armv7 --cross-compile-deps-path=/path/to/arch-sysroot--build-llvm=0 --skip-local-build --skip-build-swift

macOS Compatibility

This also can be run on macOS as long as lld is installed (brew install lld) and you have an external sysroot. Using a sysroot from https://github.com/xtremekforever/swift-armv7/releases/tag/6.0.3, I was able to run the following command:

• ./swift/utils/build-script --cross-compile-hosts=linux-armv7 --cross-compile-deps-path=sysroot-debian-bookworm --build-llvm=0 --skip-local-build --skip-build-swift

Conclusions

This is only step 1 of adding support to the Swift build scripts for cross-compiling to Linux architectures. Next will be the Swift stdlib along with libdispatch, which is required for Swift Concurrency.

@MaxDesiatov @finagolfin

[MaxDesiatov]

@swift-ci build toolchain

[MaxDesiatov]

@swift-ci test

[xtremekforever]

Hmm, what does that Windows error mean? I still do not understand these failures yet...

[finagolfin]

I added this cross-compile-deps-path flag more than four years ago for Android, a4ff4db, to contain extra dependencies like libcurl and libxml2 which aren't in the Android NDK sysroot, so I could pass in a second sysroot that does contain them to Foundation and so on. If you want to just pass in a single sysroot that has everything like this instead, this pull will help that and may work for you.

You may want to introduce a new cross-compile-sysroot flag instead, as the current build-script is limited in that regard and currently only looks in / or the passed in --android-ndk. I'd be happy to switch Android over to such a flag too. 😺 If you're more ambitious, you'd make it cross-compile-sysroots, to pass in multiple sysroots, as this build-script internally supports building for multiple platforms in one invocation, but currently that is only enabled for Darwin platforms.

However, the build commands you list above make no sense: the flags --build-llvm=0 --skip-local-build --skip-build-swift mean that only cmark will be built and nothing else! I have now asked you repeatedly what is it you intend to build finally, so I can help you figure out the correct build command for that, and have received zero response. As long as you don't know how to use the build-script properly, I cannot sign off on any changes to this script, so I will leave it up to Max if he sees any value in this.

[xtremekforever]

However, the build commands you list above make no sense: the flags --build-llvm=0 --skip-local-build --skip-build-swift mean that only cmark will be built and nothing else! I have now asked you repeatedly what is it you intend to build finally, so I can help you figure out the correct build command for that, and have received zero response.

So if you look at the ticket I filed, I stated there that my goal is be able to cross-compile Swift and all dependencies for a Linux architecture, such as armv7. I've also said this many times and in different ways, but I'll say it again. That currently doesn't work properly out of the box, so the goal here is to be able to do this. The way I envision it in the end is being able to have a buildbot like this:

./swift/utils/build-script --preset=buildbot_linux_crosscompile target-arch=armv7 sysroot=/path/to/sysroot install_destdir=swift-armv7-install installable_package=$(pwd)/swift-armv7-install.tar.gz

And as long as you have the correct stuff, this should eventually generate an entire Swift toolchain with (hopefully) everything included, but there are a lot of things that do not build for armv7 or are not setup to cross compile properly. I am taking the approach of adding support for each component to cross compile properly to reduce the number of changes and ensure each piece does not break something else.

As long as you don't know how to use the build-script properly, I cannot sign off on any changes to this script, so I will leave it up to Max if he sees any value in this.

In this PR did build-llvm=0 since this is what is used if you want to build just the Swift stdlib and not LLVM. However, I also tested this:

./swift/utils/build-script --cross-compile-hosts=linux-armv7 --cross-compile-deps-path=sysroot-debian-bookworm --skip-build-swift

I added skip-build-swift just JUST focus on LLVM. And either way, building swift will fail with other errors with this very command.

This will apparently build all of LLVM for the host (in my case, x86_64) and then it will build it for the target (armv7). However, with lld I see that it doesn't link properly when compiling for armv7 due to some symbols it cannot find. Here are the errors:

ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_store_units_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_store_set_unit_event_handler' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_occurrence_relations_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_record_reader_search_symbols' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_record_reader_symbols_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_record_reader_occurrences_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_record_reader_occurrences_in_line_range_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_record_reader_occurrences_of_symbols_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_unit_reader_dependencies_apply' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_17.0' to symbol 'indexstore_unit_reader_includes_apply' failed: symbol not defined

This does not happen with gold, but then gold struggles to find basic GCC libraries in some sysroots (debian-bookworm, debian-bullseye).

So as you see in the PR, I am using lld since it finds dependencies in sysroots when --target is provided better than gold, but we currently cannot cross-compile all of LLVM with it yet. The most that works is providing skip-build-llvm to only build a small subset of llvm instead. I am still trying to figure out a better solution, so maybe you have some ideas there?

You may want to introduce a new cross-compile-sysroot flag instead, as the current build-script is limited in that regard and currently only looks in / or the passed in --android-ndk. I'd be happy to switch Android over to such a flag too. 😺 If you're more ambitious, you'd make it cross-compile-sysroots, to pass in multiple sysroots, as this build-script internally supports building for multiple platforms in one invocation, but currently that is only enabled for Darwin platforms.

Okay, I was not aware of this. So, you think a cross-compile-sysroot or sysroots flag would be more appropriate then? I can do this instead and maybe this could be helpful down the line too for stdlib, libdispatch, Foundation, etc which all need to know where the sysroot is at. I can go ahead and make these changes to leave cross-compile-deps-path to its original design, now that I understand what it was for.

One thing I need you and anyone else to understand. I am spending time on this to reach the eventual goal of being able to easily cross-compile all of Swift so it can be possible to OFFICIALLY support other architectures than x86_64 and aarch64. The dream is being able to have toolchains you can download from swift.org for armv7, and Docker containers as well: docker pull swift:jammy --platform=linux/arm/v7. But I don't know the best way to do everything. So that's why I need the support of some people who at least know the "general way" things should work in the build scripts to make all the right choices to support this well. And, as stated in this PR as well, I'm tackling one piece at a time to focus on making changes that add the support to each component and do not break anything else.

[finagolfin]

So if you look at the ticket I filed, I stated there that my goal is be able to cross-compile Swift and all dependencies for a Linux architecture, such as armv7. I've also said this many times and in different ways, but I'll say it again. That currently doesn't work properly out of the box, so the goal here is to be able to do this... this should eventually generate an entire Swift toolchain with (hopefully) everything included

First off, you never said this was your goal in the linked ticket. I went and read it again and don't see it. If you believe I'm wrong, please directly link to where you said it before.

Second, that's not what I asked you, as building the toolchain and "all dependencies" is a long-term goal, which I don't care about. Instead, what I asked you was, what are the build products you are trying to build right now with the assorted build-script commands you're using today? You obviously have to work on a few base products first, then work your way to the rest.

there are a lot of things that do not build for armv7 or are not setup to cross compile properly

You're wrong about this, and I know this because I'm probably the only person in the world who has been cross-compiling and publicly releasing a fully 32-bit armv7 Swift toolchain for the last several years. I don't know how well it runs anymore, as my last Android arm7 device died in 2023, but it still builds. 😉 Now, of course linux armv7 is different and will require some changes, but it is probably not "a lot of things."

I added skip-build-swift just JUST focus on LLVM. And either way, building swift will fail with other errors with this very command.

Sure, it is fine to selectively disable various build products when you get started. But that makes it particularly confusing for us, as we can't know which ones you intend to build first. That is why I keep asking you which build products you want to build first, and I even suggested a possible answer a couple days ago, "a native Swift compiler for the linux host and a cross-compiled stdlib for linux armv7?" I never got a response to that very specific question.

Also, you just contradicted yourself: that flag disables building this swift repo completely, so "building swift will fail with other errors with this very command" can't happen.

I am using lld since it finds dependencies in sysroots when --target is provided better than gold

I don't think it's because lld "finds dependencies... better," because lld is written as a dumb linker that requires clang to supply all dependency paths to it. More likely, gold is doing something else that interferes with the flags clang passes to it.

I am still trying to figure out a better solution, so maybe you have some ideas there?

I can only give you ideas once I know what build products you are trying to build first. That lld issue with the indexstore symbols is known, but nobody has bothered to submit a proper fix yet.

So, you think a cross-compile-sysroot or sysroots flag would be more appropriate then?

Yes, that is a long-standing deficiency in this build-script that you could actually fix, rather than simply reusing this cross-compile-deps-path as a linux C/C++ sysroot, as you are in this pull.

One thing I need you and anyone else to understand. I am spending time on this to reach the eventual goal of being able to easily cross-compile all of Swift so it can be possible to OFFICIALLY support other architectures than x86_64 and aarch64. The dream is being able to have toolchains you can download from swift.org for armv7, and Docker containers as well: docker pull swift:jammy --platform=linux/arm/v7.

That's fine, it's good to have such long-term goals. But you should still be able to answer questions about what your short-term aims are.

But I don't know the best way to do everything. So that's why I need the support of some people who at least know the "general way" things should work in the build scripts to make all the right choices to support this well. And, as stated in this PR as well, I'm tackling one piece at a time to focus on making changes that add the support to each component and do not break anything else.

Yes, obviously that's why I'm asking you specific questions about what you're trying to build first. build-script is an extremely complicated build tool, so it is easy to slap together a bunch of flags that don't make sense and break it. Since it has very little doc and will not stop you from trying most combinations of flags that make no sense, it is very easy to use it wrong and then assume it doesn't support your usecase.

Therefore, there are only two ways to use it right:

1. Read a bunch of its source and experiment with it for several hours or days - this is what I did.
2. Ask someone who knows it well how it's supposed to be invoked for your usecase, then start fixing any remaining bugs in that invocation.

My entire goal in asking you these questions is to help you with the easier route of 2., but if you don't answer them, I'm afraid you're stuck with the harder route of 1.

[xtremekforever]

@finagolfin well it's good we have you here then since you know how everything works. Let's work together instead of talking over each other!

I started with using the --cross-compile-hosts for this since it seems to invoke cross compilation already in several projects, including cmark, llvm, and inside the build-script. So, is it fair to say that if we wanted to cross-compile to linux-armv7, as I am doing, that this is the CORRECT flag to try to use?

And the next concern is the --cross-compile-deps-path. As you stated before it was for passing a secondary sysroot for deps which is used for Android. As such that means we should implement a different option for passing a WHOLE sysroot, which could be --cross-compile-sysroot or --cross-compile-sysroots. I will look at these for this PR to not muck up the works for what already builds for Android. And, if we implement this new option, I forsee it as also being used down the line to tell the stdlib where the sysroot is, libdispatch, foundation, and so on.

Second, that's not what I asked you, as building the toolchain and "all dependencies" is a long-term goal, which I don't care about. Instead, what I asked you was, what are the build products you are trying to build right now with the assorted build-script commands you're using today? You obviously have to work on a few base products first, then work your way to the rest.

To answer this question directly. All I am doing in this PR is getting past the initial errors that cmark and llvm give when they run the cmake configure step when I provide --cross-compile-hosts=linux-armv7. I don't even care to build cmark and llvm, since we don't need them for the minimal stdlib build. BUT, if we can't get past this step, then we can't build everything else.

I can only give you ideas once I know what build products you are trying to build first. That lld issue with the indexstore symbols is known, but nobody has bothered to submit a proper fix yet.

Thanks for this. It will be useful in the future.

[al45tair]

@finagolfin Please can we tone this discussion down a bit? I appreciate you may be frustrated, but I think if you re-read what you've written above you will probably appreciate that it comes across, in places, as perhaps a bit less civil than we'd ideally like. I'm not going to comment on the PR itself right now as I don't have time, other than to thank @xtremekforever for looking at this and to note that @etcwilde and @edymtt's work on the build system is likely to make this a whole lot easier at some point in the (hopefully near) future.

[xtremekforever]

and to note that @etcwilde and @edymtt's work on the build system is likely to make this a whole lot easier at some point in the (hopefully near) future.

Thank you @al45tair . I am also looking forward to their changes, and when that is ready I would be happy to work with them to ensure that the new build system is able to cross compile! :)

[finagolfin]

I started with using the --cross-compile-hosts for this since it seems to invoke cross compilation already in several projects, including cmark, llvm, and inside the build-script. So, is it fair to say that if we wanted to cross-compile to linux-armv7, as I am doing, that this is the CORRECT flag to try to use?

No, that is my point: it is not necessary in the beginning and not a good place to start.

And the next concern is the --cross-compile-deps-path. As you stated before it was for passing a secondary sysroot for deps which is used for Android. As such that means we should implement a different option for passing a WHOLE sysroot, which could be --cross-compile-sysroot or --cross-compile-sysroots.

Yes, you can either continue with this approach or add a whole new flag, either is fine. I just don't think you should submit pulls like this until you have reached some milestone, like successfully cross-compiling the stdlib for linux armv7. That's because once you reach that milestone, you will understand the code much better and can make a better decision on which approach to upstream back. If you start upstreaming pulls now, when you really don't have anything building yet, you will likely make mistakes that you will have to undo later.

I will look at these for this PR to not muck up the works for what already builds for Android.

This pull will have no effect on building Swift for Android, which doesn't use these CMake toolchain files yet. That's not what I'm worried about: I'm only worried about guiding you along to where you're trying to go, and making sure these pulls you submit are borne out of better knowledge of the code.

And, if we implement this new option, I forsee it as also being used down the line to tell the stdlib where the sysroot is, libdispatch, foundation, and so on.

Yes, that new build-script flag would be useful when cross-compiling all the Swift repos.

To answer this question directly. All I am doing in this PR is getting past the initial errors that cmark and llvm give when they run the cmake configure step when I provide --cross-compile-hosts=linux-armv7. I don't even care to build cmark and llvm, since we don't need them for the minimal stdlib build.

Sure, but what you don't seem to understand is that you should have a goal of what initial smaller set of build products you want to cross-compile first and use the appropriate build-script command for that. Just slapping together various build-script commands with a jumble of flags is not a good way to start off.

That's why I pointed you at some existing build-script configurations a couple days ago, when I said "take a look at the standalone stdlib preset or the Android doc for the kind of flags that need to be passed in." One thing you should've noticed is that neither uses the --cross-compile-hosts flag, yet you kept trying to use that flag here.

My suggestion is that you only focus on cross-compiling the stdlib for linux armv7 first and building nothing else. For that, you will have to download the latest trunk snapshot tag of the toolchain and check out this Swift repo, LLVM, and libdispatch to that same tag (cmark not needed), then run a command like ./swift/utils/build-script --cross-compile-deps-path=sysroot-debian-bookworm --build-llvm=0 --skip-build-cmark --build-swift-tools=0 --native-swift-tools-path=/path/to/prebuilt-swift/usr/bin --native-clang-tools-path=/path/to/prebuilt-swift/usr/bin --stdlib-deployment-targets=linux-armv7, which you'll notice is a lightly modified version of that linked Android command.

After you apply this pull we're commenting on, that should pull your linux sysroot into the generated toolchain file and start cross-compiling the stdlib for linux armv7... or fail in some other corner of build-script. But the point is that command actually has a chance of working in the next week or two, once you make further small tweaks to the source, as opposed to the commands you tried above.

you will probably appreciate that it comes across, in places, as perhaps a bit less civil than we'd ideally like.

What places did you have in mind as "a bit less civil," @al45tair? I wrote my earlier comments dripping in frustration, as I wasn't getting answers to my specific technical questions, but that was my intent, to express my frustration at the lack of answers. I don't think I was ever uncivil though, and I don't think it's less civil to simply disagree strongly.

Anyway, if you look at this comment, which I wrote up to him before seeing your comment, and his last one, we may finally be getting somewhere. 🥲

[xtremekforever]

@finagolfin Okay, so I think we're getting on the same page here!

My suggestion is that you only focus on cross-compiling the stdlib for linux armv7 first and building nothing else. For that, you will have to download the latest trunk snapshot tag of the toolchain and check out this Swift repo, LLVM, and libdispatch to that same tag (cmark not needed), then run a command like ./swift/utils/build-script --cross-compile-deps-path=sysroot-debian-bookworm --build-llvm=0 --skip-build-cmark --build-swift-tools=0 --native-swift-tools-path=/path/to/prebuilt-swift/usr/bin --native-clang-tools-path=/path/to/prebuilt-swift/usr/bin --stdlib-deployment-targets=linux-armv7, which you'll notice is a lightly modified version of that linked Android command.

So this is interesting, because on latest trunk I'm running the command (with the paths populated) and it ends in:

+ /usr/bin/cmake --build ~/swift-main-build/build/Ninja-DebugAssert/swift-linux-x86_64 -- -j12 all swift-test-stdlib-linux-armv7 swift-libexec-linux-armv7
ninja: error: unknown target 'swift-test-stdlib-linux-armv7'

The reason I went down the --cross-compile-hosts flag road is because it actually influences what the stdlib compiles for. With the command above with just --stdlib-deployment-targets=linux-armv7, the stdlib prints:

-- Linux SDK:
--   Object File Format: ELF
--   Swift Standard Library Path: linux
--   Threading Package: linux
--   Static linking supported: TRUE
--   Static link only: FALSE
--   Architectures: x86_64
--   x86_64 triple: x86_64-unknown-linux-gnu
--   Module triple: x86_64-unknown-linux-gnu
--   x86_64 Path: /
-- 
-- Not building host Swift tools
-- 
-- Building Swift standard library and overlays for SDKs: LINUX
--   Build type:       Debug
--   Assertions:       TRUE

This kind of indicates to me that it wants to build for the host and not the target? However if I provide the --cross-compilation-hosts flag, this will change and it'll print armv7 in there instead. This is probably the source of the confusion. But for your android profile, I also see that you are including android and optionally an arch:

    --android \                                # Build for Android.
    --android-ndk $NDK_PATH \                  # Path to an Android NDK.
    --android-arch aarch64 \                   # Optionally specify Android architecture, alternately armv7 or x86_64
    --android-api-level 21 \                   # The Android API level to target. Swift only supports 21 or greater.

So the big question is this. How do we tell the stdlib to cross compile when we have linux-armv7 and not android? If we can figure this out without having to build llvm, I'll be very happy.

[xtremekforever]

OMG I just added --host-target=linux-armv7 as well and now the stdlib is building. My mind is blown!!

Here is my full command:

./swift/utils/build-script --cross-compile-deps-path=sysroot-debian-bookworm \
    --build-llvm=0 --skip-build-cmark --build-swift-tools=0 \
    --native-swift-tools-path=$SWIFT_NATIVE_PATH --native-clang-tools-path=$SWIFT_NATIVE_PATH \
    --stdlib-deployment-targets=linux-armv7 --host-target=linux-armv7

So, the only thing missing is that we need it to actually honor the cross-compile-deps-path, or the new cross-compile-sysroot param if we add that instead. It's just defaulting to "/":

-- Linux SDK:
--   Object File Format: ELF
--   Swift Standard Library Path: linux
--   Threading Package: linux
--   Static linking supported: TRUE
--   Static link only: FALSE
--   Architectures: armv7
--   armv7 triple: armv7-unknown-linux-gnueabihf
--   Module triple: armv7-unknown-linux-gnueabihf
--   armv7 Path: /

Would this be correct to use instead then? If this is the path to take instead of using cross-compile-hosts, then it means we don't need to mess with LLVM right now and can go straight to building the stdlib, and making sure it can get a custom sysroot instead of defaulting "/". What do you think @finagolfin @MaxDesiatov ?

[finagolfin]

Would this be correct to use instead then?

You're on your own at this point: 😄 I can only point you in the right direction with an appropriate build-script command and it is up to you now to discover the rest of the tweaks needed for your target. I think I may have seen and tried this --host-target flag years ago, but I haven't used it since then and forgot about it, and I see that none of the current build presets use it.

That suggests to me that it shouldn't be necessary, that it is being set automatically internally for other stdlib targets, but for some reason not for this one. I suggest you pass that flag externally for now, but at some point track down where it's being set internally for other targets and try to set it internally in the same way for this target too.

making sure it can get a custom sysroot instead of defaulting "/". What do you think

Sounds good, did you not apply this pull we are commenting on when building yet? If you did, that means simply setting the sysroot in this CMake toolchain file is not enough, and you will have to also set it elsewhere internally.

This is where the C/C++ sysroot is defined in the middle of this repo's CMake config. Try to track where that is being passed back from this Python build-script and pass in cross-compile-deps-path there instead, if needed.

Don't hesitate to ask us any questions at any time, but some of this you will have to read the code and learn for yourself, as no one person likely knows all of a large, continually multi-authored build tool like build-script. 😉

[xtremekforever]

Sounds good, did you not apply this pull we are commenting on when building yet? If you did, that means simply setting the sysroot in this CMake toolchain file is not enough, and you will have to also set it elsewhere internally.

This is where the C/C++ sysroot is defined in the middle of this repo's CMake config. Try to track where that is being passed back from this Python build-script and pass in cross-compile-deps-path there instead, if needed.

Nope I didn't include this pull at all since it isn't needed without that --cross-compile-hosts param. So, this makes things smoother. In fact, this pull can be thrown away now since it's not needed for the next step.

What I've done before in the past is just passing some new param to that file and set the CMAKE_SYSROOT path, like this for example:

      set(SWIFT_SDK_${prefix}_ARCH_${arch}_PATH ${CROSS_COMPILE_DEPS_PATH} CACHE STRING "CMAKE_SYSROOT for ${prefix} ${arch}")

So I'm sure something like this can be done. Maybe a CROSS_COMPILE_SYSROOT definition paired with a --cross-compile-sysroot option. I'll have a look. But, thanks for all the help thus far, I think this is some excellent progress!

[finagolfin]

Nope I didn't include this pull at all since it isn't needed without that --cross-compile-hosts param. So, this makes things smoother. In fact, this pull can be thrown away now since it's not needed for the next step.

Not quite, I suggested applying this pull above, as otherwise, --cross-compile-deps-path=sysroot-debian-bookworm does nothing in the last build command you pasted. As I noted, this build-script only looks in / or a passed in --android-ndk for the C sysroot to use for your target, so you will have to find some way to pass in your armv7 sysroot at some point.

It's fine to simply hard-code SWIFT_SDK_${prefix}_ARCH_${arch}_PATH to your armv7 sysroot for a local build right now, though you can't set it to ${CROSS_COMPILE_DEPS_PATH} as that is not passed into this repo's CMake config, but as you admit, you will have to add a new --cross-compile-sysroot flag or set something else when you try to upstream it later.

Anyway, run through the build with that hard-coded first, and see how it goes.

[xtremekforever]

Not quite, I suggested applying this pull above, as otherwise, --cross-compile-deps-path=sysroot-debian-bookworm does nothing in the last build command you pasted. As I noted, this build-script only looks in / or a passed in --android-ndk for the C sysroot to use for your target, so you will have to find some way to pass in your armv7 sysroot at some point.

It's fine to simply hard-code SWIFT_SDK_${prefix}_ARCH_${arch}_PATH to your armv7 sysroot for a local build right now, though you can't set it to ${CROSS_COMPILE_DEPS_PATH} as that is not passed into this repo's CMake config, but as you admit, you will have to add a new --cross-compile-sysroot flag or set something else when you try to upstream it later.

Anyway, run through the build with that hard-coded first, and see how it goes.

Yep, I'm coming up with a solution here. I'm already playing with setting that SWIFT_SDK_${prefix}_ARCH_${arch}_PATH and I'm able to propagate it to libdispatch as well to get a fully-built stdlib!

All I'm struggling with is the name of the sysroot flag, because it could also be something like --linux-sysroot=/path/to/sysroot. The reasoning behind this is that using a custom sysroot doesn't have to be JUST for cross-compilation- it can also be to used to compile from one distribution to another. So, be it --cross-compile-sysroot or --linux-sysroot or something else, the intention will be to propagate it down through all libraries so they can be built for whatever target is desired with the arch and sysroot.

But, I'll send a new PR with this soon so we can have a look. The stuff in this PR is now defunct and I plan to revisit it again in the future when we actually want to build LLVM with this external sysroot.

[finagolfin]

Maybe something completely generic like --target-c-sysroot? Can't just be linux-sysroot as I want to use it for Android too. 😄

[xtremekforever]

So I thought about it some more @finagolfin and I settled back on --cross-compile-sysroot. The reason for this is that although you're not compiling for a different architecture when you wanted to compile for ubuntu-jammy-aarch64 from an M1 MacBook, you are still compiling to a different platform. And that is ALSO cross-compiling. So, it makes sense to say that if you are using an external sysroot for compilation, that is cross-compilation :)

Let me know if you think otherwise. But it makes sense to me and seems pretty easy to understand. There are also sections that are now enabled when CROSS_COMPILE_SYSROOT/cross_compile_sysroot are set, so existing use cases should not be affected.

PR coming soon!

Build Percentage         Build Duration (sec)    Build Phase
================         ====================    ===========
77.6%                    307.57                  linux-armv7-swift-build
20.1%                    79.88                   linux-armv7-swift-install
2.3%                     8.94                    Building llvm
0.0%                     0.04                    linux-armv7-swift-test
0.0%                     0.03                    merged-hosts-lipo-core
0.0%                     0.03                    linux-armv7-extractsymbols
0.0%                     0.03                    linux-armv7-package
0.0%                     0.03                    merged-hosts-lipo