Skip to content

utils: build a few extra copies of swift-syntax #75206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

utils: build a few extra copies of swift-syntax #75206

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions utils/build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,7 @@ enum TargetComponent {
LLVM
Runtime
Dispatch
SwiftSyntax
Foundation
XCTest
}
Expand Down Expand Up @@ -1646,6 +1647,27 @@ function Build-Dispatch([Platform]$Platform, $Arch, [switch]$Test = $false) {
}
}

function Build-Syntax([Platform]$Platform, $Arch, [switch]$Test = $false) {
$SyntaxBinaryDir = Get-TargetProjectBinaryCache $Arch SwiftSyntax
$Targets = @("default", "install")
$InstallPath = "$($Arch.SDKInstallRoot)\usr"

Isolate-EnvVars {
Build-CMakeProject `
-Src $SourceCache\swift-syntax `
-Bin $SyntaxBinaryDir `
-InstallTo $InstallPath `
-Arch $Arch `
-Platform $Platform `
-UseBuiltCompilers Swift `
-BuildTargets $Targets `
-Defines (@{
CMAKE_BUILD_WITH_INSTALL_RPATH = "YES";
BUILD_SHARED_LIBS = "YES";
})
}
}

function Build-Foundation([Platform]$Platform, $Arch, [switch]$Test = $false) {
$DispatchBinaryCache = Get-TargetProjectBinaryCache $Arch Dispatch
$SwiftSyntaxDir = Get-HostProjectCMakeModules Compilers
Expand Down Expand Up @@ -2358,6 +2380,7 @@ if (-not $SkipBuild) {
# Build platform: SDK, Redist and XCTest
Invoke-BuildStep Build-Runtime Windows $Arch
Invoke-BuildStep Build-Dispatch Windows $Arch
Invoke-BuildStep Build-Syntax Windows $Arch
Invoke-BuildStep Build-Foundation Windows $Arch
Invoke-BuildStep Build-XCTest Windows $Arch
}
Expand All @@ -2372,6 +2395,7 @@ if (-not $SkipBuild) {
# Build platform: SDK, Redist and XCTest
Invoke-BuildStep Build-Runtime Android $Arch
Invoke-BuildStep Build-Dispatch Android $Arch
Invoke-BuildStep Build-Syntax Android $Arch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused about this part and maybe I'm misreading. Yes, the macros themselves are SDK content, but they still run on the builder with the rest of the toolchain. You shouldn't need to ship a Swift-Syntax to an android-arm*, unless you're planning on running a compiler there? For swift-syntax, if you're pre-building a copy so that you don't need to rebuild it (assuming aligning versions), I would expect that you build it for the machines that toolchain will be used with.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, this is currently needed due to the need for swiftlang/swift-foundation#714 to be merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? The macros in Swift-Foundation are built against what is on the machine doing the build, not the machine it's building for.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alas, if it were ... that is what that PR sets out to accomplish :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

swift-foundation macros should ideally build for both the machine doing the build and the machine that the toolchain will run on (but not the destination target of each SDK) - will this allow us to accomplish both of those?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that the changes in the referenced PR should allow us to do that. This change will then obviously need to be altered, but, I'm not only fine with that, I'm in favour of that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to be built "twice", once for the builder machine, and once for the toolchain that they're going into. That part's fine. Twice in quotes because we'll actually need to build them for every toolchain if we want the SDK to work across toolchains (thinking of the static Linux SDK), so maybe 22 times? macos, Ubuntu 20.04, 22.04, 23.10, 24.04, Fedora 39, Debian 12, AmazonLinux2, UBI9, centOS7, and Windows, and each of those for intel and ARM. RIP build times...

That said, I would prefer teaching the swift-foundation build how to not build the macros. We shouldn't need swift-syntax on Android unless someone is actually compiling macros on their phone, but then they can figure out how to get Swift-Syntax on their phone.

Right, this is currently needed due to the need for swiftlang/swift-foundation#714 to be merged.

I maybe misunderstood this comment. I thought you were saying these were both required, not that this PR was working around that issue.

will this allow us to accomplish both of those?

This PR, no. swiftlang/swift-foundation#714, gets us closer to that, yes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this change is working around the fact that I need a build for android currently. IMO, that is not what we want in the long term - we want to build swift-syntax only for the toolchain hosts - Windows x64, ARM64, macOS x64, ARM64, Linux ... I don't know.

That said, I would prefer teaching the swift-foundation build how to not build the macros.

I assume that you mean to not build the macros for the host but rather the build and standalone right? That would allow us to build the macros for the various toolchains that we want to support and not worry about the myriad of hosts that we want to support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that you mean to not build the macros for the host but rather the build and standalone right? That would allow us to build the macros for the various toolchains that we want to support and not worry about the myriad of hosts that we want to support.

Yep

Invoke-BuildStep Build-Foundation Android $Arch
Invoke-BuildStep Build-XCTest Android $Arch
}
Expand Down