[cmake] Make install of stdlib target libraries OPTIONAL

[benlangmuir]

This fixes using --install-swift on macOS when not building all of the
variant stdlibs (e.g. iphonesimulator, etc.).

When we build swift on macOS, by default we build only the macOS stdlib.
The other stdlib variants are still configured and there are targets to
build them if you e.g. run ninja swift-stdlib-iphonesimulator-x86_64
manually. However, we do not separate the install actions. This
meant that you couldn't install without building all the configured
stdlib variants.

[benlangmuir]

@swift-ci please smoke test

[compnerd]

Hmm, I think that this can be handled a bit better. The install targets should only be created for the targets that are not EXCLUDE_FROM_ALL. This would be the standard library targets, and we could have a separate set of install targets for those, that could be invoked as part of the --install-swift target. (This is really a missing dependency for the install target)

[compnerd]

@gottesmm - I think that the component stuff is tangentially related to this - if we create the LLVM targets for the installation and use the LLVM component system instead, this problem doesn't really arise since the build and install can be synchronised to only install that which is built.

[benlangmuir]

Thanks for taking a look, @compnerd !

The install targets should only be created for the targets that are not EXCLUDE_FROM_ALL.

I originally looked at adding the "OPTIONAL" only when doing EXCLUDE_FROM_ALL, but it seemed quite messy since that is calculated in the middle of a function that is a couple of frames up the stack from these install_in_component calls and we'd need to thread that around.

This would be the standard library targets, and we could have a separate set of install targets for those, that could be invoked as part of the --install-swift target.

I'm not sure what this means - would we have individual install-foo targets? Is the idea that we can aggregate the install action based on the individual targets in the same way as we choose the targets that go into "all"?

I think that the component stuff is tangentially related to this - if we create the LLVM targets for the installation and use the LLVM component system instead, this problem doesn't really arise since the build and install can be synchronised to only install that which is built.

Are you suggesting having components (at the level of what is now swift-install-components) for the individual stdlib variants, or something else?

[compnerd]

Yes, we would have install-... targets, which is actually how LLVM's component system works. This allows you to create a toolchain distribution that contains only the pieces that you want. I would say that the swift components are a higher level thing that can be mapped to the install-... targets through CMake caches and build-script rather than obfuscating the CMake logic with that.

[gottesmm]

@compnerd @benlangmuir what LLVM does is that it creates an install target for each individual tool/lib. Then it defines some notion of a toolchain only build and what goes into the toolchain. It is rather inflexible and makes it so that if a package maintainer wants to package LLVM. For instance:

https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/tree/7/debian

The swift component system is meant to be a higher level interface at the level of a debian package. I think that it is possible for us to support both systems. For instance, look at my PR here: #20910. I think that changing to LLVM's low level model and not keeping the high level component model will be a mistake. IIRC when LLVM switched to that model, the Debian people basically didn't push back because they had already hacked around it or something like that.

[benlangmuir]

@gottesmm I verified that with this change we get an identical set of products for the combined and separate overlay builds, and that the default install still has the dependency on the all target as expected.

@compnerd after thinking about this more and discussing with @gottesmm I'm not convinced that we need to make the change to have separated install targets here. That may be desirable in general, but I think fixing the default install setup is mostly orthogonal from that, and I don't want to block it on a much larger change. Making the components OPTIONAL has few downsides and fixes this issue. In practice, no one can install anything today on macOS unless they are explicitly building all of the stdlibs, or explicitly configuring only a subset of them. Thoughts?

[benlangmuir]

@swift-ci please smoke test

[benlangmuir]

@compnerd ping re: my last comment.

[compnerd]

Hmm, so the optional is meant to be Darwin only? I think that might be a bit nicer to restrict it to that build. There are a bunch of issues already caused by the Darwin special cases though (e.g. assumption of fat binaries breaks down with non-MachO targets in the mix). Explicitly making the OPTIONAL keyword gated on sdk IN_LIST APPLE_PLATFORMS sounds like a good compromise to me.

[benlangmuir]

SGTM, updated to check the sdk.

[benlangmuir]

@swift-ci please smoke test