[autolink-extract] Support (and ignore) LLVM IR files.

[drodriguez]

In Linux, the current implementation of swift-autolink-extract does not support LLVM IR files resulting from using LTO.

If one tries to build LLVM using LTO and then try to link one of the targets that use swiftc to link, but link against LLVM object files (like swift-plugin-server), swift-autolink-extract will fail saying that some object files are not valid.

To deal with LLVM IR files correctly, create and pass a llvm::LLVMContext around, which allows the APIs in llvm::object to read LLVM IR files. Additionally, handle the case of IRObjectFile when extracting, but perform no action.

[drodriguez]

@swift-ci please test

[drodriguez]

@swift-ci please test Linux platform

[kateinoigakukun]

@drodriguez For bitcode linker input files, we already emit llvm.linker.options metadata and lld understands the given options, so we don't need swift-autolink-extract when performing LTO. Therefore, the driver usually does not run swift-autolink-extract with LTO option.

So I'm curious about what invokes swift-autolink-extract in your scenario.

https://github.com/apple/swift-driver/blob/49daa7ef0a0e662a95ec022869409d0ed9fb502f/Sources/SwiftDriver/Jobs/AutolinkExtractJob.swift#L20-L23

[drodriguez]

The setup is complicated. We always build Swift not standalone, but using LLVM_EXTERNAL_PROJECTS (unified). We want to enable LTO for LLVM. I think the same will happen if one builds standalone and tries to LTO the LLVM sources, generates .a files with embedded LLVM IR files, and then uses that build directory, but I haven't gone that path.

swift-plugin-server (https://github.com/apple/swift/blob/main/tools/swift-plugin-server/CMakeLists.txt) depends on something that brings some LLVM libraries into the linker step. We do not enable LTO for the Swift pieces, only for the LLVM pieces. Swift is being used as a linker driver only, I am afraid. Some objects are IR objects (the LLVM ones and C/C++ ones), but some are not (the Swift ones). I think the code you linked does not check for each of the files (and static archives) to be passed to the swift-autolink-extract step, but checks if the driver is doing LTO (which is not, we are just linking at that point).

The provided test jumps over the usage of the Swift driver, but more or less reproduces what's going on: some step compiles a C++ files with LTO, the Swift driver is invoked and given those bitcode objects (both in the form of single objects and also in the form of archives of bitcode objects), the driver passes all the paths with extensions .a and .o, and autolink-extract tries to read those objects.

[kateinoigakukun]

@drodriguez Thank you for your elaboration! Given that situation, accepting and ignoring object files from foreign sources makes sense to me. I'd like to hear @compnerd opinion just in case.

[drodriguez]

@compnerd: added you as a reviewer. @kateinoigakukun above wanted to know your opinion.

[compnerd]

Yeah, this makes sense - the autolink-extract is meant to serve as a workaround for non-lld linker compatibility. The IR files will be emitted to objects and passed to the linker subsequently. We still need to use swift-autolink-extract on it at some point, but the IR files themselves would not contribute. I think that this change is insufficient for enabling thin lto, but is improving the state of the world.

[artemcm]

This change makes sense to me, thank you.