Android Mac Cross-compile: Fixes issue "ld.gold: error: cannot find -lutil"

[vgorloff]

Error while building libDispatch (after building Swift):

FAILED: src/swiftDispatch.dir/libswiftDispatch.so
cd /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src && /swift-everywhere-toolchain/ToolChain/Build/darwin/swift/bin/swiftc -emit-library -target armv7-none-linux-androideabi -use-ld=gold -L /swift-everywhere-toolchain/ToolChain/Build/darwin/swift/lib/swift/android/armv7 -L /Users/vova/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/arm-linux-androideabi/24 -tools-directory /Users/vova/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin -lDispatchStubs -L /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch -lBlocksRuntime -L /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src -ldispatch   -o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/libswiftDispatch.so /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Block.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Data.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Dispatch.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/IO.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Private.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Queue.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Source.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Time.swift.o /swift-everywhere-toolchain/ToolChain/Build/armv7a/swift-corelibs-libdispatch/src/swiftDispatch.dir/Wrapper.swift.o
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold: error: cannot find -lutil
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
[43/44] Generating swift/Dispatch.swiftmodule, swift/Dispatch.swiftdoc
ninja: build stopped: subcommand failed.

Reason of error above - reference to non-existing library in glibc.modulemap. Library libutil is not a part of Android NDK:

find . -type f -iname *.a -or -iname *.so | grep arch-arm/ | grep dl
# OK
./platforms/android-23/arch-arm/usr/lib/libdl.a
./platforms/android-23/arch-arm/usr/lib/libdl.so
...
./platforms/android-28/arch-arm/usr/lib/libdl.a
./platforms/android-28/arch-arm/usr/lib/libdl.so


find . -type f -iname *.a -or -iname *.so | grep arch-arm/ | grep util
# NO RESULTS!!!

Google Info (https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md):

Native libraries >>>> must use only public API, and must not link against
non-NDK platform libraries <<<<.

Starting with API 24 this rule is enforced
and applications are no longer able to load non-NDK platform libraries.
The rule is enforced by the dynamic linker, so non-public libraries are
not accessible regardless of the way code tries to load
them: System.loadLibrary, DT_NEEDED entries, and
direct calls to dlopen(3) will all work exactly the same.

...

References:

• Issue discussion on Forum: https://forums.swift.org/t/android-libdispatch-link-error-ld-gold-error-cannot-find-lutil-due-entry-link-util-in-bionic-modulemap-gyb/25328.

• Related PR: https://github.com/apple/swift/pull/25082/files

• File bionic.modulemap.gyb: https://github.com/apple/swift/blob/master/stdlib/public/Platform/bionic.modulemap.gyb

• Google Notes: https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md

[compnerd]

Assuming that this doesn't cause a regression in the android test suite, Im okay with this. But, this does seem likely that we are doing something wrong :/
CC: @drodriguez

[compnerd]

@swift-ci please smoke test

[drodriguez]

The Android CI, sadly, doesn’t build libdispatch nor Foundation (I would love to have them, but the build script still has limitations).

This seems like an oversight on our part. I will double check and accept this if everything seems right.

@swift-ci please smoke test

[drodriguez]

I cannot make it fail on Linux while compiling libdispatch, for some reason. I wonder why it is not linking to the glibc module (or why in your case is linking to it). In any case, the library doesn’t exist, so the linking line is wrong.

Thanks!