[WIP][SR-648] Add option to create statically linked binaries

[spevans]

This is a proposed fix for SR-648 to produce static binaries with the package manager. If this is ok then I will add some test cases for it.

It allows static binaries to be created on supported platforms:

swiftc -emit-executable -static-executable -o main.static main.swift

It involves some linker magic to work around dlopen()/dlsym() not working on static binaries to fixup the conformance and metadata sections at startup

• Adds -static-executable option to swiftc to use along with
  -emit-executable
• Only works on Linux for now, requires a stub to replace
  libdl functions
• Darwin doesnt support static binaries, see
  https://developer.apple.com/library/content/qa/qa1118/_index.html

[jckarter]

Stomping on system API with stubs like this feels brittle. If the binary is truly completely static, we have the full conformance table available statically, so the runtime could link against it. The registration stuff only exists to discover conformances from dylibs dynamically.

[spevans]

@jckarter Thanks for your comment, I have simplified the stub as the libdl functions are not actually used to find the conformance tables since the linker sets some symbols to point to them if linking statically and these are used instead. The registration stuff is then skipped entirely.

libdl is only needed to resolve the dlopen() etc however it won't actually work in a static binary and it gives the following message at link time:

warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

So I substituted them with some fatal error functions to catch any callers. I have no particular view as to linking libdl.a or using the stub functions as it will work either way although avoiding the warning message may help avoid user confusion

[jckarter]

Do you know what is triggering that warning? It would be nice if, in the static library configuration, we didn't touch dlopen at all.

[jckarter]

Why is this necessary?

[spevans]

In libswiftCore.a those symbols are marked as unresolved weak:

nm -u ~/swift-test/usr/lib/swift_static/linux/libswiftCore.a |grep ' w '|sort -u
                 w __pthread_key_create
                 w pthread_once
                 w pthread_self
                 w sched_yield

And this causes gold to not link them at all so they end up having a linear address of 0x0 in the binary. Im not sure if this is a gold bug or working as intended but I couldnt find a linker option to force these symbols to be resolved so I added these three lines to make them into normal unresolved symbols

 nm -u ~/swift-test/usr/lib/swift_static/linux/x86_64/static_stub.o 
                 U abort
                 U fprintf
                 U fputc
                 U pthread_key_create
                 U pthread_once
                 U pthread_self
                 U stderr

[spevans]

The warning is embedded in libdl.a:

$ objdump -s /usr/lib/x86_64-linux-gnu/libdl.a 
In archive /usr/lib/x86_64-linux-gnu/libdl.a:

dlopen.o:     file format elf64-x86-64

Contents of section .text:
 0000 488b1424 e9000000 00                 H..$.....       
Contents of section .gnu.warning.dlopen:
 0000 5573696e 67202764 6c6f7065 6e272069  Using 'dlopen' i
 0010 6e207374 61746963 616c6c79 206c696e  n statically lin
 0020 6b656420 6170706c 69636174 696f6e73  ked applications
 0030 20726571 75697265 73206174 2072756e   requires at run
 0040 74696d65 20746865 20736861 72656420  time the shared 
 0050 6c696272 61726965 73206672 6f6d2074  libraries from t
 0060 68652067 6c696263 20766572 73696f6e  he glibc version
 0070 20757365 6420666f 72206c69 6e6b696e   used for linkin
 0080 6700                                 g.              

Its caused by the use of libdl in ProtocolConformance.cpp. Since libswiftCore.a can be statically linked into a dynamic binary (-static-stdlib) It still needs to be able to read the conformance data from any dynamically loaded modules.

A possible solution, without creating libswiftCore.a and libswiftCoreWithoutDlopen.a, is:

1. Move the swift::_swift_initializeCallbacksToInspectDylib and related functions into a separate file and update the callers in ProtocolConformance.cpp and MetadataLookup.cpp to use it.
2. In this new file either use weak linking to the dl functions or possibly create two support libs e.g. libswiftSwiftDylibLookup.a and libswiftSwiftStaticLookup.a and link in the appropriate one at build time. This would be mirror the use of the libswiftSwiftOnoneSupport.a file

Point 1. may be worth doing anyway since there is some duplication of code between the two files and it would move all of the platform specific code with all of its #ifdefs into its own file making it easier to read ProtocolConformance.cpp

[jckarter]

I agree, I like the idea of breaking the lookup code into its own .cpp file, letting the driver choose to link the fully-static or dynamic-loading version as needed by the build product.

[spevans]

I refactored all of the lookup code into SectionData.cpp/SectionData.h which tidied up ProtocolConformance.cpp and MetadataLookup.cpp. This is the first commit which I will open as a separate PR since it doesn't add or remove any functionality but is a worthwhile cleanup anyway

I did try creating 2 drivers for a DylibLookup.a and a StaticLookup.a however libicu still requires linking to libdl for dlopen() even though it doesn't use the functionality in my testing. Also trying to create two separate shared and static libs which contain different source files made the CMake files a mess of hard to maintain hacks to make it work correctly for platforms that can use it without adding extra complexity to platforms that can't.

So I went back to just modifying the lookup code for static binaries and adding in some extra checks if new sections are added without the appropriate symbols being set, and the stub just catches any calls to dl* functions and aborts as these would need to be fixed

[jckarter]

If you're having problems getting the CMake scripts to look good, @erg might be able to help. We should look into fixing libicu upstream too. You might try reaching out to swift-dev to get more input on some of these issues. Hacking around these problems by providing our own definitions of these symbols seems to me like something that's only going to cause trouble down the line.

[spevans]

I agree that these libdl hacks arent the best solution.
I looked at libicu and it does have a configure.sh option

--disable-dyload        disable dynamic loading default=no

So it can run without plugins although I don't know what functionality this breaks. Of course libicu is configured by the distro package maintainers so apart from building it as part of stdlib (which is a rather heavy solution just to enable 1 build option) Im not sure what can be done here

The other fix needed is to remove the dladdr() from Errors.cpp for linux which I don't think works anyway (see SR-755)

I'll close this PR for now, I can always rework it if these issues are fixed and there is still a requirement for static binaries in the future

[jckarter]

Thanks for looking into this, @spevans! To be clear, I think being able to emit static binaries would be awesome, and I hope I didn't discourage you by nitpicking too heavily. I doubt we rely on libicu's plugin loading feature. @dabrahams or @parkera would know for sure.

[parkera]

Foundation doesn't do anything like that directly, but I'm unsure if something we call in ICU does it on its own.