runtime: ingest LLVMSupport into the runtime #31665

compnerd · 2020-05-08T23:22:51Z

This adds a new copy of LLVMSupport into the runtime. This is the final
step before changing the inline namespace for the runtime support. This
will allow us to avoid the ODR violations from the header definitions of
LLVMSupport.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

compnerd · 2020-05-08T23:23:06Z

CC: @mikeash @jckarter

compnerd · 2020-05-08T23:39:19Z

I'll be updating the commit message to include this, but LLVMSupport was forked at 22492eead218ec91d349c8c50439880fbeacf2b7. The allocation updates are corresponding to upstream changes.

mikeash

Yaay

jckarter · 2020-05-09T00:23:16Z

Do we need all of llvmSupport? Can we bring over only the bits we use?

compnerd · 2020-05-09T00:54:07Z

@jckarter I believe that we dont need all of LLVMSupport. My intention is to bring it in its entirety to compile. Im going to more or less immediately follow this up with 2 changes:

introduce an inline namespace for avoiding the ODR violation that this causes (which is the original motivation for all the work)
begin to strip this down to just the pieces that we are actually using.

I want to introduce the library in its complete form because its much easier to track the provenance that way. It makes it simpler to split the operation into copy, delete rather than do it as one. However, Im happy to just stack that into this change and merge the entire thing at once, as long as the steps remain to help make it easier to follow what happened.

compnerd · 2020-05-09T18:49:30Z

@swift-ci please test

swift-ci · 2020-05-09T19:10:29Z

Build failed
Swift Test OS X Platform
Git Sha - 9212b373c5713e280a16cc6baac3b84fd089efee

swift-ci · 2020-05-09T19:29:29Z

Build failed
Swift Test Linux Platform
Git Sha - 9212b373c5713e280a16cc6baac3b84fd089efee

compnerd · 2020-05-10T20:50:06Z

Please test with following PRs:
swiftlang/llvm-project#1205

@swift-ci please test

swift-ci · 2020-05-10T21:08:00Z

Build failed
Swift Test OS X Platform
Git Sha - 9212b373c5713e280a16cc6baac3b84fd089efee

swift-ci · 2020-05-10T21:25:33Z

Build failed
Swift Test Linux Platform
Git Sha - 9212b373c5713e280a16cc6baac3b84fd089efee

compnerd · 2020-05-11T16:35:18Z

Please test with following PRs:
swiftlang/llvm-project#1206

@swift-ci please test

swift-ci · 2020-05-11T16:54:37Z

Build failed
Swift Test OS X Platform
Git Sha - ff38fb4729fc480710b8f1d873cc14e13a0fa62a

compnerd · 2020-05-11T16:58:35Z

Please test with following PRs:
swiftlang/llvm-project#1206

@swift-ci please test macOS platform

compnerd · 2020-05-11T16:58:45Z

Please test with following PRs:
swiftlang/llvm-project#1206

@swift-ci please test windows platform

swift-ci · 2020-05-11T17:13:06Z

Build failed
Swift Test OS X Platform
Git Sha - ff38fb4729fc480710b8f1d873cc14e13a0fa62a

swift-ci · 2020-05-11T17:21:02Z

Build failed
Swift Test Linux Platform
Git Sha - ff38fb4729fc480710b8f1d873cc14e13a0fa62a

compnerd · 2020-05-11T18:25:33Z

Please test with following PRs:
swiftlang/llvm-project#1206

@swift-ci please test

swift-ci · 2020-05-11T18:38:28Z

Build failed
Swift Test OS X Platform
Git Sha - 09eaf69951b75037d6b78790ca9a22940d91cb72

swift-ci · 2020-05-11T19:14:01Z

Build failed
Swift Test Linux Platform
Git Sha - 09eaf69951b75037d6b78790ca9a22940d91cb72

compnerd · 2020-05-11T19:44:32Z

Please test with following PRs:
swiftlang/llvm-project#1206

@swift-ci please test

swift-ci · 2020-05-11T20:05:04Z

Build failed
Swift Test OS X Platform
Git Sha - 09eaf69951b75037d6b78790ca9a22940d91cb72

swift-ci · 2020-05-11T20:32:36Z

Build failed
Swift Test Linux Platform
Git Sha - 70d4f3454e13ddf98274f04f2d8cec7cd359c041

compnerd · 2020-05-12T02:38:36Z

@swift-ci please test

swift-ci · 2020-05-12T03:21:09Z

Build failed
Swift Test Linux Platform
Git Sha - 15983dfb10b51d85a6f126bdd74521cec4a26cb2

swift-ci · 2020-05-12T05:18:24Z

Build failed
Swift Test OS X Platform
Git Sha - 70d4f3454e13ddf98274f04f2d8cec7cd359c041

compnerd · 2020-05-12T21:16:04Z

@swift-ci please test

swift-ci · 2020-05-12T21:26:25Z

Build failed
Swift Test OS X Platform
Git Sha - 15983dfb10b51d85a6f126bdd74521cec4a26cb2

swift-ci · 2020-05-12T22:15:30Z

Build failed
Swift Test Linux Platform
Git Sha - 147d6afe954da2a9740f40154117c09887edf5a8

compnerd · 2020-05-13T21:31:55Z

@swift-ci please test windows platform

compnerd · 2020-05-13T21:49:50Z

@swift-ci please test Linux platform

compnerd · 2020-05-14T01:42:35Z

@swift-ci please test

swift-ci · 2020-05-14T01:54:59Z

Build failed
Swift Test OS X Platform
Git Sha - fafbdeb3abe2c4d3787575ce7d810a4b3ebec501

compnerd · 2020-05-14T02:49:49Z

@swift-ci please test Windows platform

compnerd · 2020-05-14T21:43:49Z

$ nm --defined-only -demangle ./lib/swift/macosx/x86_64/libswiftCore.dylib | grep -v -e '__swift::__runtime::llvm' | grep -e 'llvm::'
00000000003cb3c0 t (anonymous namespace)::OldDemangler::demangleIdentifier(llvm::Optional<swift::Demangle::Node::Kind>)
0000000000441c30 s (anonymous namespace)::OldDemangler::demangleIdentifier(llvm::Optional<swift::Demangle::Node::Kind>)::op_char_table

This symbol is actually leaked on all the platforms, and can be cleaned up subsequently

compnerd · 2020-05-14T23:21:35Z

@swift-ci please test macOS platform

compnerd · 2020-05-15T00:14:13Z

$ nm --defined-only --demangle lib/swift/linux/libswiftCore.so | grep -v
 -e '__swift::__runtime::llvm' | grep llvm::
0000000000589810 t (anonymous namespace)::OldDemangler::demangleIdentifier(llvm::Optional<swift::Demangle::Node::Kind>)
000000000001a6f0 r (anonymous namespace)::OldDemangler::demangleIdentifier(llvm::Optional<swift::Demangle::Node::Kind>)::op_char_table

Linux is also in sync with macOS here

swift-ci · 2020-05-15T00:23:41Z

Build failed
Swift Test OS X Platform
Git Sha - fafbdeb3abe2c4d3787575ce7d810a4b3ebec501

This adds a new copy of LLVMSupport into the runtime. This is the final step before changing the inline namespace for the runtime support. This will allow us to avoid the ODR violations from the header definitions of LLVMSupport. LLVMSupport forked at: 22492eead218ec91d349c8c50439880fbeacf2b7 Changes made to LLVMSupport from that revision: process.inc forward declares `_beginthreadex` due to compilation issues due to custom flag handling API changes required that we alter the `Deallocate` routine to account for the alignment. This is a temporary state, meant to simplify the process. We do not use the entire LLVMSupport library and there is no value in keeping the entire library. Subsequent commits will prune the library to the needs for the runtime.

This adds the `__swift::__runtime` inline namespace to the LLVMSupport interfaces. This avoids an ODR violation when LLVM and Swift are in the same address space. It also will aid in the process of pruning the LLVMSupport library by ensuring that accidental leakage of the llvm namespace does not allow us to remove symbols which we rely on.

Reduce LLVMSupport to the subset required for the runtime. This reduces the TCB and the overheads of the runtime. The inline namespace's preservation ensures that ODR violations do not occur.

compnerd · 2020-05-15T16:57:07Z

@swift-ci please test

compnerd · 2020-05-15T18:00:00Z

@swift-ci please test Windows platform

compnerd · 2020-05-15T19:53:40Z

Spoke with JoeG and MikeA about this offline. We all agree that this is bringing us into a better state for stability moving forward, addresses the ODR violation issue.

This allows for a transitioning state where we could start vending the ADTs as part of the runtime and work towards removing the some of the C++-ness of the ABI definitions to allow for easier sharing between the runtime and the compiler. It would also allow for easier use of the ABI type definitions in the Swift code.

compnerd requested a review from mikeash May 8, 2020 23:24

mikeash approved these changes May 8, 2020

View reviewed changes

compnerd force-pushed the indigestion branch from d50417f to 9212b37 Compare May 9, 2020 18:42

compnerd force-pushed the indigestion branch from 9212b37 to ff38fb4 Compare May 11, 2020 16:34

compnerd force-pushed the indigestion branch from ff38fb4 to 09eaf69 Compare May 11, 2020 18:00

compnerd force-pushed the indigestion branch 2 times, most recently from 51afc8b to 70d4f34 Compare May 11, 2020 19:42

compnerd force-pushed the indigestion branch from 70d4f34 to 41d8e4d Compare May 11, 2020 21:24

compnerd force-pushed the indigestion branch from 8700277 to 15983df Compare May 12, 2020 02:28

compnerd force-pushed the indigestion branch from 15983df to 147d6af Compare May 12, 2020 20:45

compnerd force-pushed the indigestion branch from 147d6af to 3e3a9db Compare May 12, 2020 23:18

compnerd force-pushed the indigestion branch from 3e3a9db to fafbdeb Compare May 13, 2020 23:25

compnerd added 3 commits May 15, 2020 09:55

runtime: prune LLVMSupport

bb10270

Reduce LLVMSupport to the subset required for the runtime. This reduces the TCB and the overheads of the runtime. The inline namespace's preservation ensures that ODR violations do not occur.

compnerd force-pushed the indigestion branch from fafbdeb to bb10270 Compare May 15, 2020 16:56

compnerd merged commit 7b70120 into swiftlang:master May 15, 2020

compnerd deleted the indigestion branch May 15, 2020 19:54

edymtt mentioned this pull request May 21, 2020

[build] hide some symbols in LLVMSupport #31951

Closed

kubamracek mentioned this pull request Nov 13, 2021

Re-import LLVMSupport from llvm-project and enforce header includes only being used from the Swift fork when building stdlib #40173

Merged

runtime: ingest LLVMSupport into the runtime #31665

runtime: ingest LLVMSupport into the runtime #31665

Uh oh!

Conversation

compnerd commented May 8, 2020

Uh oh!

compnerd commented May 8, 2020

Uh oh!

compnerd commented May 8, 2020

Uh oh!

mikeash left a comment

Choose a reason for hiding this comment

Uh oh!

jckarter commented May 9, 2020

Uh oh!

compnerd commented May 9, 2020

Uh oh!

compnerd commented May 9, 2020

Uh oh!

swift-ci commented May 9, 2020

Uh oh!

swift-ci commented May 9, 2020

Uh oh!

compnerd commented May 10, 2020

Uh oh!

swift-ci commented May 10, 2020

Uh oh!

swift-ci commented May 10, 2020

Uh oh!

compnerd commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

compnerd commented May 11, 2020

Uh oh!

compnerd commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

compnerd commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

compnerd commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

swift-ci commented May 11, 2020

Uh oh!

compnerd commented May 12, 2020

Uh oh!

swift-ci commented May 12, 2020

Uh oh!

swift-ci commented May 12, 2020

Uh oh!

compnerd commented May 12, 2020

Uh oh!

swift-ci commented May 12, 2020

Uh oh!

swift-ci commented May 12, 2020

Uh oh!

compnerd commented May 13, 2020

Uh oh!

compnerd commented May 13, 2020

Uh oh!

compnerd commented May 14, 2020

Uh oh!

swift-ci commented May 14, 2020

Uh oh!

compnerd commented May 14, 2020

Uh oh!

compnerd commented May 14, 2020

Uh oh!

compnerd commented May 14, 2020

Uh oh!

compnerd commented May 15, 2020