Skip to content

Order-independent protocol witness table instantiation #15387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 30, 2018

Conversation

slavapestov
Copy link
Contributor

As part of resilience, we want to have the ability to re-order the requirements in a protocol without breaking client code. The caller side of this is already implemented by having protocol method calls go through thunks that are emitted as part of the protocol definition. This PR adds the missing piece, which is order-independent witness table instantiation.

The idea is that instead of emitting the witness table directly, we emit a "resilient" witness table consisting of a list of requirement/witness pairs which can appear in any order. Missing entires are permitted as long as the requirement has a default. The requirement is identified by its protocol method thunk.

@slavapestov slavapestov force-pushed the protocol-resilience branch 8 times, most recently from f2e2f12 to ee16e7e Compare March 28, 2018 01:18
@slavapestov slavapestov force-pushed the protocol-resilience branch from ee16e7e to 844dc38 Compare March 28, 2018 08:57
@slavapestov slavapestov changed the title [WIP] Order-independent protocol witness table instantiation Order-independent protocol witness table instantiation Mar 28, 2018
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov slavapestov requested a review from rjmccall March 28, 2018 10:08
@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 844dc3870b78b1962d72bd0b0ba120cc58afe67b

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 844dc3870b78b1962d72bd0b0ba120cc58afe67b

@slavapestov slavapestov force-pushed the protocol-resilience branch from 844dc38 to 1b27dac Compare March 28, 2018 12:04
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 844dc3870b78b1962d72bd0b0ba120cc58afe67b

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 844dc3870b78b1962d72bd0b0ba120cc58afe67b

@slavapestov slavapestov force-pushed the protocol-resilience branch 2 times, most recently from 5e2f457 to 63fb724 Compare March 29, 2018 05:22
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 1b27dac4e41a05a3994f8eb3479753f2ae672fdd

@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 63fb724c252502e08c20357f00826ded4fdbd5a5

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 63fb724c252502e08c20357f00826ded4fdbd5a5

@slavapestov slavapestov force-pushed the protocol-resilience branch from 63fb724 to 9f7a0cc Compare March 29, 2018 09:08
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 9f7a0ccff7d20fe7b7c8cea7aa2a1f6053a808f7

@@ -834,6 +834,10 @@ void Remangler::mangleGenericProtocolWitnessTable(Node *node) {
mangleSingleChildNode(node); // protocol conformance
}

void Remangler::mangleResilientProtocolWitnessTable(Node *node) {
unreachable("todo");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're definitively not going to support remangling these things with the old remangler, we should probably do something consistent for all them. At any rate, "todo" does not seem like the right message. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only want to remangle things that come up in a nominal type, and these new symbols are definitely not it. I'll change the todo to something else.

@@ -1056,6 +1056,11 @@ void Remangler::mangleGenericProtocolWitnessTableInstantiationFunction(Node *nod
Buffer << "WI";
}

void Remangler::mangleResilientProtocolWitnessTable(Node *node) {
mangleSingleChildNode(node);
Buffer << "Wr";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add this to Mangling.rst. Also, I think I poked you about an earlier mangling you should've done that for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, will do.

// Use it as the initializer for an anonymous constant. LLVM can treat this as
// equivalent to the global's GOT entry.
llvm::SmallString<64> name;
entity.mangle(name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this trying to give the GOT equivalent the same name as the original global?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createGOTEquivalent() prepends the "got."

}
}

void forceLocalEmitOfLazyFunction(SILFunction *f) {
DefaultIGMForFunction[f] = CurrentIGM;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work if the function is needed locally in multiple IGMs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing that can relative reference a witness thunk is a witness table and we don’t share thunks between tables. If we ever did we would need a different hack.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

…on't have to go at the end

NFC until the new witness table instantiation mechanism is enabled.
…itness table

Otherwise, they might end up in a different translation unit,
and we will be unable to form a relative reference.
- Outlined value operations are now 'WO' and not 'W'
- Protocol witness table pattern 'Wp'
- Resilient protocol witness table 'Wr'
- Protocol requirement table 'WR'
@slavapestov slavapestov force-pushed the protocol-resilience branch from 9f7a0cc to be07366 Compare March 29, 2018 22:23
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

1 similar comment
@slavapestov
Copy link
Contributor Author

@swift-ci Please test

@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

1 similar comment
@slavapestov
Copy link
Contributor Author

@swift-ci Please test source compatibility

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - be07366

@slavapestov
Copy link
Contributor Author

@aciidb0mb3r Still seeing this:

20:04:52 Test Case '-[WorkspaceTests.WorkspaceTests testCanResolveWithIncompatiblePins]' started.
20:04:52 Fatal error: file /Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/stdlib/public/core/Dictionary.swift, line 2314
20:04:52 Current stack trace:
20:04:52 0    libswiftCore.dylib                 0x000000010f78fd20 _swift_stdlib_reportFatalErrorInFile + 214
20:04:52 1    libswiftCore.dylib                 0x000000010f719c40 partial apply for closure #1 in closure #1 in closure #1 in _fatalErrorMessage(_:_:file:line:flags:) + 364
20:04:52 2    libswiftCore.dylib                 0x000000010f7198f0 partial apply for closure #1 in closure #1 in closure #1 in _fatalErrorMessage(_:_:file:line:flags:) + 25
20:04:52 3    libswiftCore.dylib                 0x000000010f41f890 specialized StaticString.withUTF8Buffer<A>(_:) + 58
20:04:52 4    libswiftCore.dylib                 0x000000010f719b50 partial apply for closure #1 in closure #1 in _assertionFailure(_:_:file:line:flags:) + 141
20:04:52 5    libswiftCore.dylib                 0x000000010f7198b0 partial apply for closure #1 in closure #1 in _fatalErrorMessage(_:_:file:line:flags:) + 23
20:04:52 6    libswiftCore.dylib                 0x000000010f41f890 specialized StaticString.withUTF8Buffer<A>(_:) + 58
20:04:52 7    libswiftCore.dylib                 0x000000010f6a4360 partial apply for closure #1 in _fatalErrorMessage(_:_:file:line:flags:) + 154
20:04:52 8    libswiftCore.dylib                 0x000000010f6a4300 partial apply for closure #1 in _fatalErrorMessage(_:_:file:line:flags:) + 23
20:04:52 9    libswiftCore.dylib                 0x000000010f41f890 specialized StaticString.withUTF8Buffer<A>(_:) + 58
20:04:52 10   libswiftCore.dylib                 0x000000010f41ef70 _fatalErrorMessage(_:_:file:line:flags:) + 136
20:04:52 11   libswiftCore.dylib                 0x000000010f491e80 _VariantDictionaryBuffer.nativeUpdateValue(_:forKey:) + 2279
20:04:52 12   SwiftPMPackageTests                0x000000010e81d890 specialized Dictionary.subscript.setter + 286
20:04:52 13   SwiftPMPackageTests                0x000000010ea9bd40 DependencyResolver.resolveSubtree(_:subjectTo:excluding:) + 4131
20:04:52 14   SwiftPMPackageTests                0x000000010ea9fee0 closure #2 in closure #3 in DependencyResolver.merge(constraints:into:subjectTo:excluding:) + 158
20:04:52 15   SwiftPMPackageTests                0x000000010eaab120 partial apply for closure #2 in closure #3 in DependencyResolver.merge(constraints:into:subjectTo:excluding:) + 21
20:04:52 16   SwiftPMPackageTests                0x000000010eaab160 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed VersionAssignmentSet<A.Container>, @guaranteed PackageContainerConstraintSet<A.Container>) -> (@owned AnySequence<(VersionAssignmentSet<A.Container>, PackageContainerConstraintSet<A.Container>)>) + 30
20:04:52 17   libswiftCore.dylib                 0x000000010f539660 LazyMapSequence.Iterator.next() + 366
20:04:52 18   libswiftCore.dylib                 0x000000010f539810 protocol witness for IteratorProtocol.next() in conformance LazyMapSequence<A, B>.Iterator + 9
20:04:52 19   libswiftCore.dylib                 0x000000010f4b37d0 FlattenSequence.Iterator.next() + 740
20:04:52 20   libswiftCore.dylib                 0x000000010f4b3c00 protocol witness for IteratorProtocol.next() in conformance FlattenSequence<A>.Iterator + 9
20:04:52 21   libswiftCore.dylib                 0x000000010f60f150 _IteratorBox.next() + 29
20:04:52 22   libswiftCore.dylib                 0x000000010f46aff0 AnyIterator.next() + 16
20:04:52 23   libswiftCore.dylib                 0x000000010f60eba0 protocol witness for IteratorProtocol.next() in conformance AnyIterator<A> + 13
20:04:52 24   libswiftCore.dylib                 0x000000010f539660 LazyMapSequence.Iterator.next() + 193
20:04:52 25   libswiftCore.dylib                 0x000000010f539810 protocol witness for IteratorProtocol.next() in conformance LazyMapSequence<A, B>.Iterator + 9
20:04:52 26   libswiftCore.dylib                 0x000000010f4b37d0 FlattenSequence.Iterator.next() + 740
20:04:52 27   libswiftCore.dylib                 0x000000010f4b3c00 protocol witness for IteratorProtocol.next() in conformance FlattenSequence<A>.Iterator + 9
20:04:52 28   libswiftCore.dylib                 0x000000010f60f150 _IteratorBox.next() + 29
20:04:52 29   libswiftCore.dylib                 0x000000010f46aff0 AnyIterator.next() + 16
20:04:52 30   libswiftCore.dylib                 0x000000010f60eba0 protocol witness for IteratorProtocol.next() in conformance AnyIterator<A> + 13
20:04:52 31   libswiftCore.dylib                 0x000000010f539660 LazyMapSequence.Iterator.next() + 193
20:04:52 32   libswiftCore.dylib                 0x000000010f539810 protocol witness for IteratorProtocol.next() in conformance LazyMapSequence<A, B>.Iterator + 9
20:04:52 33   libswiftCore.dylib                 0x000000010f396ca0 Sequence.forEach(_:) + 253
20:04:52 34   libswiftCore.dylib                 0x000000010f539d70 protocol witness for Sequence.forEach(_:) in conformance LazyMapSequence<A, B> + 59
20:04:52 35   libswiftCore.dylib                 0x000000010f60fea0 _SequenceBox._forEach(_:) + 114
20:04:52 36   libswiftCore.dylib                 0x000000010f61a620 AnySequence.forEach(_:) + 62
20:04:52 37   libswiftCore.dylib                 0x000000010f61a610 protocol witness for Sequence.forEach(_:) in conformance AnySequence<A> + 13
20:04:52 38   libswiftCore.dylib                 0x000000010f564580 Sequence.first(where:) + 163
20:04:52 39   SwiftPMPackageTests                0x000000010ea9b620 DependencyResolver.resolveAssignment(constraints:pins:) + 1551
20:04:52 40   SwiftPMPackageTests                0x000000010ea9afd0 DependencyResolver.resolve(constraints:pins:) + 25
20:04:52 41   SwiftPMPackageTests                0x000000010eaa31f0 ResolverDebugger.satisfies(_:) + 162
20:04:52 42   SwiftPMPackageTests                0x000000010eaa2850 closure #4 in ResolverDebugger.debug(dependencies:pins:) + 1679
20:04:52 43   SwiftPMPackageTests                0x000000010eaab3b0 partial apply for closure #4 in ResolverDebugger.debug(dependencies:pins:) + 65
20:04:52 44   SwiftPMPackageTests                0x000000010e8122e0 DeltaAlgorithm.run(changes:predicate:) + 81
20:04:52 45   SwiftPMPackageTests                0x000000010ea9a4d0 ResolverDebugger.debug(dependencies:pins:) + 1950
20:04:52 46   SwiftPMPackageTests                0x000000010ea9a340 DependencyResolver.resolve(dependencies:pins:) + 290
20:04:52 47   SwiftPMPackageTests                0x000000010eda1dc0 Workspace.resolveDependencies(resolver:dependencies:pins:diagnostics:) + 534
20:04:52 48   SwiftPMPackageTests                0x000000010edc2e00 specialized Workspace._resolve(root:extraConstraints:diagnostics:) + 3209
20:04:52 49   SwiftPMPackageTests                0x000000010edc4620 specialized Workspace.loadPackageGraph(root:createMultipleTestProducts:diagnostics:) + 87
20:04:52 50   SwiftPMPackageTests                0x000000010eda32d0 Workspace.loadPackageGraph(root:createMultipleTestProducts:diagnostics:) + 21
20:04:52 51   SwiftPMPackageTests                0x000000010edd8220 TestWorkspace.checkPackageGraph(roots:dependencies:_:) + 1004
20:04:52 52   SwiftPMPackageTests                0x000000010edd65b0 TestWorkspace.checkPackageGraph(roots:deps:_:) + 1347
20:04:52 53   SwiftPMPackageTests                0x000000010edf1800 specialized WorkspaceTests.testCanResolveWithIncompatiblePins() + 7658
20:04:52 54   SwiftPMPackageTests                0x000000010ede1740 @objc WorkspaceTests.testBasics() + 34
20:04:52 55   CoreFoundation                     0x00007fff3e640230 __invoking___ + 140
20:04:52 56   CoreFoundation                     0x00007fff3e640050 -[NSInvocation invoke] + 320
20:04:52 57   XCTest                             0x000000010a16ea0f __24-[XCTestCase invokeTest]_block_invoke_2.187 + 65
20:04:52 58   XCTest                             0x000000010a1d82b4 -[XCTMemoryChecker _assertInvalidObjectsDeallocatedAfterScope:] + 51
20:04:52 59   XCTest                             0x000000010a1776be -[XCTestCase assertInvalidObjectsDeallocatedAfterScope:] + 116
20:04:52 60   XCTest                             0x000000010a16e90d __24-[XCTestCase invokeTest]_block_invoke.181 + 210
20:04:52 61   XCTest                             0x000000010a1ca82e +[XCTestCase(Failures) performFailableBlock:shouldInterruptTest:] + 36
20:04:52 62   XCTest                             0x000000010a1ca766 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 54
20:04:52 63   XCTest                             0x000000010a16e2c8 __24-[XCTestCase invokeTest]_block_invoke + 854
20:04:52 64   XCTest                             0x000000010a1cf64c -[XCUITestContext performInScope:] + 237
20:04:52 65   XCTest                             0x000000010a16e204 -[XCTestCase invokeTest] + 175
20:04:52 66   XCTest                             0x000000010a16ffbf __26-[XCTestCase performTest:]_block_invoke_2 + 42
20:04:52 67   XCTest                             0x000000010a1ca82e +[XCTestCase(Failures) performFailableBlock:shouldInterruptTest:] + 36
20:04:52 68   XCTest                             0x000000010a1ca766 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 54
20:04:52 69   XCTest                             0x000000010a16fe3c __26-[XCTestCase performTest:]_block_invoke.326 + 90
20:04:52 70   XCTest                             0x000000010a1d44a0 +[XCTContext runInContextForTestCase:block:] + 225
20:04:52 71   XCTest                             0x000000010a16f2f6 -[XCTestCase performTest:] + 673
20:04:52 72   XCTest                             0x000000010a1b2643 -[XCTest runTest] + 57
20:04:52 73   XCTest                             0x000000010a16ab06 __27-[XCTestSuite performTest:]_block_invoke + 365
20:04:52 74   XCTest                             0x000000010a16a418 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
20:04:52 75   XCTest                             0x000000010a16a5e8 -[XCTestSuite performTest:] + 296
20:04:52 76   XCTest                             0x000000010a1b2643 -[XCTest runTest] + 57
20:04:52 77   XCTest                             0x000000010a16ab06 __27-[XCTestSuite performTest:]_block_invoke + 365
20:04:52 78   XCTest                             0x000000010a16a418 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
20:04:52 79   XCTest                             0x000000010a16a5e8 -[XCTestSuite performTest:] + 296
20:04:52 80   XCTest                             0x000000010a1b2643 -[XCTest runTest] + 57
20:04:52 81   XCTest                             0x000000010a16ab06 __27-[XCTestSuite performTest:]_block_invoke + 365
20:04:52 82   XCTest                             0x000000010a16a418 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
20:04:52 83   XCTest                             0x000000010a16a5e8 -[XCTestSuite performTest:] + 296
20:04:52 84   XCTest                             0x000000010a1b2643 -[XCTest runTest] + 57
20:04:52 85   XCTest                             0x000000010a1e83c0 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 40
20:04:52 86   XCTest                             0x000000010a18c973 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 600
20:04:52 87   XCTest                             0x000000010a1e807d -[XCTTestRunSession runTestsAndReturnError:] + 369
20:04:52 88   XCTest                             0x000000010a151b8d -[XCTestDriver runTestsAndReturnError:] + 440
20:04:52 89   XCTest                             0x000000010a1d2d07 _XCTestMain + 1228
20:04:52 90   xctest                             0x000000010a0f0151 <unavailable> + 8529
20:04:52 91   libdyld.dylib                      0x00007fff65ffc114 start + 1

@slavapestov
Copy link
Contributor Author

@swift-ci Please smoke test macOS

@slavapestov slavapestov merged commit c8b886f into swiftlang:master Mar 30, 2018
@aciidgh
Copy link
Contributor

aciidgh commented Mar 30, 2018

@slavapestov Sorry about that. I thought the crash was a linux only thing. I'll disable it completely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants