Skip to content

[stdlib] Round out ~Copyable generalizations in stdlib primitives #73045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 18, 2024

Conversation

lorentey
Copy link
Member

@lorentey lorentey commented Apr 16, 2024

This is a followup to #71688, completing the generalization of pointer types, Optional, Result and ManagedBuffer, as well as the temporary allocation facility.

  • Optional.map, .flatMap: Allow noncopyable results. Implement typed throws.

  • Result.map, .flatMap: Allow noncopyable types for the new success. (This reapplies the changes that got temporarily reverted in [stdlib] Undo the generalization of Result.map & .flatMap for now #72477.)

  • Result.flatMapError: Generalize for noncopyable Success.

  • Unsafe[Mutable]BufferPointer: Generalize CustomDebugStringConvertible conformance for noncopyable Element`s.

  • Unsafe[Mutable][Buffer]Pointer.withMemoryRebound: Convert to typed throws.

  • OpaquePointer.init(_: UnsafeMutablePointer), .init(_: UnsafeMutablePointer?): Allow noncopyable pointee types.

  • ManagedBuffer.withUnsafeMutablePointerToHeader, .withUnsafeMutablePointerToElements, .withUnsafeMutablePointers: Generalize for noncopyable return types and typed throw . Avoid @_preInverseGenerics.

  • ManagedBufferPointer: Ditto.

  • withUnsafeTemporaryAllocation: Generalize for noncopyable result types. (Not typed throws though -- they involve stack allocations that don't mix with explicit

  • withExtendedLifetime: Use typed throws. Stop using @_preInverseGenerics.

rdar://117753275

@lorentey
Copy link
Member Author

There'll be some api-digester failures (at least); I'll fix those later.

- `Optional.map`, `.flatMap`: Allow noncopyable results. Implement typed throws.

- `Result.map`, `.flatMap`: Allow noncopyable types for the new success.
- `Result.flatMapError`: Generalize for noncopyable Success.

- `Unsafe[Mutable][Buffer]Pointer.withMemoryRebound`: Alllow typed throws.
- `Unsafe[Mutable]BufferPointer: Generalize CustomDebugStringConvertible conformance for noncopyable `Element`s.

- `OpaquePointer.init(_: UnsafeMutablePointer)`, `.init(_: UnsafeMutablePointer?)`: Allow noncopyable pointee types.

- `ManagedBuffer.withUnsafeMutablePointerToHeader`, `.withUnsafeMutablePointerToElements`, `.withUnsafeMutablePointers`: Generalize for typed throws and noncopyable return types. Avoid `@_preInverseGenerics`.
- `ManagedBufferPointer`: Ditto.

- `withExtendedLifetime`: Use typed throws. Stop using `@_preInverseGenerics`.

rdar://117753275
We can’t do typed throws here yet. (They involve implicit stack allocations that interfere with the explicit stack allocation builtins.)
@lorentey lorentey force-pushed the noncopyable-primitives-refinements branch from 2a1417a to fab25dc Compare April 16, 2024 08:15
@lorentey
Copy link
Member Author

@swift-ci test

@lorentey
Copy link
Member Author

@swift-ci test source compatibility

@lorentey
Copy link
Member Author

lorentey commented Apr 16, 2024

Source compat failures:

GRDB:

    func testCloseFailedDueToWriter() throws {
        let dbPool = try makeDatabasePool()
        let statement = try dbPool.write { db in
            try db.makeStatement(sql: "SELECT * FROM sqlite_master")
        }
        
        try withExtendedLifetime(statement) { // 💥silgen crash
            do {
                try dbPool.close()
                XCTFail("Expected Error")
            } catch DatabaseError.SQLITE_BUSY { }
        }

The obvious workaround is to revert to untyped rethrows for withExtendedLifetime.

AsyncNinja:

/Users/ec2-user/jenkins/workspace/swift-PR-source-compat-suite-debug-macos/swift-source-compat-suite/project_cache/AsyncNinja/Sources/AsyncNinja/MergeFutures.swift:67:34: error: ambiguous use of 'map'
65 |     executor: .immediate
66 |   ) { [weak promise] (completion, originalExecutor) in
67 |     promise?.complete(completion.map(Either.left), from: originalExecutor)
   |                                  `- error: ambiguous use of 'map'
68 |   }
69 |   promise._asyncNinja_retainHandlerUntilFinalization(handlerA)

/Users/ec2-user/jenkins/workspace/swift-PR-source-compat-suite-debug-macos/swift-source-compat-suite/project_cache/AsyncNinja/Sources/AsyncNinja/Fallible.swift:143:8: note: found this candidate 
141 |   ///   - success: success value of original Fallible
142 |   /// - Returns: transformed Fallible
143 |   func map<T>(_ transform: (_ success: Success) throws -> T) -> Fallible<T> {
    |        `- note: found this candidate 
144 |     return self.flatMap { .success(try transform($0)) }
145 |   }

Swift.Result:2:17: note: found this candidate in module 'Swift'
1 | extension Result {
2 |     public func map<NewSuccess>(_ transform: (Success) -> NewSuccess) -> Result<NewSuccess, Failure> where NewSuccess : ~Copyable
  |                 `- note: found this candidate in module 'Swift'
3 | }

This is the same issue as triggered the original revert in #72477. Only workaround is to revert again. (Actually, the terrible hack of temporarily marking these @_disfavoredOverload works and it does not seem to produce new regressions, so we went with that.)

swiftlang#73045 has uncovered a compiler crash with existing code of this form:

       try withExtendedLifetime(statement) { // 💥silgen crash
            do {
                try dbPool.close()
                XCTFail("Expected Error")
            } catch DatabaseError.SQLITE_BUSY { }
        }

This patterns can happen with any of the newly typed throwing entry points, but the source compat suite only seems to have caught this with `withExtendedLifetime`; let’s see if we can get away with only rolling back this one.
…ound a shadowing issue

The new ~Copyable generalizations have changed the function signature enough that alternative definitions of `map`/`flatMap` in existing code that used to be considered to shadow the originals no longer do so. This leads to use sites becoming ambiguous — a source break.

While we consider approaches to resolve this on the compiler side, let’s try slapping a `@_disfavoredOverload` on these and see if that helps.
@lorentey
Copy link
Member Author

@swift-ci test

@lorentey
Copy link
Member Author

@swift-ci test source compatibility

@lorentey
Copy link
Member Author

@swift-ci build toolchain macOS platform

@lorentey
Copy link
Member Author

Hmm, this source compat issue in swift-distributed-actors is a preexisting regression on main. (It actually showed up in the earlier run, too.) It is unrelated to this PR; I'm ignoring it.

https://ci.swift.org/job/swift-PR-source-compat-suite-debug-macos/1529/artifact/swift-source-compat-suite/FAIL_swift-distributed-actors_5.7_BuildSwiftPackage.log

Assertion failed: (Mapping.count(requirement) == 0 && "Witness already known"), function setWitness, file ProtocolConformance.cpp, line 687.
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace.
Stack dump:
0.	Program arguments: [...]
1.	Apple Swift version 6.0-dev (LLVM 26a2b9777480433, Swift b48448f487bff4c)
2.	Compiling with effective version 5.10
3.	Contents of /var/folders/bb/hcrjxg1s0b96pfst0ymhmp240000gn/T/TemporaryDirectory.Nm4hfh/sources-1:
[...]
4.	While evaluating request ASTLoweringRequest(Lowering AST to SIL for file "/Users/ec2-user/jenkins/workspace/swift-PR-source-compat-suite-debug-macos/swift-source-compat-suite/project_cache/swift-distributed-actors/Sources/DistributedCluster/Plugins/ClusterSingleton/ClusterSingletonBoss.swift")
5.	While generating SIL witness table protocol conformance Self: Actor at extension of DistributedActor (in module 'Distributed')
6.	While reading from module 'Distributed', builder version '6.0(5.10)/Apple Swift version 6.0-dev (LLVM 26a2b9777480433, Swift b48448f487bff4c)', built from source, resilient, loaded from '/Users/ec2-user/jenkins/workspace-private/swift-PR-source-compat-suite-debug-macos/build/compat_macos/install/toolchain/usr/lib/swift/macosx/Distributed.swiftmodule/x86_64-apple-macos.swiftmodule'
7.	While finishing conformance for protocol conformance Self: Actor at extension of DistributedActor (in module 'Distributed')
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x000000011241ada8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 40
1  swift-frontend           0x0000000112418f48 llvm::sys::RunSignalHandlers() + 248
2  swift-frontend           0x000000011241b40e SignalHandler(int) + 270
3  libsystem_platform.dylib 0x00007ff8122085ed _sigtramp + 29
4  libsystem_platform.dylib 0x00007ff7b413e808 _sigtramp + 18446744072131666488
5  libsystem_c.dylib        0x00007ff812101b45 abort + 123
6  libsystem_c.dylib        0x00007ff812100e5e err + 0
7  swift-frontend           0x0000000112b0fe03 swift::NormalProtocolConformance::setWitness(swift::ValueDecl*, swift::Witness) const (.cold.6) + 35
8  swift-frontend           0x000000010dac476b swift::NormalProtocolConformance::setWitness(swift::ValueDecl*, swift::Witness) const + 187
9  swift-frontend           0x000000010ce732f0 swift::ModuleFile::finishNormalConformance(swift::NormalProtocolConformance*, unsigned long long) + 4880
10 swift-frontend           0x000000010dac2c3c swift::NormalProtocolConformance::getWitness(swift::ValueDecl*) const + 92
11 swift-frontend           0x000000010c801f3f (anonymous namespace)::SILGenWitnessTable<(anonymous namespace)::SILGenConformance>::addMethod(swift::SILDeclRef) + 335
12 swift-frontend           0x000000010c801d8f void llvm::function_ref<void (swift::AccessorDecl*)>::callback_fn<swift::SILWitnessVisitor<(anonymous namespace)::SILGenConformance>::visitAbstractStorageDecl(swift::AbstractStorageDecl*)::'lambda'(swift::AccessorDecl*)>(long, swift::AccessorDecl*) + 127
13 swift-frontend           0x000000010d90e8a0 swift::AbstractStorageDecl::visitExpectedOpaqueAccessors(llvm::function_ref<void (swift::AccessorKind)>) const + 48
14 swift-frontend           0x000000010d90e6dc swift::AbstractStorageDecl::visitOpaqueAccessors(llvm::function_ref<void (swift::AccessorDecl*)>) const + 44
15 swift-frontend           0x000000010c8019b2 swift::SILWitnessVisitor<(anonymous namespace)::SILGenConformance>::visitProtocolDecl(swift::ProtocolDecl*) + 2210
16 swift-frontend           0x000000010c7fe870 swift::Lowering::SILGenModule::getWitnessTable(swift::NormalProtocolConformance*) + 368
17 swift-frontend           0x000000010c6d9bdf swift::ASTLoweringRequest::evaluate(swift::Evaluator&, swift::ASTLoweringDescriptor) const + 2911
18 swift-frontend           0x000000010c7ea3a3 swift::SimpleRequest<swift::ASTLoweringRequest, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule>> (swift::ASTLoweringDescriptor), (swift::RequestFlags)9>::evaluateRequest(swift::ASTLoweringRequest const&, swift::Evaluator&) + 115
19 swift-frontend           0x000000010c6deece swift::ASTLoweringRequest::OutputType swift::Evaluator::getResultUncached<swift::ASTLoweringRequest, swift::ASTLoweringRequest::OutputType swift::evaluateOrFatal<swift::ASTLoweringRequest>(swift::Evaluator&, swift::ASTLoweringRequest)::'lambda'()>(swift::ASTLoweringRequest const&, swift::ASTLoweringRequest::OutputType swift::evaluateOrFatal<swift::ASTLoweringRequest>(swift::Evaluator&, swift::ASTLoweringRequest)::'lambda'()) + 318
20 swift-frontend           0x000000010c6da014 swift::performASTLowering(swift::FileUnit&, swift::Lowering::TypeConverter&, swift::SILOptions const&, swift::IRGenOptions const*) + 100
21 swift-frontend           0x000000010c0016d0 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 656
22 swift-frontend           0x000000010c0159bf withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) + 143
23 swift-frontend           0x000000010c0044b0 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 928
24 swift-frontend           0x000000010c003201 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3137
25 swift-frontend           0x000000010bdee30b swift::mainEntry(int, char const**) + 2539
26 dyld                     0x00007ff811e8141f start + 1903

@lorentey
Copy link
Member Author

macOS failure is in the stability-stdlib-source check. I'm very tempted to delete that test entirely, as it keeps getting in the way and it has so limited understanding of recent Swift changes that it no longer provides much value. But for now, I'll add another patch to its exception list.

@lorentey
Copy link
Member Author

@swift-ci test

lorentey added a commit to lorentey/swift that referenced this pull request Apr 17, 2024
swiftlang#73045 has uncovered a compiler crash with existing code of this form:

       try withExtendedLifetime(statement) { // 💥silgen crash
            do {
                try dbPool.close()
                XCTFail("Expected Error")
            } catch DatabaseError.SQLITE_BUSY { }
        }

This patterns can happen with any of the newly typed throwing entry points, but the source compat suite only seems to have caught this with `withExtendedLifetime`; let’s see if we can get away with only rolling back this one.

(cherry picked from commit 153c001)
@glessard
Copy link
Contributor

@swift-ci please test Windows platform

@lorentey
Copy link
Member Author

Joe's fix for the typed throw issue is up at #73092. I'll submit a separate PR to re-generalize withExtendedLifetime after that lands -- this doesn't need to hold up this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants