[Distributed] distributed decls working across files #39087

ktoso · 2021-08-28T08:42:39Z

Looking into what's missing or done wrong because we can't get distributed actors work well across files it seems right now.

ktoso · 2021-08-28T09:12:26Z

E.g. Serialization/distributed.swift

+ /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift-frontend -target x86_64-apple-macosx10.9 -module-cache-path /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache -sdk /Applications/Xcode-Internal.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -swift-version 4 -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -typo-correction-limit 10 -emit-module -o /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Serialization/Output/distributed.swift.tmp-scratch/def_distributed~partial.swiftmodule -primary-file /Users/ktoso/code/swift-project/swift/test/Serialization/Inputs/def_distributed.swift -module-name def_distributed -disable-availability-checking -enable-experimental-distributed
[/Users/ktoso/code/swift-project/swift/lib/Sema/TypeCheckDistributed.cpp:218] (checkDistributedActor) CHECK DIST [DA]
[/Users/ktoso/code/swift-project/swift/lib/Sema/DerivedConformanceDistributedActor.cpp:173] (deriveDistributedActor) DERIVE: [resolve]
[/Users/ktoso/code/swift-project/swift/lib/Sema/DerivedConformanceDistributedActor.cpp:163] (deriveDistributedActor) DERIVE: [actorTransport]
[/Users/ktoso/code/swift-project/swift/lib/Sema/DerivedConformanceDistributedActor.cpp:163] (deriveDistributedActor) DERIVE: [id]
+ : 'RUN: at line 5'
+ echo '---- STEP 2 ---------------------------------------------------'
+ : 'RUN: at line 6'
+ /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift-frontend -target x86_64-apple-macosx10.9 -module-cache-path /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache -sdk /Applications/Xcode-Internal.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -swift-version 4 -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -typo-correction-limit 10 -merge-modules -emit-module -parse-as-library -enable-testing /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Serialization/Output/distributed.swift.tmp-scratch/def_distributed~partial.swiftmodule -module-name def_distributed -o /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Serialization/Output/distributed.swift.tmp/def_distributed.swiftmodule -disable-availability-checking -enable-experimental-distributed
+ : 'RUN: at line 7'
+ echo '---- STEP 3 ---------------------------------------------------'
+ : 'RUN: at line 8'
+ /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift-frontend -target x86_64-apple-macosx10.9 -module-cache-path /Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache -sdk /Applications/Xcode-Internal.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -swift-version 4 -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -typo-correction-limit 10 -typecheck -I/Users/ktoso/code/swift-project/build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Serialization/Output/distributed.swift.tmp -verify /Users/ktoso/code/swift-project/swift/test/Serialization/distributed.swift -verify-ignore-unknown -disable-availability-checking -enable-experimental-distributed
[/Users/ktoso/code/swift-project/swift/lib/Sema/TypeCheckDeclPrimary.cpp:2933] (visitExtensionDecl) VISIT EXTENSION
[/Users/ktoso/code/swift-project/swift/lib/Sema/TypeCheckDistributed.cpp:218] (checkDistributedActor) CHECK DIST [DA]
[/Users/ktoso/code/swift-project/swift/lib/Sema/TypeCheckDeclPrimary.cpp:2938] (visitExtensionDecl) VISIT DIST ACTOR MEMBERS FROM EXTENSION
[/Users/ktoso/code/swift-project/swift/lib/Sema/TypeCheckDeclPrimary.cpp:2943] (visitExtensionDecl) VISIT MEMBER: [_impl_doSomethingDistributed]
/Users/ktoso/code/swift-project/swift/test/Serialization/distributed.swift:17:14: error: unexpected error produced: incorrect argument label in call (have 'transport:', expected 'from:')
  let da = DA(transport: transport)
             ^
/Users/ktoso/code/swift-project/swift/test/Serialization/distributed.swift:17:26: error: unexpected error produced: argument type 'ActorTransport' does not conform to expected type 'Decoder'
  let da = DA(transport: transport)
                         ^
def_distributed.DA:5:16: error: diagnostic produced elsewhere: property 'actorTransport' cannot be defined explicitly, as it conflicts with distributed actor synthesized stored property
    public let actorTransport: ActorTransport
               ^
def_distributed.DA:6:16: error: diagnostic produced elsewhere: property 'id' cannot be defined explicitly, as it conflicts with distributed actor synthesized stored property
    public let id: AnyActorIdentity
               ^

Pretty weird since the synthesis did trigger nicely and once when it's supposed to.. but then in the checking of the second file, it seems as if all those are not visible.

ktoso · 2021-08-28T09:15:07Z

@swift-ci please smoke test

ktoso · 2021-08-28T09:15:21Z

@swift-ci please build toolchain macOS

ktoso · 2021-08-28T09:16:45Z

Huh, actually uncovered and fixed an initializer visibility issue -- otherwise things seem good actually 🤔

swift-ci · 2021-08-28T14:38:32Z

macOS Toolchain
Download Toolchain
Git Sha - e004e6a

Install command
tar -zxf swift-PR-39087-1109-osx.tar.gz --directory ~/

ktoso · 2021-08-29T12:04:24Z

@swift-ci please smoke test

Failure was only windows, but let's see if hardening helped there now.

ktoso · 2021-08-30T00:37:07Z

lib/Sema/CodeSynthesis.cpp

@@ -321,6 +321,9 @@ static ConstructorDecl *createImplicitConstructor(NominalTypeDecl *decl,
           "Only 'distributed actor' type can gain implicit distributed actor init");

    if (swift::ensureDistributedModuleLoaded(decl)) {
+      // copy access level of distributed actor init from the nominal decl
+      accessLevel = decl->getEffectiveAccess();


this fixes the accessibility of the created init to be the same as the class for a dist actor.

ktoso · 2021-08-30T12:48:25Z

lib/Sema/TypeCheckDecl.cpp

+
+      if (auto *func = dyn_cast<FuncDecl>(member)) {
+        (void) func->getDistributedActorRemoteFuncDecl();
+      }


so basically we need to trigger synthesis/"get" in same spots as property wrappers most of the time -- one is here, and another is in lookups

(The lookups solve the across files case.)

ktoso · 2021-08-30T12:52:55Z

@swift-ci please smoke test

ktoso · 2021-08-30T12:53:15Z

@swift-ci please build toolchain macOS

ktoso · 2021-08-30T14:47:41Z

Linux failure seems to be unrelated


<unknown>:0: error: module file '/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/buildbot_linux/foundation-linux-x86_64/module-cache/3NQ8WC0WFC565/SwiftShims-FOQQR403J715.pcm' is out of date and needs to be rebuilt: signature mismatch
23:10:59 <unknown>:0: note: imported by module 'SwiftOverlayShims' in '/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/buildbot_linux/foundation-linux-x86_64/module-cache/3NQ8WC0WFC565/SwiftOverlayShims-FOQQR403J715.pcm'
23:10:59 
/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swift-corelibs-foundation/Sources/Foundation/Data.swift:25:8: error: missing required module 'SwiftOverlayShims'
23:10:59 import Glibc
23:10:59        ^
23:10:59 ninja: build stopped: subcommand failed.

ktoso · 2021-08-30T14:47:53Z

@swift-ci please smoke test Linux

swift-ci · 2021-08-30T17:51:46Z

macOS Toolchain
Download Toolchain
Git Sha - d76690e

Install command
tar -zxf swift-PR-39087-1111-osx.tar.gz --directory ~/

ktoso · 2021-08-30T21:42:20Z

Confirmed things work thanks to this addition!

WIP serialization, many files etc

9f1fedb

ktoso requested review from slavapestov, DougGregor and kavon and removed request for slavapestov August 28, 2021 08:57

[Distributed] fix access level of synthesized default dist init

e004e6a

[Distributed] Harden distributed_actor_deinit.swift

7763528

ktoso commented Aug 30, 2021

View reviewed changes

ktoso force-pushed the wip-serialization-dist branch from 707c902 to 7dec875 Compare August 30, 2021 12:46

ktoso commented Aug 30, 2021

View reviewed changes

[Distributed] improve getting _remote funcs; cache gets

d76690e

ktoso force-pushed the wip-serialization-dist branch from 7dec875 to d76690e Compare August 30, 2021 12:51

ktoso merged commit 38c8544 into swiftlang:main Aug 30, 2021

ktoso added the distributed Feature → concurrency: distributed actor label Aug 30, 2021

ktoso deleted the wip-serialization-dist branch August 30, 2021 21:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Distributed] distributed decls working across files #39087

[Distributed] distributed decls working across files #39087

Uh oh!

ktoso commented Aug 28, 2021 •

edited

Loading

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

swift-ci commented Aug 28, 2021

Uh oh!

ktoso commented Aug 29, 2021 •

edited

Loading

Uh oh!

ktoso Aug 30, 2021

Uh oh!

ktoso Aug 30, 2021

Uh oh!

ktoso Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

swift-ci commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

Uh oh!

[Distributed] distributed decls working across files #39087

[Distributed] distributed decls working across files #39087

Uh oh!

Conversation

ktoso commented Aug 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

ktoso commented Aug 28, 2021

Uh oh!

swift-ci commented Aug 28, 2021

Uh oh!

ktoso commented Aug 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ktoso Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

ktoso Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

ktoso Aug 30, 2021

Choose a reason for hiding this comment

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

swift-ci commented Aug 30, 2021

Uh oh!

ktoso commented Aug 30, 2021

Uh oh!

Uh oh!

ktoso commented Aug 28, 2021 •

edited

Loading

ktoso commented Aug 29, 2021 •

edited

Loading