[Distributed] Remove @_dynamic replacements; impl remoteCall ad-hoc reqs #41036

ktoso · 2022-01-27T08:57:57Z

This gets rid of the dynamic member replacement mechanism for remote calls completely -- meaning... we don't need source generators anymore!

Remote calls are performed via synthesized SIL thunks which record call representations using an Encoder type. This is then passed to an ad-hoc protocol requirement "remoteCall" which is able to perform the call. This is a huge improvement in how actor systems can be developed and also an increase in expressive power -- we can express and call generic methods, support property wrappers on parameters and also potentially carry any additional information about the called functions with new record methods if we ever need to.

The SILGenDistributed emission is quite messy still but I will continue working on cleaning it up in the coming days.
We will also cleanup and use more of SerializationRequirement which today is hardcoded a little bit still -- pending changes that we accepted in SE-0336

Squashed commit of the following: commit e5a05ff Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 27 17:45:31 2022 +0900 cleanup commit 1f751ce Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 27 14:50:33 2022 +0900 cleanups commit c632f32 Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 27 14:01:09 2022 +0900 add test for generic from actor decl commit 09b8bd5 Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 27 14:00:58 2022 +0900 cleanups commit 31f4d0c Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 27 11:40:51 2022 +0900 fix test commit ad4db2f Merge: 97227ed 07e2dfd Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 26 23:31:41 2022 +0900 Merge branch 'main' into wip-zzz commit 97227ed Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 26 21:01:25 2022 +0900 remove @_dynamic methods! fix tests commit 1c79344 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 19 12:51:09 2022 +0900 cleanup wip stuck fixed the stack cleanups cleanups pretty good now weird load rki works remove hack add take + throw + return fix test more tests fixed more tests fixed more tests fixed commit 3ed494c Author: Konrad `ktoso` Malawski <[email protected]> Date: Tue Jan 18 21:09:28 2022 +0900 stack issues in SIL verification commit 5cf43a7 Author: Konrad `ktoso` Malawski <[email protected]> Date: Tue Jan 18 09:19:51 2022 +0900 about to call the remoteCall goot to return, but missing subs commit df8e471 Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 13 14:09:49 2022 +0900 [Distributed] Refactor Invocation to Decoder/Encoder getting there done-recording working on the string init stuck trying to get String initializer SILFunction created the remote call target commit fc7bd62 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 23:01:14 2022 +0900 [Distributed] Pass arguments from Invocation to HBuffer commit cafc2cc Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 22:08:47 2022 +0900 works commit a7d0183 Author: Pavel Yaskevich <[email protected]> Date: Tue Jan 11 15:48:58 2022 -0800 [Distributed] Adjust interface of `swift_distributed_execute_target` Since this is a special function, `calleeContext` doesn't point to a direct parent but instead both parent context (uninitialized) and resume function are passed as last arguments which means that `callContext` has to act as an intermediate context in call to accessor. commit c1f830b Author: Pavel Yaskevich <[email protected]> Date: Tue Jan 11 17:00:08 2022 -0800 [Distributed] Drop optionality from result buffer in `_executeDistributedTarget` `RawPointer?` is lowered into a two arguments since it's a struct, to make it easy let's just allocate an empty pointer for `Void` result. commit c83c2c3 Author: Pavel Yaskevich <[email protected]> Date: Tue Jan 11 17:02:45 2022 -0800 [Distributed] NFC: Update _remoteCall test-case to check multiple different result types commit 29e7cf5 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 21:32:37 2022 +0900 wip commit 9128ecc Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 20:46:20 2022 +0900 wip commit a6b2a62 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 20:38:22 2022 +0900 wip commit 8b188f0 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 16:55:10 2022 +0900 wip commit 3796bec Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Jan 12 16:55:02 2022 +0900 wip commit 0ffc68b Author: Konrad `ktoso` Malawski <[email protected]> Date: Tue Jan 11 21:44:58 2022 +0900 [Distributed] Implementing ad-hoc protocol requirements commit 7886257 Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 6 18:03:54 2022 +0900 cleanup commit 5f4ab89 Merge: 24a628e fdda6f2 Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 6 15:51:39 2022 +0900 Merge branch 'main' into wip-impl-execute-swift commit 24a628e Author: Konrad `ktoso` Malawski <[email protected]> Date: Thu Jan 6 15:33:21 2022 +0900 wip commit 69e7fed Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Dec 22 06:36:45 2021 +0900 [Distributed] comment out distributed_actor_remoteCall for now commit 376733a Author: Konrad `ktoso` Malawski <[email protected]> Date: Tue Dec 21 16:00:06 2021 +0900 reimplement distributed get type info impls commit 74ab478 Author: Konrad `ktoso` Malawski <[email protected]> Date: Wed Dec 15 21:37:08 2021 +0900 [Distributed] Implement func metadata and executeDistributedTarget dont expose new entrypoints able to get all the way to calling _execute

ktoso · 2022-01-27T08:58:04Z

@swift-ci please smoke test

ktoso · 2022-01-27T10:19:19Z

@swift-ci please smoke test

ktoso · 2022-01-27T11:00:36Z

lib/AST/NameLookup.cpp

@@ -1385,31 +1385,6 @@ NominalTypeDecl::lookupDirect(DeclName name,
                           DirectLookupRequest({this, name, flags}), {});
 }

-AbstractFunctionDecl*
-NominalTypeDecl::lookupDirectRemoteFunc(AbstractFunctionDecl *func) {


removed all code about dynamic _remote funcs 🥳

ktoso · 2022-01-27T11:00:52Z

lib/AST/NameLookup.cpp

@@ -1816,9 +1762,6 @@ QualifiedLookupRequest::evaluate(Evaluator &eval, const DeclContext *DC,
    // Make sure we've resolved property wrappers, if we need them.
    installPropertyWrapperMembersIfNeeded(current, member);

-    // Make sure we've resolved synthesized _remote funcs for distributed actors.
-    installDistributedRemoteFunctionsIfNeeded(current, member);


less terrible hacks! 🥳

ktoso · 2022-01-27T11:54:13Z

lib/SILGen/SILGenDistributed.cpp

+      auto remoteCallTargetMetatype = getLoweredType(MetatypeType::get(remoteCallTargetTy));
+      auto remoteCallTargetMetatypeValue = B.createMetatype(loc, remoteCallTargetMetatype);
+
+      // %30 = string_literal utf8 "MANGLED_NAME" // user: %35


I'm working on making this an util on SGF

Done! a5c2236 🥳
Proud of that one, finally figured it out...

ktoso · 2022-01-27T12:29:49Z

test/Distributed/distributed_actor_accessor_thunks_64bit.swift

-// CHECK: {{.*}} = call i1 (i8*, i1, ...) @llvm.coro.end.async({{.*}}, i8* {{.*}}, %swift.context* {{.*}}, %swift.error* {{.*}})
+// SKIP: [[THUNK_REF:%.*]] = bitcast void (%swift.context*, %T27distributed_actor_accessors7MyActorC*)* {{.*}} to i8*
+// SKIP: {{.*}} = call { i8*, %swift.error* } (i32, i8*, i8*, ...) @llvm.coro.suspend.async.sl_p0i8p0s_swift.errorss({{.*}}, i8* [[THUNK_REF]], %swift.context* {{.*}}, %T27distributed_actor_accessors7MyActorC* {{.*}})
+// SKIP: {{.*}} = call i1 (i8*, i1, ...) @llvm.coro.end.async({{.*}}, i8* {{.*}}, %swift.context* {{.*}}, %swift.error* {{.*}})


those seem gone... and direct calls are just made

ktoso · 2022-01-27T12:33:45Z

@swift-ci please smoke test

ktoso · 2022-01-27T12:39:12Z

lib/SILGen/SILGenFunction.h

@@ -1072,6 +1072,13 @@ class LLVM_LIBRARY_VISIBILITY SILGenFunction
  SourceLocArgs
  emitSourceLocationArgs(SourceLoc loc, SILLocation emitLoc);

+  ManagedValue


this gained comments in the next commit

ktoso · 2022-01-27T23:20:35Z

@swift-ci please smoke test Linux

ktoso added the distributed Feature → concurrency: distributed actor label Jan 27, 2022

ktoso force-pushed the wip-distributed-remove-dynamic-sil-remotecall-squashed branch 2 times, most recently from 2d4ff3a to 1075912 Compare January 27, 2022 10:19

ktoso force-pushed the wip-distributed-remove-dynamic-sil-remotecall-squashed branch from 1075912 to 86b080f Compare January 27, 2022 10:19

[Distributed] IRGen seems to not cast but call direct now

1dd4372

ktoso force-pushed the wip-distributed-remove-dynamic-sil-remotecall-squashed branch from 86b080f to 1dd4372 Compare January 27, 2022 10:20

ktoso commented Jan 27, 2022

View reviewed changes

[Distributed] remove unused code

6622a3c

ktoso commented Jan 27, 2022

View reviewed changes

ktoso added 2 commits January 27, 2022 21:19

[SGF][Distributed] emitStringLiteral and use it in distributed

a5c2236

[Distributed] cleanups: use more managed values and cleanups

6da455e

ktoso commented Jan 27, 2022

View reviewed changes

ktoso merged commit 89b0a4c into swiftlang:main Jan 28, 2022

ktoso deleted the wip-distributed-remove-dynamic-sil-remotecall-squashed branch January 28, 2022 02:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Distributed] Remove @_dynamic replacements; impl remoteCall ad-hoc reqs #41036

[Distributed] Remove @_dynamic replacements; impl remoteCall ad-hoc reqs #41036

Uh oh!

ktoso commented Jan 27, 2022 •

edited

Loading

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022 •

edited

Loading

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

Uh oh!

[Distributed] Remove @_dynamic replacements; impl remoteCall ad-hoc reqs #41036

[Distributed] Remove @_dynamic replacements; impl remoteCall ad-hoc reqs #41036

Uh oh!

Conversation

ktoso commented Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

ktoso Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

ktoso Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

ktoso Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

ktoso Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

ktoso Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

ktoso commented Jan 27, 2022

Uh oh!

Uh oh!

ktoso commented Jan 27, 2022 •

edited

Loading

ktoso Jan 27, 2022 •

edited

Loading