Skip to content

[Distributed] Re-implement ad-hoc requirements into dynamic witness table lookup for SerializationRequirement conformance #71435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

xedin
Copy link
Contributor

@xedin xedin commented Feb 7, 2024

  • SerializationRequirement of DistributedActorSystem became a primary associated type
  • remoteCall and remoteCallVoid of DistributedActorSystem are now requirements
  • Sema create abstract conformances for Res generic parameter of each remoteCall witness
  • IRGen is adjusted to produce witness tables for Res: SerializationRequirement that is required
    for every witness (but cannot be expressed on the requirement) of DistributedActorSystem.remoteCall
    dynamically in a protocol witness thunk.

@xedin
Copy link
Contributor Author

xedin commented Feb 7, 2024

This is still a draft because we need to adjust distributed thunk synthesis to not assume that actorSystem is always concrete and add some more Sema checks to make sure that SerializationRequirement is always supplied to distributed actor declarations.

@ktoso ktoso added the distributed Feature → concurrency: distributed actor label Feb 7, 2024
Copy link
Contributor

@ktoso ktoso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great so far

xedin added 19 commits February 12, 2024 13:23
…computeSubstitutions`

This is going to be used to determine whether the substitutions are
computed for `DistributedActorSystem::remoteCall` and adjust the
generic signature with witness conformance requirements.
…f `DistributedActorSystem.remoteCall`

Conformance requirement between on `Res` and `SerializationRequirement`
of `DistributedActorSystem.remoteCall` are not expressible at the moment
but they are verified by Sema so it's okay to omit them here and lookup
dynamically during IRGen.
…tedActorSystem.remoteCall` witnesses

When IRGen is building a protocol witness thunk for a
`DistributedActorSystem.remoteCall` requirement we
need to supply witness tables associated with `Res`
generic parameter which are not expressible on the
requirement because they come from `SerializationRequirement`
associated type.
…equirements

Instead of handing failures in `matchWitness` let's syntehsize
conformances during solving. Next step would be to record them
and use in `Solution::computeSubstitutions`.
…InvocationEncoder.record{Argument, ReturnType}`
…thesis

This is no longer necessary because `onReturn` is a protocol
requirement now.
…nesses with ad-hoc serialization requirement
…es for `actorSystem` references

Things like `makeInvocationEncoder()` and all of the `record*`
methods can now be called through witnesses.
…ss thunk

`decodeNextArgument` is now a requirement, so thunks should be
always calling that instead of trying to dispatch directly.
…ILFunction

Ad-hoc requirements are now obsolete by making `remoteCall`,
`record{Argument, ReturnType}`, `decodeNextArgument` protocols
requirements and injecting witness tables for `SerializationRequirement`
conformances during IRGen.
@xedin xedin force-pushed the fix-need-for-adhoc-distributed-requirements branch from 4e40cc4 to 0cc26cf Compare February 12, 2024 22:27
@xedin xedin marked this pull request as ready for review February 12, 2024 22:27
@xedin
Copy link
Contributor Author

xedin commented Feb 12, 2024

@swift-ci please test

@ktoso ktoso self-requested a review February 12, 2024 22:29
Copy link
Contributor

@ktoso ktoso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incredible work here, thanks so much @xedin. Great to see all the horrible ad hoc requirements gone.

Read through in depth, LGTM!

/// 'decodeNextArgument' which must be retained, as it is only used from IRGen
/// and such, appears as-if unused in SIL and would get optimized away.
// TODO: Consider making this a general "references adhoc functions" and make it an array?
SILFunction *RefAdHocRequirementFunction = nullptr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see this gone

void EmitPolymorphicParameters::injectAdHocDistributedRequirements() {
// FIXME: We need a better way to recognize that function is
// a thunk for witness of `remoteCall` requirement.
if (!Fn.hasLocation())
Copy link
Contributor

@ktoso ktoso Feb 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no this is a SILFunction I guess, so the below doesn't really help.

--

In my other PR I was actually going to add:

  /// Record which function this is a thunk for, we'll need this to link back
   /// calls in case this is a distributed requirement witness.
   thunk->getAttrs().add(
       new (C) DistributedThunkTargetAttr(func));

so we could check for it to detect a thunk;
https://github.com/apple/swift/pull/70928/files#diff-c5ee3c123c92773a4f07b456c2c0051c7a9dfdc81c5c0234ba5da6389bd82a07R793

I'll cleanup that PR shortly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Once that's there we can use it both here and in FunctionSignatureOpts optimizer pass.

@xedin xedin changed the title [Distributed] Implement dynamic lookup for Res of DistributedActorSystem.remoteCall [Distributed] Re-implement ad-hoc requirements into dynamic witness table lookup for SerializationRequirement conformance Feb 12, 2024
@xedin
Copy link
Contributor Author

xedin commented Feb 12, 2024

@swift-ci please test

@xedin
Copy link
Contributor Author

xedin commented Feb 13, 2024

@swift-ci please build toolchain macOS platform

@xedin xedin merged commit db020c1 into swiftlang:main Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed Feature → concurrency: distributed actor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants