SE-0309: Covariant erasure for dependent member types #39492

AnthonyLatsis · 2021-09-28T18:33:04Z

This change enables us to access existential members when their type signatures, as seen in the context of the base type, reference Self-rooted associated types only in covariant positions, by type-erasing these references to their upper bounds. This implies that you can use a protocol value to call a method that returns an associated type, or a method that accepts one if the associated type is bound to a fully concrete type via generic requirements on the protocol. Accessing these members becomes possible because their types are adjusted so as to become well-defined from outside their protocol context, which is key to type-safe member accesses on a type-erased value.

protocol P {
  associatedtype A: P
  func method1() -> A
  func method2(_: A) 
}
protocol PBool: P where A == Bool {}

func test(p: P, pBool: PBool) {
  let _: P = p.method1() // ok, p.method1 has type () -> P
  
  pBool.method2(true) // ok, pBool.method2 has type (Bool) -> Void
}

include/swift/AST/Type.h

include/swift/AST/Types.h

CodaFi · 2021-09-29T17:28:04Z

lib/AST/Type.cpp

+        continue;
+      }
+      inputInfo |= ::findExistentialSelfReferences(
+          sig, param.getParameterType(), TypePosition::Contravariant);


This isn't quite right. In ((T) -> U) -> V, the input (T) -> U is contravariant, so (T) is covariant and U is contravariant in those positions. In general, recurring under the input side of an arrow flips variance.

This function is the entry point for a member ValueDecl. We do flip variance in the recursive findExistentialSelfReferences that takes a type and a relative variance position.

AnthonyLatsis · 2021-12-21T19:08:37Z

@swift-ci please smoke test

lib/Sema/TypeCheckDecl.cpp

AnthonyLatsis · 2021-12-21T19:53:13Z

lib/AST/Type.cpp

    auto gather = ExpansionGatherer{fn};
-    Type transformedPat = expand->getPatternType().transformRec(gather);
+    Type transformedPat =
+        expand->getPatternType().transformWithPosition(pos, gather);


@CodaFi I am intuitively treating packs as invariant (because they seem like they cannot appear within another type), and expansions as covariant here; let me know if anything sounds fishy about that.

AnthonyLatsis · 2021-12-21T19:54:47Z

@swift-ci please smoke test macOS

AnthonyLatsis · 2021-12-21T22:43:45Z

@swift-ci please clean smoke test macOS

AnthonyLatsis · 2021-12-22T06:40:09Z

@swift-ci please smoke test

AnthonyLatsis · 2021-12-25T07:10:16Z

@swift-ci please smoke test

AnthonyLatsis · 2021-12-25T09:40:06Z

@swift-ci please test source compatibility

AnthonyLatsis · 2022-01-21T08:08:48Z

@swift-ci please smoke test

AnthonyLatsis · 2022-01-22T11:48:28Z

@swift-ci please smoke test

AnthonyLatsis · 2022-01-22T15:02:08Z

@swift-ci please clean smoke test Linux

AnthonyLatsis · 2022-01-22T15:02:33Z

@swift-ci please test source compatibility

AnthonyLatsis · 2022-01-22T22:40:44Z

@slavapestov This is finally ready for another review.

DougGregor · 2022-02-01T17:24:00Z

@swift-ci please smoke test

…and co.

DougGregor

I haven't reviewed earlier drafts of this pull request, but in it's current state it's looking fantastic. I have just a few review comments, but nothing that would prevent us from merging this.

DougGregor · 2022-02-01T17:43:29Z

include/swift/AST/Types.h

@@ -616,7 +616,7 @@ class alignas(1 << TypeAlignInBits) TypeBase

  /// Determine whether the type involves the given opened existential
  /// archetype.
-  bool hasOpenedExistential(OpenedArchetypeType *opened);
+  bool hasOpenedExistentialWithRoot(const OpenedArchetypeType *root) const;


I was looking at this recently, and wondering whether the GenericEnvironment itself is a better discriminator than the root archetype for the long term. Right now, any opened existential always has a single generic parameter at the top level, but it's not inconceivable that we would want to allow matching existentials to structural types with type parameters, e.g., x as <T, U>(T, U) to open something as a 2-tuple. In such cases, the root no longer uniquely identifies the opened existential. We don't have to make this change to use GenericEnvironment---your change here makes sure clients are properly reasoning about the roots so they handle nested archetypes properly, which is a huge step forward.

DougGregor · 2022-02-01T17:48:41Z

lib/AST/GenericEnvironment.cpp

-      return mapTypeIntoContext(type, conformanceLookupFn);
-
+      if (type->hasTypeParameter()) {
+        return mapTypeIntoContext(type, conformanceLookupFn);


I don't understand why we need this check for type parameters here. Are we tripping the assertion in mapTypeIntoContext in some way?

Yeah, we trip it when we encounter an archetype from somewhere else. Consider the following:

class Class<A>: P {} protocol P { associatedtype A } protocol Q: P { func method(_: A) } extension Q { func test(arg: Q & Class<Self>) { arg.method(self) // OK } }

The opened archetype signature for Q & Class<Self> is <Self where Self : Q, Self : Class<archetype>>.

We were discussing this internally and we concluded the best approach is to model the opened existential's environment as being derived from the outer generic signature. So in your example, it becomes

<A, Self where Self : Q, Self : Class<A>>

However this is a big refactoring and outside of the scope of this PR.

DougGregor · 2022-02-01T17:49:23Z

lib/AST/GenericEnvironment.cpp

+      // building a root OpenedArchetypeType, and the latter when building
+      // nested archetypes.
+      // For compatibility, continue using the existential layout's version when
+      // the interface type is a generic parameter. We should align these at


Good choice; we might as well get the better answer for nested types.

DougGregor · 2022-02-01T17:52:56Z

lib/Sema/ConstraintSystem.cpp

@@ -1727,6 +1727,103 @@ static bool isMainDispatchQueueMember(ConstraintLocator *locator) {
  return true;
 }

+/// Type-erase occurences of covariant 'Self'-rooted type parameters to their


Typo "occurences"

DougGregor · 2022-02-01T18:02:22Z

lib/Sema/ConstraintSystem.cpp

+      return None;
+    });
+
+    // Then, type-erase occurences of covariant 'Self'-rooted type parameters.


Typo occurences. This three-step transform makes a ton of sense. getTypeOfMemberReference is getting huge, so at some point I'd like us to factor the transform out into a separate function.

DougGregor · 2022-02-01T18:06:09Z

include/swift/AST/Types.h

@@ -5686,6 +5687,12 @@ class OpenedArchetypeType final : public ArchetypeType,
  /// Retrieve the opened existential type
  Type getOpenedExistentialType() const;


This function is a bit of a footgun, because even on a nested opened archetype it still returns the top-level existential type, rather than the erased type for this particular archetype. We probably want to do a quick audit of clients to ensure that they are only using this on the root archetype, and consider renaming the API (getOpenedExistentialTypeAtRoot or similar) or removing it entirely (clients can ask for the opened existential type on the generic environment, which is obviously shared amongst all archetypes in that opened existential).

Noticed this too; I will explore your suggestions in a follow-up.

DougGregor · 2022-02-01T19:14:51Z

Ah, looks like some minor bit-rot due to the change in the signature of transformRec. @AnthonyLatsis would you like to fix things up to get this merged, or would you like some help?

AnthonyLatsis · 2022-02-01T19:20:08Z

@DougGregor already at it!

While we're here, also tidy up the documentation for the type transform APIs

…uild "nested" opened archetypes

AnthonyLatsis · 2022-02-01T21:49:46Z

@swift-ci please smoke test

…existential member availability

…ember accesses

…in requirement failure messages

…iagnostics

…ExistentialSelfReferences'

AnthonyLatsis · 2022-02-01T23:15:54Z

@swift-ci please smoke test

AnthonyLatsis · 2022-02-02T04:07:08Z

Thanks to all for reviewing!

slavapestov reviewed Sep 28, 2021

View reviewed changes

include/swift/AST/Type.h Outdated Show resolved Hide resolved

CodaFi reviewed Sep 29, 2021

View reviewed changes

include/swift/AST/Types.h Outdated Show resolved Hide resolved

CodaFi reviewed Sep 29, 2021

View reviewed changes

AnthonyLatsis force-pushed the se-309-2 branch 2 times, most recently from eca1fae to 9f21b12 Compare October 1, 2021 09:48

AnthonyLatsis force-pushed the se-309-2 branch from 9f21b12 to f053281 Compare October 7, 2021 20:58

AnthonyLatsis marked this pull request as ready for review October 7, 2021 21:30

This was referenced Oct 8, 2021

TypeResolution: Abolish TypeResolutionStage::Contextual #39652

Merged

Sema: Explicit stricter minimization of protocol composition types for redeclaration checking #39694

Merged

AnthonyLatsis force-pushed the se-309-2 branch from f053281 to 7f3d9b3 Compare December 2, 2021 22:11

AnthonyLatsis force-pushed the se-309-2 branch 2 times, most recently from dcf6b2f to 761b98f Compare December 20, 2021 20:21

AnthonyLatsis force-pushed the se-309-2 branch from 761b98f to 371b7a0 Compare December 21, 2021 19:07

AnthonyLatsis requested a review from slavapestov December 21, 2021 19:08

AnthonyLatsis commented Dec 21, 2021

View reviewed changes

lib/Sema/TypeCheckDecl.cpp Outdated Show resolved Hide resolved

AnthonyLatsis commented Dec 21, 2021

View reviewed changes

AnthonyLatsis force-pushed the se-309-2 branch from 371b7a0 to c627a2c Compare December 22, 2021 06:36

AnthonyLatsis force-pushed the se-309-2 branch from c627a2c to 4581c95 Compare December 25, 2021 07:07

AnthonyLatsis force-pushed the se-309-2 branch 3 times, most recently from 9f24e75 to 891689f Compare January 21, 2022 08:07

AnthonyLatsis force-pushed the se-309-2 branch from 891689f to 1723baf Compare January 22, 2022 11:48

AnthonyLatsis added 3 commits February 1, 2022 20:55

[NFC] Preemptively relocate ProtocolDecl::isAvailableInExistential() …

3206f5a

…and co.

Sema: Handle generic-signature-based existential availability

427190d

[NFC] Turn SelfReferenceKind into a standalone utility type

b2fe028

DougGregor approved these changes Feb 1, 2022

View reviewed changes

hborla mentioned this pull request Feb 1, 2022

[5.6][SE-0309] Disable SE-0309 in Swift 5.6. #41131

Merged

AnthonyLatsis added 2 commits February 1, 2022 23:00

AST: Integrate variance positions into the type transform

f90120e

While we're here, also tidy up the documentation for the type transform APIs

GenericEnvironment: Don't use the existential layout's protocols to b…

756bf30

…uild "nested" opened archetypes

AnthonyLatsis force-pushed the se-309-2 branch from 1723baf to 9bbe2f9 Compare February 1, 2022 21:49

AnthonyLatsis added 8 commits February 2, 2022 02:09

AST, Sema: Use the opened archetype's generic signature to determine …

b3ee4b0

…existential member availability

Diag: Silence argument mismatch failures in unsupported existential m…

8e0e414

…ember accesses

Diag: Use 'Self'-rooted interface types instead of opened archetypes …

c47f99a

…in requirement failure messages

Sema: Enable covariant erasure for dependent member types

24c19d4

Sema: Use the type of the closed existential in 'unused expression' d…

57e8064

…iagnostics

SILGen: Handle nested opened archetypes and add tests

60c85b0

[NFC] Rename some test files

e849375

AST: Minor optimization for ProtocolCompositionType handling in 'find…

6a25198

…ExistentialSelfReferences'

AnthonyLatsis force-pushed the se-309-2 branch from 9bbe2f9 to 6a25198 Compare February 1, 2022 23:15

AnthonyLatsis merged commit c7fd60f into swiftlang:main Feb 2, 2022

AnthonyLatsis deleted the se-309-2 branch February 2, 2022 04:07

AnthonyLatsis changed the title ~~SE-309: Covariant erasure for dependent member types~~ SE-0309: Covariant erasure for dependent member types Apr 29, 2022

AnthonyLatsis added the swift evolution approved Flag → feature: A feature that was approved through the Swift evolution process label Apr 29, 2022

		@@ -5686,6 +5687,12 @@ class OpenedArchetypeType final : public ArchetypeType,
		/// Retrieve the opened existential type
		Type getOpenedExistentialType() const;

SE-0309: Covariant erasure for dependent member types #39492

SE-0309: Covariant erasure for dependent member types #39492

Uh oh!

Conversation

AnthonyLatsis commented Sep 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnthonyLatsis commented Dec 21, 2021

Uh oh!

Uh oh!

AnthonyLatsis Dec 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnthonyLatsis commented Dec 21, 2021

Uh oh!

AnthonyLatsis commented Dec 21, 2021

Uh oh!

AnthonyLatsis commented Dec 22, 2021

Uh oh!

AnthonyLatsis commented Dec 25, 2021

Uh oh!

AnthonyLatsis commented Dec 25, 2021

Uh oh!

AnthonyLatsis commented Jan 21, 2022

Uh oh!

AnthonyLatsis commented Jan 22, 2022

Uh oh!

AnthonyLatsis commented Jan 22, 2022

Uh oh!

AnthonyLatsis commented Jan 22, 2022

Uh oh!

AnthonyLatsis commented Jan 22, 2022

Uh oh!

DougGregor commented Feb 1, 2022

Uh oh!

DougGregor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnthonyLatsis Feb 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DougGregor commented Feb 1, 2022

Uh oh!

AnthonyLatsis commented Feb 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AnthonyLatsis commented Feb 1, 2022

Uh oh!

AnthonyLatsis commented Feb 1, 2022

Uh oh!

AnthonyLatsis commented Feb 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

AnthonyLatsis commented Sep 28, 2021 •

edited

Loading

AnthonyLatsis Dec 21, 2021 •

edited

Loading

AnthonyLatsis Feb 1, 2022 •

edited

Loading

AnthonyLatsis commented Feb 1, 2022 •

edited

Loading

AnthonyLatsis commented Feb 2, 2022 •

edited

Loading