Skip to content

[GSB] Term rewriting for same-type constraints #14454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 8, 2018

Conversation

DougGregor
Copy link
Member

@DougGregor DougGregor commented Feb 7, 2018

Introduce a new representation of same-type constraints as rewrite
rules in a term-rewriting system, where each same-type constraint maps
to a rewrite rule that produces a "more canonical"
term. Fully-simplifying a type according to these rewrite rules will
produce the canonical representation of that type, i.e., the "anchor"
of its equivalence class. Use this term-rewriting system to replace the
existing, weird "anchor" computation.

The rewrite rules are stored as a prefix tree, such that a "match"
operation walks the tree matching the prefix of a path of associated
type references, e.g., SubSequence -> SubSequence -> Element, and any
node within the tree can provide a replacement path (e.g.,
"Element"). The "dump" operation provides a visualization of the tree,
e.g.,

`--(cont'd)
    `--Sequence.Iterator
    |   `--IteratorProtocol.Element --> [Sequence.Element]
    `--Sequence.SubSequence --> []
    |   `--Sequence.Element --> [Sequence.Iterator -> IteratorProtocol.Element]
    |   |   `--(cont'd) --> [Sequence.Element]
    |   `--Sequence.Iterator --> [Sequence.Iterator]
    |   |   `--IteratorProtocol.Element --> [Sequence.Iterator -> IteratorProtocol.Element]
    |   `--Sequence.SubSequence --> []
    |   |   `--Sequence.Element --> [Sequence.Element]
    |   |   `--Sequence.Iterator --> [Sequence.Iterator]
    |   `--C.Index --> [C.Index]
    |   `--C.Indices --> [C.Indices]
    |       `--Sequence.Element --> [C.Indices -> Sequence.Element]
    |       `--Sequence.SubSequence --> [C.Indices -> Sequence.SubSequence]
    |       `--C.Index --> [C.Indices -> C.Index]
    |       `--C.Indices --> [C.Indices -> C.Indices]
    `--C.Index --> [Sequence.Element]
    `--C.Indices
        `--Sequence.Element --> [C.Index]

Introduce a new representation of same-type constraints as rewrite
rules in a term-rewriting system, where each same-type constraint maps
to a rewrite rule that produces a "more canonical"
term. Fully-simplifying a type according to these rewrite rules will
produce the canonical representation of that type, i.e., the "anchor"
of its equivalence class.

The rewrite rules are stored as a prefix tree, such that a "match"
operation walks the tree matching the prefix of a path of associated
type references, e.g., SubSequence -> SubSequence -> Element, and any
node within the tree can provide a replacement path (e.g.,
"Element"). The "dump" operation provides a visualization of the tree,
e.g.,

::
  `--(cont'd)
      `--Sequence.Iterator
      |   `--IteratorProtocol.Element --> [Sequence.Element]
      `--Sequence.SubSequence --> []
      |   `--Sequence.Element --> [Sequence.Iterator -> IteratorProtocol.Element]
      |   |   `--(cont'd) --> [Sequence.Element]
      |   `--Sequence.Iterator --> [Sequence.Iterator]
      |   |   `--IteratorProtocol.Element --> [Sequence.Iterator -> IteratorProtocol.Element]
      |   `--Sequence.SubSequence --> []
      |   |   `--Sequence.Element --> [Sequence.Element]
      |   |   `--Sequence.Iterator --> [Sequence.Iterator]
      |   `--C.Index --> [C.Index]
      |   `--C.Indices --> [C.Indices]
      |       `--Sequence.Element --> [C.Indices -> Sequence.Element]
      |       `--Sequence.SubSequence --> [C.Indices -> Sequence.SubSequence]
      |       `--C.Index --> [C.Indices -> C.Index]
      |       `--C.Indices --> [C.Indices -> C.Indices]
      `--C.Index --> [Sequence.Element]
      `--C.Indices
          `--Sequence.Element --> [C.Index]

Thus far, there are no clients for this information: this commit
starts building these rewriting trees and can visualize them for debug
information.
Fully rewriting a given type will produce the canonical representation of that type, because all rewrite rules take a step toward a more-canonical type. Use this approach to compute the anchor of an equivalence class, replacing the existing ad hoc approach of enumerating known potential 
archetypes—which was overly dependent on having the “right” set of
potential archetypes already computed.
Previously, the term rewriting logic for same-type constraints was only 
able to express “relative” rewrites, where the base of the type being 
rewritten (i.e., the generic type at the root) is unchanged. This meant
that we were unable to capture same-type constraints such as “T == U” or
“T.Foo == T” within the rewriting system.

Separate out the notion of a rewrite path and extend it with an optional
new base. When we’re simplifying a term and we encounter one of these
replacements, the whole type from the base up through the last-matched
path component gets replaced with the replacement path.
Use term rewriting exclusively in the computation of the canonical
dependent type (anchor) for an equivalence class, eliminating any dependence
on the specific potential archetypes and eliminating the hacks around
generic type parameters.
@DougGregor DougGregor changed the title [WIP] [GSB] Term rewriting for same-type constraints [GSB] Term rewriting for same-type constraints Feb 8, 2018
@DougGregor
Copy link
Member Author

@swift-ci please smoke test

@DougGregor
Copy link
Member Author

@swift-ci please test source compatibility

@DougGregor
Copy link
Member Author

@swift-ci please test compiler performance

@DougGregor DougGregor requested a review from huonw February 8, 2018 00:54
@swift-ci
Copy link
Contributor

swift-ci commented Feb 8, 2018

Build comment file:

Summary for master full

Unexpected test results, stats may be off for FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-tvOS.log, FAIL_RxDataSources-Pods-Pods.xcodeproj_3.0_BuildXcodeProjectTarget_Pods-RxDataSources_generic-platform-iOS.log, FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-iOS.log, FAIL_JSQDataSourcesKit-JSQDataSourcesKit.xcodeproj_4.0_BuildXcodeProjectTarget_JSQDataSourcesKit-iOS_generic-platform-iOS.log, FAIL_Dollar-Dollar.xcodeproj_3.0_BuildXcodeProjectScheme_Dollar_generic-platform-macOS.log, 3, FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-macOS.log, FAIL_RxDataSources-Pods-Pods.xcodeproj_3.0_BuildXcodeProjectTarget_Pods-Example_generic-platform-iOS.log

Regressions found (see below)

Debug

debug brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (2)
name old new delta delta_pct
LLVM.NumLLVMBytesOutput 964,360,326 964,746,786 386,460 0.04%
time.swift-driver.wall 1578.6s 1592.5s 13.9s 0.88%

debug detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (23)
name old new delta delta_pct
AST.NumImportedExternalDefinitions 1,538,452 1,540,098 1,646 0.11%
AST.NumLoadedModules 305,170 305,379 209 0.07%
AST.NumTotalClangImportedEntities 4,759,934 4,764,418 4,484 0.09%
AST.NumUsedConformances 137,809 137,984 175 0.13%
IRModule.NumIRBasicBlocks 3,003,458 3,005,364 1,906 0.06%
IRModule.NumIRFunctions 1,440,484 1,441,116 632 0.04%
IRModule.NumIRGlobals 1,483,406 1,483,835 429 0.03%
IRModule.NumIRInsts 30,731,706 30,748,861 17,155 0.06%
IRModule.NumIRValueSymbols 2,449,526 2,450,433 907 0.04%
LLVM.NumLLVMBytesOutput 964,360,326 964,746,786 386,460 0.04%
SILModule.NumSILGenFunctions 963,559 964,023 464 0.05%
SILModule.NumSILOptFunctions 1,359,198 1,359,864 666 0.05%
Sema.NumConformancesDeserialized 5,434,471 5,439,648 5,177 0.1%
Sema.NumConstraintScopes 10,937,480 10,945,503 8,023 0.07%
Sema.NumDeclsDeserialized 44,806,513 44,847,461 40,948 0.09%
Sema.NumDeclsValidated 1,927,312 1,929,593 2,281 0.12%
Sema.NumFunctionsTypechecked 927,650 928,686 1,036 0.11%
Sema.NumGenericSignatureBuilders 1,482,381 1,483,879 1,498 0.1%
Sema.NumLazyGenericEnvironments 8,746,083 8,755,098 9,015 0.1%
Sema.NumLazyGenericEnvironmentsLoaded 811,215 812,068 853 0.11%
Sema.NumLazyIterableDeclContexts 6,966,681 6,973,101 6,420 0.09%
Sema.NumTypesDeserialized 46,728,706 46,773,244 44,538 0.1%
Sema.NumTypesValidated 4,467,579 4,471,794 4,215 0.09%

Debug-opt

debug-opt brief

Regressed (0)
name old new delta delta_pct
Improved (1)
name old new delta delta_pct
time.swift-driver.wall 2996.4s 2955.2s -41.2s -1.37% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (1)
name old new delta delta_pct
LLVM.NumLLVMBytesOutput 933,671,929 933,640,617 -31,312 -0.0%

debug-opt detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (23)
name old new delta delta_pct
AST.NumImportedExternalDefinitions 1,513,143 1,512,781 -362 -0.02%
AST.NumLoadedModules 289,931 289,856 -75 -0.03%
AST.NumTotalClangImportedEntities 4,859,265 4,858,235 -1,030 -0.02%
AST.NumUsedConformances 136,267 136,264 -3 -0.0%
IRModule.NumIRBasicBlocks 3,301,953 3,302,353 400 0.01%
IRModule.NumIRFunctions 1,127,811 1,127,898 87 0.01%
IRModule.NumIRGlobals 1,226,949 1,226,785 -164 -0.01%
IRModule.NumIRInsts 25,331,345 25,334,202 2,857 0.01%
IRModule.NumIRValueSymbols 2,025,089 2,025,020 -69 -0.0%
LLVM.NumLLVMBytesOutput 933,671,929 933,640,617 -31,312 -0.0%
SILModule.NumSILGenFunctions 946,081 945,916 -165 -0.02%
SILModule.NumSILOptFunctions 1,874,901 1,874,805 -96 -0.01%
Sema.NumConformancesDeserialized 11,648,291 11,647,096 -1,195 -0.01%
Sema.NumConstraintScopes 10,851,872 10,851,595 -277 -0.0%
Sema.NumDeclsDeserialized 50,255,955 50,253,370 -2,585 -0.01%
Sema.NumDeclsValidated 1,905,539 1,905,484 -55 -0.0%
Sema.NumFunctionsTypechecked 912,077 911,876 -201 -0.02%
Sema.NumGenericSignatureBuilders 1,510,461 1,510,293 -168 -0.01%
Sema.NumLazyGenericEnvironments 9,684,816 9,684,303 -513 -0.01%
Sema.NumLazyGenericEnvironmentsLoaded 823,253 823,187 -66 -0.01%
Sema.NumLazyIterableDeclContexts 7,241,560 7,240,936 -624 -0.01%
Sema.NumTypesDeserialized 54,226,891 54,223,809 -3,082 -0.01%
Sema.NumTypesValidated 4,418,629 4,418,473 -156 -0.0%

Wmo-onone

wmo-onone brief

None

wmo-onone detailed

None

Release

release brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (2)
name old new delta delta_pct
LLVM.NumLLVMBytesOutput 927,864,014 927,873,870 9,856 0.0%
time.swift-driver.wall 2723.6s 2733.2s 9.6s 0.35%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (23)
name old new delta delta_pct
AST.NumImportedExternalDefinitions 469,581 469,674 93 0.02%
AST.NumLoadedModules 57,869 57,879 10 0.02%
AST.NumTotalClangImportedEntities 1,497,149 1,497,532 383 0.03%
AST.NumUsedConformances 145,076 145,140 64 0.04%
IRModule.NumIRBasicBlocks 2,795,880 2,796,722 842 0.03%
IRModule.NumIRFunctions 1,099,036 1,099,554 518 0.05%
IRModule.NumIRGlobals 1,269,855 1,270,094 239 0.02%
IRModule.NumIRInsts 23,443,536 23,446,499 2,963 0.01%
IRModule.NumIRValueSymbols 2,027,725 2,028,386 661 0.03%
LLVM.NumLLVMBytesOutput 927,864,014 927,873,870 9,856 0.0%
SILModule.NumSILGenFunctions 532,217 532,339 122 0.02%
SILModule.NumSILOptFunctions 960,379 960,865 486 0.05%
Sema.NumConformancesDeserialized 4,192,267 4,195,433 3,166 0.08%
Sema.NumConstraintScopes 10,624,564 10,626,957 2,393 0.02%
Sema.NumDeclsDeserialized 13,402,939 13,407,729 4,790 0.04%
Sema.NumDeclsValidated 1,096,475 1,096,591 116 0.01%
Sema.NumFunctionsTypechecked 415,571 415,635 64 0.02%
Sema.NumGenericSignatureBuilders 467,300 467,385 85 0.02%
Sema.NumLazyGenericEnvironments 2,513,041 2,513,823 782 0.03%
Sema.NumLazyGenericEnvironmentsLoaded 237,510 237,585 75 0.03%
Sema.NumLazyIterableDeclContexts 1,839,400 1,839,836 436 0.02%
Sema.NumTypesDeserialized 14,893,367 14,899,070 5,703 0.04%
Sema.NumTypesValidated 2,042,287 2,042,350 63 0.0%

@DougGregor DougGregor merged commit ce83c8b into swiftlang:master Feb 8, 2018
@DougGregor DougGregor deleted the gsb-term-rewriting branch February 8, 2018 17:38
Copy link
Contributor

@huonw huonw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's remarkably... neat and simple! Very cool.

I do notice a lack of tests though; is it not quite working fully?

/// types.
Optional<RewritePath> static createPath(Type type);

/// Decompose a potential archetype into a patch.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(s/patch/path/)

Copy link
Member Author

@DougGregor DougGregor Feb 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks. A later PR (#14685) is going to delete this code (yay!)

/// with concrete declarations.
Optional<RewritePath> static createPath(PotentialArchetype *pa);

/// Compute the common path between this path and \c other, if one exists.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Common path being the longest prefix? What's the conditions under which one doesn't exist? It looks like it's the bases being different from the code. Why's that case different to a path being empty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's the longest common prefix. The failure case is when the bases are different... but you're right that I can express this as a completely-empty path.


/// Retrieve the base of the given rewrite path.
///
/// When present, it indicates that the entire path will be rebased on
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what's it mean when absent? A floating path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, "floating" or "relative".

void enumerateRewritePaths(
RelativeRewritePath matchPath,
llvm::function_ref<void(unsigned, RewritePath)> callback,
unsigned depth = 0) const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do callers need to do anything with depth?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I'm being lazy; I'll split it out.

SmallVector<AssociatedTypeDecl *, 4> path(initialPath->getPath().begin(),
initialPath->getPath().end());
bool simplified = false;
do {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking my understanding: this goes along a path like A.B.C.D looking for rewrites first for B.C.D within {A}, then for C.D within {A.B}, etc., and if one succeeds (say, {A.B}.C.D -> X.Y.Z) it starts again? And, the loop terminates because the compareDependentPaths/swap calls in addSameTypeRewriteRule ensure that the target of each rule is smaller in some sense?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct.

// Merge the second rewrite tree into the first.
rewriteRoot2->mergeInto(rewriteRoot1);
Impl->RewriteTreeRoots.erase(equivClass2);
delete rewriteRoot2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious: since there's at least two of these manual clean-ups, unique_ptr or similar mustn't work with DenseMap (and how it's being used); why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of doing something clever with re-using root nodes, but I don't think it's worthwhile. I'll switch it over to using std::unique_ptr for the roots.

@DougGregor
Copy link
Member Author

@huonw, regarding your testing comment, now that we're computing all of the anchors of equivalence classes via this code path (and cough getting it right modulo the issues resolved by #14685), we're getting quite a lot of coverage for these code paths. I could maybe do some FileCheck-based testing to see that we're getting the expected rewrite trees for particular generic signatures, I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants