[Constraint solver] Favor more-specialized overload among two generics. #14499

DougGregor · 2018-02-09T00:59:23Z

When we form an overload set containing two generic functions, where
one is more specialized than the other, "favor" the more-specialized
function in the constraint system so we won't explore any paths
involving the less-specialized function when the more-specialized
version applies. This should be a strict improvement, because
previously we would have always gone down both paths.

Fixes rdar://problem/37371815, taking this exponential example linear.

DougGregor · 2018-02-09T00:59:32Z

@swift-ci please test and merge

DougGregor · 2018-02-09T01:18:50Z

And... it looks like this breaks when I put it on master:

    Swift(macosx-x86_64) :: Constraints/optional.swift
    Swift(macosx-x86_64) :: Sema/type_checker_perf/slow/rdar23620262.swift
    Swift(macosx-x86_64) :: Compatibility/enum_cases.swift
    Swift(macosx-x86_64) :: Constraints/enum_cases.swift

The "enum cases" examples are changes in diagnostics that are mostly good; the "slow" test is now fast (which is also good); but the "optional" change is potentially a regression.

xedin · 2018-02-09T01:23:21Z

lib/Sema/ConstraintSystem.cpp

@@ -1391,6 +1391,29 @@ void ConstraintSystem::addOverloadSet(Type boundType,
    return;
  }

+  // Performance hack: if there are two generic overloads, and one is
+  // more specialized than the other, prefer the more-specialized one.
+  if (!favoredChoice && choices.size() == 2 &&


The idea is great! I'm trying to implement similar thing as well but instead of hard-coding this to 2 overloads only, maybe it would be better to try and group overload choices based on declaration context and rank them in groups so at the end there are N favored choices in the set?... Another idea might be to move from favoring to making it a ranking criteria since we erase worse intermediate solutions...

You're absolutely right that we could generalize this. In general, if an one overload is more specialized than another, we should try it first. We could probably encode this information in an tree of disjunctions using favored constraints... but it would probably be better in the long run to compute the lattice describing the relationships and have the solver take an appropriate path through the lattice.

As for moving it to ranking criteria, that won't prune as efficiently, because we often still explore those other options. I'd rather

When we form an overload set containing two generic functions, where one is more specialized than the other, "favor" the more-specialized function in the constraint system so we won't explore any paths involving the less-specialized function when the more-specialized version applies. This should be a strict improvement, because previously we would have always gone down both paths. Fixes rdar://problem/37371815, taking this exponential example linear.

DougGregor · 2018-02-09T05:02:04Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T05:02:25Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T06:32:24Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T06:32:31Z

@swift-ci please smoke test

DougGregor · 2018-02-09T06:32:34Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T06:32:44Z

@swift-ci please smoke test compiler performance

DougGregor · 2018-02-09T06:32:55Z

@swift-ci please smoke test compiler performance

DougGregor · 2018-02-09T06:33:05Z

@swift-ci please smoke test

swift-ci · 2018-02-09T06:33:16Z

Build comment file:

Compilation-performance test failed

DougGregor · 2018-02-09T06:33:57Z

@swift-ci please smoke test compiler performance

swift-ci · 2018-02-09T06:34:20Z

Build comment file:

Compilation-performance test failed

DougGregor · 2018-02-09T06:57:30Z

@swift-ci please clean test source compatibility

DougGregor · 2018-02-09T06:58:19Z

@swift-ci please clean test source compatibility

…tlang#14497)" This reverts commit 43435af.

… serialize syntax trees. (swiftlang#14424)" (swiftlang#14465)" This reverts commit f8c77e1.

DougGregor · 2018-02-09T06:59:08Z

@swift-ci please smoke test

DougGregor · 2018-02-09T06:59:20Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T06:59:27Z

@swift-ci please smoke test

DougGregor · 2018-02-09T06:59:32Z

@swift-ci please test source compatibility

DougGregor · 2018-02-09T06:59:41Z

@swift-ci please smoke test compiler performance

DougGregor · 2018-02-09T06:59:48Z

@swift-ci please smoke test compiler performance

swift-ci · 2018-02-09T07:00:13Z

Build comment file:

Compilation-performance test failed

DougGregor · 2018-02-09T07:01:01Z

@swift-ci please smoke test compiler performance

swift-ci · 2018-02-09T07:01:22Z

Build comment file:

Compilation-performance test failed

DougGregor · 2018-02-09T07:05:30Z

@swift-ci please test compiler performance

DougGregor · 2018-02-09T07:05:39Z

@swift-ci please test compiler performance

DougGregor · 2018-02-09T07:05:49Z

@swift-ci please test compiler performance

DougGregor · 2018-02-09T07:06:00Z

@swift-ci please test compiler performance

swift-ci · 2018-02-09T23:56:54Z

Build comment file:

Summary for master full

Unexpected test results, stats may be off for FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-tvOS.log, FAIL_RxDataSources-Pods-Pods.xcodeproj_3.0_BuildXcodeProjectTarget_Pods-RxDataSources_generic-platform-iOS.log, FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-iOS.log, FAIL_JSQDataSourcesKit-JSQDataSourcesKit.xcodeproj_4.0_BuildXcodeProjectTarget_JSQDataSourcesKit-iOS_generic-platform-iOS.log, FAIL_Dollar-Dollar.xcodeproj_3.0_BuildXcodeProjectScheme_Dollar_generic-platform-macOS.log, 3, FAIL_Kronos-Kronos.xcodeproj_3.0_BuildXcodeProjectTarget_Kronos_generic-platform-macOS.log, FAIL_RxDataSources-Pods-Pods.xcodeproj_3.0_BuildXcodeProjectTarget_Pods-Example_generic-platform-iOS.log

No regressions above thresholds

Debug
- debug brief
- debug detailed
Debug-opt
- debug-opt brief
- debug-opt detailed
Wmo-onone
- wmo-onone brief
- wmo-onone detailed
Release
- release brief
- release detailed

Debug

debug brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
time.swift-driver.wall	1922.0s	1833.0s	-89.1s	-4.63% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (1)

name	old	new	delta	delta_pct
LLVM.NumLLVMBytesOutput	965,114,389	964,813,193	-301,196	-0.03%

debug detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
Sema.NumConstraintScopes	10,959,558	10,397,710	-561,848	-5.13% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (22)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	1,543,692	1,543,123	-569	-0.04%
AST.NumLoadedModules	305,807	305,741	-66	-0.02%
AST.NumTotalClangImportedEntities	4,774,403	4,772,821	-1,582	-0.03%
AST.NumUsedConformances	138,403	138,356	-47	-0.03%
IRModule.NumIRBasicBlocks	2,994,655	2,993,575	-1,080	-0.04%
IRModule.NumIRFunctions	1,442,422	1,441,943	-479	-0.03%
IRModule.NumIRGlobals	1,484,731	1,484,103	-628	-0.04%
IRModule.NumIRInsts	30,739,296	30,729,559	-9,737	-0.03%
IRModule.NumIRValueSymbols	2,452,432	2,451,520	-912	-0.04%
LLVM.NumLLVMBytesOutput	965,114,389	964,813,193	-301,196	-0.03%
SILModule.NumSILGenFunctions	964,793	964,146	-647	-0.07%
SILModule.NumSILOptFunctions	1,361,235	1,360,745	-490	-0.04%
Sema.NumConformancesDeserialized	5,452,678	5,450,192	-2,486	-0.05%
Sema.NumDeclsDeserialized	44,926,717	44,914,295	-12,422	-0.03%
Sema.NumDeclsValidated	1,936,109	1,935,731	-378	-0.02%
Sema.NumFunctionsTypechecked	931,021	930,698	-323	-0.03%
Sema.NumGenericSignatureBuilders	1,487,247	1,486,731	-516	-0.03%
Sema.NumLazyGenericEnvironments	8,774,822	8,772,685	-2,137	-0.02%
Sema.NumLazyGenericEnvironmentsLoaded	814,120	813,161	-959	-0.12%
Sema.NumLazyIterableDeclContexts	6,987,466	6,985,653	-1,813	-0.03%
Sema.NumTypesDeserialized	46,861,787	46,848,171	-13,616	-0.03%
Sema.NumTypesValidated	4,481,119	4,477,453	-3,666	-0.08%

Debug-opt

debug-opt brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
time.swift-driver.wall	3213.1s	3083.0s	-130.1s	-4.05% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (1)

name	old	new	delta	delta_pct
LLVM.NumLLVMBytesOutput	941,137,878	941,139,982	2,104	0.0%

debug-opt detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
Sema.NumConstraintScopes	10,960,151	10,401,863	-558,288	-5.09% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (22)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	1,543,785	1,543,934	149	0.01%
AST.NumLoadedModules	294,094	294,105	11	0.0%
AST.NumTotalClangImportedEntities	4,941,910	4,942,402	492	0.01%
AST.NumUsedConformances	138,392	138,395	3	0.0%
IRModule.NumIRBasicBlocks	3,322,785	3,322,647	-138	-0.0%
IRModule.NumIRFunctions	1,136,325	1,136,375	50	0.0%
IRModule.NumIRGlobals	1,236,822	1,236,883	61	0.0%
IRModule.NumIRInsts	25,462,164	25,461,302	-862	-0.0%
IRModule.NumIRValueSymbols	2,041,660	2,041,758	98	0.0%
LLVM.NumLLVMBytesOutput	941,137,878	941,139,982	2,104	0.0%
SILModule.NumSILGenFunctions	964,169	964,283	114	0.01%
SILModule.NumSILOptFunctions	1,883,910	1,883,933	23	0.0%
Sema.NumConformancesDeserialized	11,741,360	11,741,240	-120	-0.0%
Sema.NumDeclsDeserialized	50,881,084	50,882,115	1,031	0.0%
Sema.NumDeclsValidated	1,936,217	1,936,056	-161	-0.01%
Sema.NumFunctionsTypechecked	931,086	931,171	85	0.01%
Sema.NumGenericSignatureBuilders	1,537,319	1,537,343	24	0.0%
Sema.NumLazyGenericEnvironments	9,811,337	9,811,594	257	0.0%
Sema.NumLazyGenericEnvironmentsLoaded	835,884	835,192	-692	-0.08%
Sema.NumLazyIterableDeclContexts	7,350,312	7,350,606	294	0.0%
Sema.NumTypesDeserialized	54,893,437	54,894,534	1,097	0.0%
Sema.NumTypesValidated	4,481,483	4,478,663	-2,820	-0.06%

Wmo-onone

wmo-onone brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (2)

name	old	new	delta	delta_pct
LLVM.NumLLVMBytesOutput	843,900,046	844,107,676	207,630	0.02%
time.swift-driver.wall	1386.0s	1373.4s	-12.6s	-0.91%

wmo-onone detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
Sema.NumConstraintScopes	10,325,100	9,770,067	-555,033	-5.38% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (22)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	180,144	180,330	186	0.1%
AST.NumLoadedModules	10,315	10,335	20	0.19%
AST.NumTotalClangImportedEntities	569,020	569,784	764	0.13%
AST.NumUsedConformances	141,817	141,944	127	0.09%
IRModule.NumIRBasicBlocks	2,362,472	2,364,068	1,596	0.07%
IRModule.NumIRFunctions	1,196,950	1,197,725	775	0.06%
IRModule.NumIRGlobals	1,179,892	1,180,546	654	0.06%
IRModule.NumIRInsts	26,839,548	26,852,894	13,346	0.05%
IRModule.NumIRValueSymbols	2,004,516	2,005,735	1,219	0.06%
LLVM.NumLLVMBytesOutput	843,900,046	844,107,676	207,630	0.02%
SILModule.NumSILGenFunctions	494,860	495,104	244	0.05%
SILModule.NumSILOptFunctions	538,163	538,542	379	0.07%
Sema.NumConformancesDeserialized	1,225,976	1,228,395	2,419	0.2%
Sema.NumDeclsDeserialized	4,227,311	4,234,948	7,637	0.18%
Sema.NumDeclsValidated	876,601	876,657	56	0.01%
Sema.NumFunctionsTypechecked	279,817	279,945	128	0.05%
Sema.NumGenericSignatureBuilders	149,939	150,089	150	0.1%
Sema.NumLazyGenericEnvironments	722,941	724,349	1,408	0.19%
Sema.NumLazyGenericEnvironmentsLoaded	87,944	88,024	80	0.09%
Sema.NumLazyIterableDeclContexts	460,340	461,133	793	0.17%
Sema.NumTypesDeserialized	4,315,715	4,323,517	7,802	0.18%
Sema.NumTypesValidated	1,054,982	1,052,160	-2,822	-0.27%

Release

release brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (2)

name	old	new	delta	delta_pct
LLVM.NumLLVMBytesOutput	927,583,926	927,555,792	-28,134	-0.0%
time.swift-driver.wall	3044.0s	3016.5s	-27.6s	-0.91%

release detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (1)

name	old	new	delta	delta_pct
Sema.NumConstraintScopes	10,619,408	10,050,276	-569,132	-5.36% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (22)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	469,488	469,417	-71	-0.02%
AST.NumLoadedModules	57,859	57,853	-6	-0.01%
AST.NumTotalClangImportedEntities	1,496,767	1,496,572	-195	-0.01%
AST.NumUsedConformances	145,013	144,788	-225	-0.16%
IRModule.NumIRBasicBlocks	2,794,560	2,794,366	-194	-0.01%
IRModule.NumIRFunctions	1,098,458	1,098,482	24	0.0%
IRModule.NumIRGlobals	1,269,549	1,269,569	20	0.0%
IRModule.NumIRInsts	23,425,118	23,423,671	-1,447	-0.01%
IRModule.NumIRValueSymbols	2,026,937	2,026,975	38	0.0%
LLVM.NumLLVMBytesOutput	927,583,926	927,555,792	-28,134	-0.0%
SILModule.NumSILGenFunctions	532,094	532,094	0	0.0%
SILModule.NumSILOptFunctions	959,778	959,772	-6	-0.0%
Sema.NumConformancesDeserialized	4,189,456	4,188,610	-846	-0.02%
Sema.NumDeclsDeserialized	13,398,357	13,395,393	-2,964	-0.02%
Sema.NumDeclsValidated	1,096,601	1,094,845	-1,756	-0.16%
Sema.NumFunctionsTypechecked	415,507	415,035	-472	-0.11%
Sema.NumGenericSignatureBuilders	467,217	467,051	-166	-0.04%
Sema.NumLazyGenericEnvironments	2,511,914	2,511,358	-556	-0.02%
Sema.NumLazyGenericEnvironmentsLoaded	237,435	237,138	-297	-0.13%
Sema.NumLazyIterableDeclContexts	1,838,937	1,838,581	-356	-0.02%
Sema.NumTypesDeserialized	14,886,733	14,883,960	-2,773	-0.02%
Sema.NumTypesValidated	2,042,130	2,037,828	-4,302	-0.21%

DougGregor requested review from xedin and rudkx February 9, 2018 00:59

xedin reviewed Feb 9, 2018

View reviewed changes

DougGregor force-pushed the solver-favor-more-specialized branch from 48f8501 to a7239c7 Compare February 9, 2018 06:32

Update test cases for constraint solver performance hack

dee02c4

DougGregor force-pushed the solver-favor-more-specialized branch from a7239c7 to dee02c4 Compare February 9, 2018 06:57

DougGregor added 2 commits February 8, 2018 22:58

Revert "[sourcekitd] Rename module SwiftSourceKit to SwiftLang. (swif…

b060667

…tlang#14497)" This reverts commit 43435af.

Revert "Re-apply "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to…

774bee2

… serialize syntax trees. (swiftlang#14424)" (swiftlang#14465)" This reverts commit f8c77e1.

DougGregor merged commit ae89cf0 into swiftlang:master Feb 9, 2018

DougGregor deleted the solver-favor-more-specialized branch February 9, 2018 16:29

DougGregor mentioned this pull request Feb 9, 2018

Re-apply "SwiftSyntax: Teach SwiftSyntax to use SourceKitd to serialize syntax trees. (#14424)" #14465

Merged

DougGregor mentioned this pull request Feb 10, 2018

[4.1] [Constraint solver] Favor more-specialized overload among two generics. #14507

Closed

rudkx mentioned this pull request Jun 6, 2018

[SR-7884] Overload resolution picks optional variant even when value is not optional #50419

Closed

[Constraint solver] Favor more-specialized overload among two generics. #14499

[Constraint solver] Favor more-specialized overload among two generics. #14499

Uh oh!

Conversation

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

xedin Feb 9, 2018

Choose a reason for hiding this comment

Uh oh!

DougGregor Feb 9, 2018

Choose a reason for hiding this comment

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

swift-ci commented Feb 9, 2018

Build comment file:

Compilation-performance test failed

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

swift-ci commented Feb 9, 2018

Build comment file:

Compilation-performance test failed

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

swift-ci commented Feb 9, 2018

Build comment file:

Compilation-performance test failed

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

swift-ci commented Feb 9, 2018

Build comment file:

Compilation-performance test failed

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

DougGregor commented Feb 9, 2018

Uh oh!

swift-ci commented Feb 9, 2018

Build comment file:

Summary for master full

Debug