ClangImporter: Build type-checked AST for constants #25009

slavapestov · 2019-05-23T05:35:37Z

Follow-up to #24797. Now everything on ExternalDeclarations is fully type checked, paving the way for removing that list altogether.

slavapestov · 2019-05-23T05:35:47Z

@swift-ci Please test

slavapestov · 2019-05-23T05:36:01Z

@swift-ci Please test source compatibility

slavapestov · 2019-05-23T05:36:07Z

@swift-ci Please test compiler performance

swift-ci · 2019-05-23T07:03:24Z

Build failed
Swift Test OS X Platform
Git Sha - fc30048c858599e390e148d2c8405134fcf9e674

swift-ci · 2019-05-23T13:51:24Z

Summary for master full

Unexpected test results, excluded stats for ProcedureKit, Tagged, Wordy, Deferred

Regressions found (see below)

Debug-batch
- debug-batch brief
- debug-batch detailed
Release
- release brief
- release detailed

Debug-batch

debug-batch brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (2)

name	old	new	delta	delta_pct
Frontend.NumInstructionsExecuted	25,287,964,280,096	24,686,539,964,496	-601,424,315,600	-2.38% ✅
time.swift-driver.wall	1779.0s	1754.4s	-24.6s	-1.38% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (1)

name	old	new	delta	delta_pct
LLVM.NumLLVMBytesOutput	1,169,273,858	1,169,274,752	894	0.0%

debug-batch detailed

Regressed (1)

name	old	new	delta	delta_pct
AST.NumSourceLinesPerSecond	3,282,901	3,391,652	108,751	3.31% ⛔

Improved (28)

name	old	new	delta	delta_pct
AST.NumASTBytesAllocated	55,533,197,003	54,842,187,279	-691,009,724	-1.24% ✅
AST.NumImportedExternalDefinitions	900,558	883,811	-16,747	-1.86% ✅
AST.NumReferencedTopLevelNames	323,183	319,466	-3,717	-1.15% ✅
Driver.NumDriverPipePolls	226,708	208,225	-18,483	-8.15% ✅
Driver.NumDriverPipeReads	231,829	210,954	-20,875	-9.0% ✅
Frontend.MaxMallocUsage	525,968,554,632	519,941,499,152	-6,027,055,480	-1.15% ✅
Frontend.NumInstructionsExecuted	25,287,964,280,096	24,686,539,964,496	-601,424,315,600	-2.38% ✅
Sema.AttachedPropertyDelegateRequest	1,305,655	1,285,816	-19,839	-1.52% ✅
Sema.DefaultTypeRequest	338,166	327,989	-10,177	-3.01% ✅
Sema.ExtendedNominalRequest	2,791,428	2,659,747	-131,681	-4.72% ✅
Sema.InheritedDeclsReferencedRequest	3,185,834	3,108,592	-77,242	-2.42% ✅
Sema.IsDynamicRequest	1,869,180	1,785,742	-83,438	-4.46% ✅
Sema.IsFinalRequest	3,598,658	3,515,834	-82,824	-2.3% ✅
Sema.IsObjCRequest	1,627,925	1,593,017	-34,908	-2.14% ✅
Sema.NominalTypeLookupDirectCount	31,397,737	24,530,000	-6,867,737	-21.87% ✅
Sema.NumConstraintScopes	19,101,196	17,896,537	-1,204,659	-6.31% ✅
Sema.NumConstraintsConsideredForEdgeContraction	51,626,565	51,100,364	-526,201	-1.02% ✅
Sema.NumDeclsDeserialized	40,307,033	39,364,786	-942,247	-2.34% ✅
Sema.NumDeclsFinalized	1,678,508	1,577,081	-101,427	-6.04% ✅
Sema.NumFunctionsTypechecked	770,717	376,057	-394,660	-51.21% ✅
Sema.NumGenericSignatureBuilders	914,345	839,968	-74,377	-8.13% ✅
Sema.NumLazyGenericEnvironments	8,070,724	7,795,066	-275,658	-3.42% ✅
Sema.NumLazyIterableDeclContexts	5,006,488	4,870,151	-136,337	-2.72% ✅
Sema.NumLeafScopes	12,558,723	11,617,153	-941,570	-7.5% ✅
Sema.NumTypesDeserialized	13,065,766	12,714,668	-351,098	-2.69% ✅
Sema.NumUnloadedLazyIterableDeclContexts	3,046,094	3,003,078	-43,016	-1.41% ✅
Sema.OverriddenDeclsRequest	6,628,146	6,493,786	-134,360	-2.03% ✅
Sema.SelfBoundsFromWhereClauseRequest	4,232,998	4,138,265	-94,733	-2.24% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (80)

name	old	new	delta	delta_pct
AST.NumDecls	94,750	94,750	0	0.0%
AST.NumDependencies	133,989	133,993	4	0.0%
AST.NumInfixOperators	35,705	35,705	0	0.0%
AST.NumLinkLibraries	0	0	0	0.0%
AST.NumLoadedModules	168,435	168,435	0	0.0%
AST.NumLocalTypeDecls	123	123	0	0.0%
AST.NumObjCMethods	15,429	15,429	0	0.0%
AST.NumPostfixOperators	18	18	0	0.0%
AST.NumPrecedenceGroups	17,488	17,488	0	0.0%
AST.NumPrefixOperators	90	90	0	0.0%
AST.NumReferencedDynamicNames	122	122	0	0.0%
AST.NumReferencedMemberNames	4,149,622	4,149,622	0	0.0%
AST.NumSourceBuffers	216,901	216,901	0	0.0%
AST.NumSourceLines	3,165,267	3,165,267	0	0.0%
AST.NumStoredPropertiesQueries	9,359,928	9,342,806	-17,122	-0.18%
AST.NumTotalClangImportedEntities	3,486,836	3,466,275	-20,561	-0.59%
Driver.ChildrenMaxRSS	111,480,569,856	111,339,251,712	-141,318,144	-0.13%
Driver.DriverDepCascadingDynamic	0	0	0	0.0%
Driver.DriverDepCascadingExternal	0	0	0	0.0%
Driver.DriverDepCascadingMember	0	0	0	0.0%
Driver.DriverDepCascadingNominal	0	0	0	0.0%
Driver.DriverDepCascadingTopLevel	0	0	0	0.0%
Driver.DriverDepDynamic	0	0	0	0.0%
Driver.DriverDepExternal	0	0	0	0.0%
Driver.DriverDepMember	0	0	0	0.0%
Driver.DriverDepNominal	0	0	0	0.0%
Driver.DriverDepTopLevel	0	0	0	0.0%
Driver.NumDriverJobsRun	19,330	19,330	0	0.0%
Driver.NumDriverJobsSkipped	0	0	0	0.0%
Driver.NumProcessFailures	0	0	0	0.0%
Frontend.NumProcessFailures	0	0	0	0.0%
IRModule.NumIRAliases	121,583	121,583	0	0.0%
IRModule.NumIRBasicBlocks	4,681,557	4,681,557	0	0.0%
IRModule.NumIRComdatSymbols	0	0	0	0.0%
IRModule.NumIRFunctions	2,132,801	2,132,801	0	0.0%
IRModule.NumIRGlobals	2,238,405	2,238,405	0	0.0%
IRModule.NumIRIFuncs	0	0	0	0.0%
IRModule.NumIRInsts	61,539,984	61,539,984	0	0.0%
IRModule.NumIRNamedMetaData	93,930	93,930	0	0.0%
IRModule.NumIRValueSymbols	3,901,671	3,901,671	0	0.0%
LLVM.NumLLVMBytesOutput	1,169,273,858	1,169,274,752	894	0.0%
Parse.NumFunctionsParsed	174,293	174,293	0	0.0%
Parse.NumIterableDeclContextParsed	635,386	635,386	0	0.0%
SILModule.NumSILGenDefaultWitnessTables	0	0	0	0.0%
SILModule.NumSILGenFunctions	1,100,041	1,100,041	0	0.0%
SILModule.NumSILGenGlobalVariables	41,604	41,604	0	0.0%
SILModule.NumSILGenVtables	12,415	12,415	0	0.0%
SILModule.NumSILGenWitnessTables	46,421	46,421	0	0.0%
SILModule.NumSILOptDefaultWitnessTables	0	0	0	0.0%
SILModule.NumSILOptFunctions	1,565,175	1,565,175	0	0.0%
SILModule.NumSILOptGlobalVariables	42,499	42,499	0	0.0%
SILModule.NumSILOptVtables	20,589	20,589	0	0.0%
SILModule.NumSILOptWitnessTables	101,981	101,981	0	0.0%
Sema.AccessLevelRequest	2,438,317	2,437,327	-990	-0.04%
Sema.AttachedPropertyDelegateTypeRequest	241,021	241,021	0	0.0%
Sema.CustomAttrNominalRequest	0	0	0	0.0%
Sema.DefaultAndMaxAccessLevelRequest	52,735	52,690	-45	-0.09%
Sema.DefaultDefinitionTypeRequest	5,374	5,370	-4	-0.07%
Sema.EnumRawTypeRequest	14,416	14,416	0	0.0%
Sema.InheritedTypeRequest	204,812	204,821	9	0.0%
Sema.MangleLocalTypeDeclRequest	246	246	0	0.0%
Sema.NamedLazyMemberLoadFailureCount	18,780	18,772	-8	-0.04%
Sema.NamedLazyMemberLoadSuccessCount	18,732,561	18,797,869	65,308	0.35%
Sema.NumConformancesDeserialized	5,489,115	5,474,716	-14,399	-0.26%
Sema.NumDeclsTypechecked	1,034,315	1,034,315	0	0.0%
Sema.NumDeclsValidated	2,077,935	2,077,825	-110	-0.01%
Sema.NumLazyGenericEnvironmentsLoaded	208,048	208,056	8	0.0%
Sema.NumTypesValidated	1,553,318	1,553,266	-52	-0.0%
Sema.PropertyDelegateBackingPropertyInfoRequest	238,686	238,686	0	0.0%
Sema.PropertyDelegateBackingPropertyTypeRequest	241,021	241,021	0	0.0%
Sema.PropertyDelegateTypeInfoRequest	0	0	0	0.0%
Sema.RequirementRequest	64,105	64,105	0	0.0%
Sema.RequirementSignatureRequest	52,134	52,124	-10	-0.02%
Sema.SetterAccessLevelRequest	136,560	136,560	0	0.0%
Sema.StructuralTypeRequest	0	0	0	0.0%
Sema.SuperclassDeclRequest	67,525	67,539	14	0.02%
Sema.SuperclassTypeRequest	33,726	33,726	0	0.0%
Sema.TypeDeclsFromWhereClauseRequest	28,712	28,667	-45	-0.16%
Sema.USRGenerationRequest	10,678,346	10,620,325	-58,021	-0.54%
Sema.UnderlyingTypeDeclsReferencedRequest	136,660	136,561	-99	-0.07%

Release

release brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (3)

name	old	new	delta	delta_pct
Frontend.NumInstructionsExecuted	30,858,908,774,914	30,745,046,246,718	-113,862,528,196	-0.37%
LLVM.NumLLVMBytesOutput	994,185,122	994,185,046	-76	-0.0%
time.swift-driver.wall	3339.2s	3319.3s	-19.9s	-0.6%

release detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (5)

name	old	new	delta	delta_pct
Sema.NumConstraintScopes	17,773,296	17,413,106	-360,190	-2.03% ✅
Sema.NumDeclsDeserialized	6,310,377	6,242,644	-67,733	-1.07% ✅
Sema.NumFunctionsTypechecked	365,156	257,289	-107,867	-29.54% ✅
Sema.NumGenericSignatureBuilders	200,859	190,200	-10,659	-5.31% ✅
Sema.NumLazyGenericEnvironments	1,299,874	1,276,475	-23,399	-1.8% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (17)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	233,079	231,868	-1,211	-0.52%
AST.NumLoadedModules	16,875	16,875	0	0.0%
AST.NumTotalClangImportedEntities	801,464	800,052	-1,412	-0.18%
IRModule.NumIRBasicBlocks	3,971,710	3,971,710	0	0.0%
IRModule.NumIRFunctions	1,790,638	1,790,638	0	0.0%
IRModule.NumIRGlobals	1,971,030	1,971,030	0	0.0%
IRModule.NumIRInsts	36,342,853	36,342,853	0	0.0%
IRModule.NumIRValueSymbols	3,493,213	3,493,213	0	0.0%
LLVM.NumLLVMBytesOutput	994,185,122	994,185,046	-76	-0.0%
SILModule.NumSILGenFunctions	762,557	762,557	0	0.0%
SILModule.NumSILOptFunctions	1,019,892	1,019,892	0	0.0%
Sema.NumConformancesDeserialized	2,329,936	2,330,053	117	0.01%
Sema.NumDeclsValidated	1,212,685	1,212,685	0	0.0%
Sema.NumLazyGenericEnvironmentsLoaded	22,496	22,495	-1	-0.0%
Sema.NumLazyIterableDeclContexts	778,279	770,491	-7,788	-1.0%
Sema.NumTypesDeserialized	3,312,803	3,291,375	-21,428	-0.65%
Sema.NumTypesValidated	751,070	751,070	0	0.0%

These will be used shortly to avoid name lookups into the imported type.

This allows Sema to skip checking external definitions altogether.

slavapestov · 2019-05-23T14:43:28Z

@swift-ci Please test source compatibility

slavapestov · 2019-05-23T14:43:39Z

@swift-ci Please test

swift-ci · 2019-05-23T14:45:46Z

Build failed
Swift Test Linux Platform
Git Sha - fc30048c858599e390e148d2c8405134fcf9e674

swift-ci · 2019-05-23T14:45:47Z

Build failed
Swift Test OS X Platform
Git Sha - fc30048c858599e390e148d2c8405134fcf9e674

slavapestov · 2019-05-23T14:47:18Z

@swift-ci Please test source compatibility

jrose-apple · 2019-05-23T15:55:47Z

What's the motivation here? Just to speed up build times? It seems unfortunate that we're increasing the duplication of information across Sema and ClangImporter.

Are there higher-level operations that can be factored out that'd we'd also use for synthesized code in Sema?

slavapestov · 2019-05-23T17:24:05Z

@jrose-apple I wrote up a blurb here: #24665

Basically we want to be able to synthesize declarations at any time, instead of everything happening before the magic point where Sema walks an 'external declarations' list.

Doing this for accessors was a requirement for lazily checking conformances from SILGen instead of relying on Sema 'guessing' and populating a UsedConformances list. I did it for literals for completeness for performance and because I want to remove ExternalDeclarations.

I agree there's a bit more duplication but we can factor it out, and also in this specific case of literal constants its a lot easier now that we have LiteralExpr that is lowered directly by SILGen, rather than Sema using callWitness() to build calls to the literal protocols.

jrose-apple · 2019-05-23T17:29:51Z

I see the point, but this is definitely harder to maintain, and more likely to become incorrect if we change how the AST works again. That's why we moved away from type-checked ASTs in the first place.

…an alias This comes up when an imported error enum has duplicate cases. The FooError.Code enum has an alias for the duplicate case, and the wrapper type FooError defines aliases for every member of FooError.Code. The implementation of importEnumCaseAlias() assumed that 'original' was an EnumElementDecl, and built AST accordingly; however if it is a 'VarDecl', we have to build a MemberRefExpr instead. This regression was introduced in swiftlang#25009. Fixes <rdar://problem/58552618>.

slavapestov added 7 commits May 23, 2019 10:40

AST: Add utilities to ASTContext for looking up builtin initializers

b26ba2d

Sema: Build fully type-checked AST for stub initializers

140a5b0

Sema: Build fully type-checked AST for dynamic member lookups

2cfc459

ClangImporter: Remove ConstantConvertKind::{Coerce,Downcast}

95843e3

ClangImporter: Track raw types of imported struct and enum wrappers

7a630f4

These will be used shortly to avoid name lookups into the imported type.

ClangImporter: Build type-checked AST for constants

57254cb

ClangImporter: Set the 'throws' bit on a few CallExprs

8075b09

This allows Sema to skip checking external definitions altogether.

slavapestov force-pushed the type-checked-constants branch from fc30048 to 8075b09 Compare May 23, 2019 14:41

slavapestov merged commit 3d4541d into swiftlang:master May 23, 2019

slavapestov mentioned this pull request May 24, 2019

Remove used conformances list #25044

Merged

slavapestov mentioned this pull request Feb 6, 2020

ClangImporter: Fix importEnumCaseAlias() when the original is itself an alias #29679

Merged

slavapestov mentioned this pull request Feb 6, 2020

ClangImporter: Fix importEnumCaseAlias() when the original is itself an alias [5.2] #29688

Merged

ClangImporter: Build type-checked AST for constants #25009

ClangImporter: Build type-checked AST for constants #25009

Uh oh!

Conversation

slavapestov commented May 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

swift-ci commented May 23, 2019

Uh oh!

swift-ci commented May 23, 2019

Summary for master full

Debug-batch

debug-batch brief

debug-batch detailed

Release

release brief

release detailed

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

swift-ci commented May 23, 2019

Uh oh!

swift-ci commented May 23, 2019

Uh oh!

slavapestov commented May 23, 2019

Uh oh!

jrose-apple commented May 23, 2019

Uh oh!

slavapestov commented May 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jrose-apple commented May 23, 2019

Uh oh!

Uh oh!

slavapestov commented May 23, 2019 •

edited

Loading

slavapestov commented May 23, 2019 •

edited

Loading