Skip to content

[Experiment] Redo Globals As Member Tables #29007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[Experiment] Redo Globals As Member Tables #29007

wants to merge 1 commit into from

Conversation

CodaFi
Copy link
Contributor

@CodaFi CodaFi commented Jan 4, 2020

⚠️ DO NOT MERGE ⚠️

Globals-as-members has an antagonistic relationship with lazy member
loading. The old PCM extension filed them away indexed by context, which
was super efficient for the general case of "just import this entire
extension", but ridiculously inefficient for the specific case of "just
load this one member for me".

Instead, store the table in named order. Switch over all the callers
and unblock lazy member loading.

This naturally changes the order things are printed in, but does not
change their contents.

@CodaFi
[Experiment] Redo Globals As Member Tables
5837529 
Globals-as-members has an antagonistic relationship with lazy member
loading. The old PCM extension filed them away indexed by context, which
was super efficient for the general case of "just import this entire
extension", but ridiculously inefficient for the specific case of "just
load this one member for me".

Instead, store the table in named order.  Switch over all the callers
and unblock lazy member loading.

This naturally changes the order things are printed in, but does not
change their contents.
@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 4, 2020

@swift-ci test

@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 4, 2020

@swift-ci test source compatibility

@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 4, 2020

@swift-ci test compiler performance

@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 4, 2020

Note that this implementation probably needs one more cache in front of it that maps stored contexts to names, but let's see how this one fares first.

@swift-ci
Copy link
Contributor

swift-ci commented Jan 5, 2020

Summary for master full

Unexpected test results, excluded stats for RxCocoa, Base64CoderSwiftUI, SwifterSwift

Regressions found (see below)

Debug-batch

debug-batch brief

Regressed (2)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 50,817,528,181,672 54,199,829,707,229 3,382,301,525,557 6.66% ⛔
time.swift-driver.wall 5519.6s 5664.5s 144.9s 2.62% ⛔
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (1)
name old new delta delta_pct
LLVM.NumLLVMBytesOutput 1,800,251,500 1,800,250,820 -680 -0.0%

debug-batch detailed

Regressed (9)
name old new delta delta_pct
Driver.ChildrenMaxRSS 242,510,247,936 247,259,068,416 4,748,820,480 1.96% ⛔
Driver.NumDriverPipePolls 78,079 78,960 881 1.13% ⛔
Driver.NumDriverPipeReads 65,544 66,241 697 1.06% ⛔
Frontend.MaxMallocUsage 1,233,825,751,576 1,307,831,944,048 74,006,192,472 6.0% ⛔
Frontend.NumInstructionsExecuted 50,817,528,181,672 54,199,829,707,229 3,382,301,525,557 6.66% ⛔
Sema.AccessLevelRequest 12,502,723 12,702,147 199,424 1.6% ⛔
Sema.CollectOverriddenDeclsRequest 7,455,066 7,652,191 197,125 2.64% ⛔
Sema.ProvideDefaultImplForRequest 7,455,066 7,652,191 197,125 2.64% ⛔
Sema.USRGenerationRequest 8,857,277 9,062,193 204,916 2.31% ⛔
Improved (5)
name old new delta delta_pct
AST.ModuleShadowCacheHit 1,533,713 1,491,298 -42,415 -2.77% ✅
AST.NumSourceLinesPerSecond 3,258,396 3,163,128 -95,268 -2.92% ✅
AST.NumTotalClangImportedEntities 6,203,614 6,126,771 -76,843 -1.24% ✅
Sema.IsStaticRequest 3,730,249 3,668,720 -61,529 -1.65% ✅
Sema.NamedLazyMemberLoadFailureCount 17,606 15,242 -2,364 -13.43% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (193)
name old new delta delta_pct
AST.ImportSetCacheHit 1,809,707 1,810,819 1,112 0.06%
AST.ImportSetCacheMiss 1,119,145 1,119,441 296 0.03%
AST.ImportSetFoldHit 650,145 650,144 -1 -0.0%
AST.ImportSetFoldMiss 469,000 469,297 297 0.06%
AST.ModuleShadowCacheMiss 13,572 13,575 3 0.02%
AST.ModuleVisibilityCacheHit 33,149 33,149 0 0.0%
AST.ModuleVisibilityCacheMiss 8,458 8,458 0 0.0%
AST.NumASTBytesAllocated 64,704,108,606 65,026,795,517 322,686,911 0.5%
AST.NumASTScopeLookups 4,289,995 4,290,205 210 0.0%
AST.NumBraceStmtASTScopeExpansions 624,906 624,906 0 0.0%
AST.NumBraceStmtASTScopes 624,906 624,906 0 0.0%
AST.NumDecls 139,853 139,853 0 0.0%
AST.NumDependencies 429,062 429,017 -45 -0.01%
AST.NumInfixOperators 53,695 53,695 0 0.0%
AST.NumIterableTypeBodyASTScopeExpansions 271,584 271,590 6 0.0%
AST.NumIterableTypeBodyASTScopes 355,184 355,258 74 0.02%
AST.NumLinkLibraries 0 0 0 0.0%
AST.NumLoadedModules 404,163 404,163 0 0.0%
AST.NumLocalTypeDecls 250 250 0 0.0%
AST.NumModuleLookupClassMember 6,927 6,927 0 0.0%
AST.NumModuleLookupValue 48,544,001 48,582,117 38,116 0.08%
AST.NumObjCMethods 24,240 24,240 0 0.0%
AST.NumPostfixOperators 49 49 0 0.0%
AST.NumPrecedenceGroups 25,840 25,840 0 0.0%
AST.NumPrefixOperators 99 99 0 0.0%
AST.NumReferencedDynamicNames 189 189 0 0.0%
AST.NumReferencedMemberNames 6,309,732 6,309,732 0 0.0%
AST.NumReferencedTopLevelNames 476,243 476,243 0 0.0%
AST.NumSourceBuffers 602,301 602,301 0 0.0%
AST.NumSourceLines 4,717,684 4,717,684 0 0.0%
Driver.DriverDepCascadingDynamic 0 0 0 0.0%
Driver.DriverDepCascadingExternal 0 0 0 0.0%
Driver.DriverDepCascadingMember 0 0 0 0.0%
Driver.DriverDepCascadingNominal 0 0 0 0.0%
Driver.DriverDepCascadingTopLevel 0 0 0 0.0%
Driver.DriverDepDynamic 0 0 0 0.0%
Driver.DriverDepExternal 0 0 0 0.0%
Driver.DriverDepMember 0 0 0 0.0%
Driver.DriverDepNominal 0 0 0 0.0%
Driver.DriverDepTopLevel 0 0 0 0.0%
Driver.NumDriverJobsRun 28,030 28,030 0 0.0%
Driver.NumDriverJobsSkipped 0 0 0 0.0%
Driver.NumProcessFailures 0 0 0 0.0%
Frontend.NumProcessFailures 0 0 0 0.0%
IRModule.NumIRAliases 196,319 196,319 0 0.0%
IRModule.NumIRBasicBlocks 6,863,469 6,863,469 0 0.0%
IRModule.NumIRComdatSymbols 0 0 0 0.0%
IRModule.NumIRFunctions 3,362,194 3,362,194 0 0.0%
IRModule.NumIRGlobals 3,672,622 3,672,622 0 0.0%
IRModule.NumIRIFuncs 0 0 0 0.0%
IRModule.NumIRInsts 87,971,190 87,971,190 0 0.0%
IRModule.NumIRNamedMetaData 135,380 135,380 0 0.0%
IRModule.NumIRValueSymbols 6,379,699 6,379,699 0 0.0%
LLVM.NumLLVMBytesOutput 1,800,251,500 1,800,250,820 -680 -0.0%
Parse.NumFunctionsParsed 268,976 268,976 0 0.0%
Parse.NumIterableDeclContextParsed 939,424 939,432 8 0.0%
Parse.ParseAbstractFunctionBodyRequest 246,370 246,370 0 0.0%
Parse.ParseMembersRequest 770,800 770,808 8 0.0%
SILModule.NumSILGenDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILGenFunctions 1,701,394 1,701,394 0 0.0%
SILModule.NumSILGenGlobalVariables 53,554 53,554 0 0.0%
SILModule.NumSILGenVtables 18,627 18,627 0 0.0%
SILModule.NumSILGenWitnessTables 72,395 72,395 0 0.0%
SILModule.NumSILOptDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILOptFunctions 2,522,862 2,522,862 0 0.0%
SILModule.NumSILOptGlobalVariables 55,187 55,187 0 0.0%
SILModule.NumSILOptVtables 31,428 31,428 0 0.0%
SILModule.NumSILOptWitnessTables 169,304 169,304 0 0.0%
Sema.AbstractGenericSignatureRequest 34,854 34,854 0 0.0%
Sema.AnyObjectLookupRequest 281 281 0 0.0%
Sema.AreAllStoredPropertiesDefaultInitableRequest 23,241 23,241 0 0.0%
Sema.AttachedFunctionBuilderRequest 3 3 0 0.0%
Sema.AttachedPropertyWrapperTypeRequest 540,381 540,381 0 0.0%
Sema.AttachedPropertyWrappersRequest 2,374,807 2,374,805 -2 -0.0%
Sema.CallerSideDefaultArgExprRequest 78,637 78,637 0 0.0%
Sema.ClassAncestryFlagsRequest 111,574 111,574 0 0.0%
Sema.CompareDeclSpecializationRequest 472,080 474,442 2,362 0.5%
Sema.CursorInfoRequest 0 0 0 0.0%
Sema.CustomAttrNominalRequest 3 3 0 0.0%
Sema.DefaultAndMaxAccessLevelRequest 58,492 58,489 -3 -0.01%
Sema.DefaultArgumentExprRequest 38,177 38,177 0 0.0%
Sema.DefaultArgumentInitContextRequest 454 454 0 0.0%
Sema.DefaultDefinitionTypeRequest 8,214 8,214 0 0.0%
Sema.DefaultTypeRequest 455,957 455,957 0 0.0%
Sema.DynamicallyReplacedDeclRequest 1,069,474 1,069,474 0 0.0%
Sema.EmittedMembersRequest 27,960 27,960 0 0.0%
Sema.EnumRawTypeRequest 25,725 25,725 0 0.0%
Sema.EnumRawValuesRequest 10,587 10,587 0 0.0%
Sema.ExistentialConformsToSelfRequest 14,503 14,517 14 0.1%
Sema.ExistentialTypeSupportedRequest 17,897 17,897 0 0.0%
Sema.ExpandASTScopeRequest 7,675,213 7,675,321 108 0.0%
Sema.ExtendedNominalRequest 916,704 916,704 0 0.0%
Sema.ExtendedTypeRequest 83,496 83,500 4 0.0%
Sema.FunctionBuilderTypeRequest 3 3 0 0.0%
Sema.FunctionOperatorRequest 73,361 73,361 0 0.0%
Sema.GenericParamListRequest 12,907,858 13,036,758 128,900 1.0%
Sema.GenericSignatureRequest 5,659,621 5,665,074 5,453 0.1%
Sema.GetDestructorRequest 28,229 28,229 0 0.0%
Sema.HasCircularInheritanceRequest 23,920 23,920 0 0.0%
Sema.HasCircularInheritedProtocolsRequest 10,224 10,224 0 0.0%
Sema.HasCircularRawValueRequest 8,958 8,958 0 0.0%
Sema.HasDefaultInitRequest 56,884 56,884 0 0.0%
Sema.HasDynamicMemberLookupAttributeRequest 711,854 712,475 621 0.09%
Sema.HasMemberwiseInitRequest 20,394 20,394 0 0.0%
Sema.HasUserDefinedDesignatedInitRequest 56,894 56,894 0 0.0%
Sema.InferredGenericSignatureRequest 189,619 189,630 11 0.01%
Sema.InheritedDeclsReferencedRequest 6,957,775 7,001,221 43,446 0.62%
Sema.InheritedTypeRequest 314,838 314,993 155 0.05%
Sema.InheritsSuperclassInitializersRequest 30,032 30,005 -27 -0.09%
Sema.InitKindRequest 97,988 97,988 0 0.0%
Sema.InterfaceTypeRequest 17,077,541 17,223,647 146,106 0.86%
Sema.IsABICompatibleOverrideRequest 136,210 136,210 0 0.0%
Sema.IsAccessorTransparentRequest 319,462 319,462 0 0.0%
Sema.IsCallableNominalTypeRequest 2,089 2,089 0 0.0%
Sema.IsDeclApplicableRequest 0 0 0 0.0%
Sema.IsDynamicRequest 1,753,080 1,753,080 0 0.0%
Sema.IsFinalRequest 2,681,885 2,701,018 19,133 0.71%
Sema.IsGetterMutatingRequest 434,466 434,466 0 0.0%
Sema.IsImplicitlyUnwrappedOptionalRequest 2,526,525 2,527,063 538 0.02%
Sema.IsObjCRequest 1,599,928 1,601,019 1,091 0.07%
Sema.IsSetterMutatingRequest 342,898 342,898 0 0.0%
Sema.LazyStoragePropertyRequest 2,516 2,516 0 0.0%
Sema.LookupInModuleRequest 6,823,550 6,850,031 26,481 0.39%
Sema.LookupPrecedenceGroupRequest 175,955 175,955 0 0.0%
Sema.MangleLocalTypeDeclRequest 500 500 0 0.0%
Sema.ModuleQualifiedLookupRequest 3,009,899 3,036,179 26,280 0.87%
Sema.NamedLazyMemberLoadSuccessCount 12,875,927 12,877,173 1,246 0.01%
Sema.NamingPatternRequest 207,682 207,680 -2 -0.0%
Sema.NeedsNewVTableEntryRequest 696,060 696,060 0 0.0%
Sema.NominalTypeLookupDirectCount 34,702,976 34,710,964 7,988 0.02%
Sema.NumAccessorBodiesSynthesized 189,152 189,152 0 0.0%
Sema.NumAccessorsSynthesized 295,089 295,089 0 0.0%
Sema.NumConformancesDeserialized 9,770,720 9,855,323 84,603 0.87%
Sema.NumConstraintScopes 27,689,204 27,699,614 10,410 0.04%
Sema.NumConstraintsConsideredForEdgeContraction 92,820,186 92,824,199 4,013 0.0%
Sema.NumCyclicOneWayComponentsCollapsed 0 0 0 0.0%
Sema.NumDeclsDeserialized 77,728,096 78,390,795 662,699 0.85%
Sema.NumDeclsTypechecked 1,412,011 1,412,011 0 0.0%
Sema.NumFunctionsTypechecked 529,260 529,260 0 0.0%
Sema.NumGenericSignatureBuilders 1,468,195 1,471,900 3,705 0.25%
Sema.NumLazyIterableDeclContexts 9,841,213 9,868,066 26,853 0.27%
Sema.NumLazyRequirementSignatures 1,044,421 1,044,627 206 0.02%
Sema.NumLazyRequirementSignaturesLoaded 753,880 754,166 286 0.04%
Sema.NumLeafScopes 17,648,725 17,657,412 8,687 0.05%
Sema.NumTypesDeserialized 23,818,263 23,914,880 96,617 0.41%
Sema.NumTypesValidated 1,526,781 1,526,783 2 0.0%
Sema.NumUnloadedLazyIterableDeclContexts 6,292,133 6,290,574 -1,559 -0.02%
Sema.OpaqueReadOwnershipRequest 282,056 282,056 0 0.0%
Sema.OpaqueResultTypeRequest 0 0 0 0.0%
Sema.OperatorPrecedenceGroupRequest 747 747 0 0.0%
Sema.OverriddenDeclsRequest 3,057,524 3,064,493 6,969 0.23%
Sema.ParamSpecifierRequest 1,309,277 1,309,281 4 0.0%
Sema.PatternBindingEntryRequest 464,936 464,934 -2 -0.0%
Sema.PatternTypeRequest 586,251 586,249 -2 -0.0%
Sema.PreCheckFunctionBuilderRequest 0 0 0 0.0%
Sema.PropertyWrapperBackingPropertyInfoRequest 535,004 535,004 0 0.0%
Sema.PropertyWrapperBackingPropertyTypeRequest 540,381 540,381 0 0.0%
Sema.PropertyWrapperMutabilityRequest 630,613 630,613 0 0.0%
Sema.PropertyWrapperTypeInfoRequest 1 1 0 0.0%
Sema.ProtocolRequiresClassRequest 76,383 76,198 -185 -0.24%
Sema.QualifiedLookupRequest 7,377,517 7,311,603 -65,914 -0.89%
Sema.RangeInfoRequest 0 0 0 0.0%
Sema.RequirementRequest 113,859 113,864 5 0.0%
Sema.RequirementSignatureRequest 833,564 834,131 567 0.07%
Sema.RequiresOpaqueAccessorsRequest 1,314,548 1,314,548 0 0.0%
Sema.RequiresOpaqueModifyCoroutineRequest 273,195 273,195 0 0.0%
Sema.ResilienceExpansionRequest 1,865,551 1,865,563 12 0.0%
Sema.ResolveImplicitMemberRequest 373,779 374,192 413 0.11%
Sema.ResolveProtocolNameRequest 0 0 0 0.0%
Sema.ResultTypeRequest 695,555 695,560 5 0.0%
Sema.RootAndResultTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.RootTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.SelfAccessKindRequest 3,177,737 3,180,873 3,136 0.1%
Sema.SelfBoundsFromWhereClauseRequest 1,816,049 1,830,435 14,386 0.79%
Sema.SetterAccessLevelRequest 145,111 145,111 0 0.0%
Sema.StorageImplInfoRequest 1,423,621 1,423,621 0 0.0%
Sema.StoredPropertiesAndMissingMembersRequest 32,428 32,428 0 0.0%
Sema.StoredPropertiesRequest 369,964 369,964 0 0.0%
Sema.StructuralTypeRequest 2,318 2,318 0 0.0%
Sema.SuperclassDeclRequest 501,657 502,283 626 0.12%
Sema.SuperclassTypeRequest 60,087 60,087 0 0.0%
Sema.SynthesizeAccessorRequest 295,089 295,089 0 0.0%
Sema.SynthesizeDefaultInitRequest 5,394 5,394 0 0.0%
Sema.SynthesizeMemberwiseInitRequest 3,199 3,199 0 0.0%
Sema.TypeCheckFunctionBodyUntilRequest 529,260 529,260 0 0.0%
Sema.TypeCheckSourceFileRequest 27,076 27,076 0 0.0%
Sema.TypeDeclsFromWhereClauseRequest 30,494 30,491 -3 -0.01%
Sema.TypeRelationCheckRequest 0 0 0 0.0%
Sema.TypeWitnessRequest 9,277 9,280 3 0.03%
Sema.UnderlyingTypeDeclsReferencedRequest 382,852 382,858 6 0.0%
Sema.UnderlyingTypeRequest 38,660 38,660 0 0.0%
Sema.UnqualifiedLookupRequest 4,434,814 4,435,024 210 0.0%
Sema.ValueWitnessRequest 68,677 68,677 0 0.0%

Release

release brief

Regressed (2)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 51,104,271,265,928 51,816,384,494,484 712,113,228,556 1.39% ⛔
time.swift-driver.wall 9488.2s 9617.0s 128.8s 1.36% ⛔
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (1)
name old new delta delta_pct
LLVM.NumLLVMBytesOutput 1,924,890,852 1,924,893,644 2,792 0.0%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (1)
name old new delta delta_pct
AST.NumTotalClangImportedEntities 1,182,860 1,126,674 -56,186 -4.75% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (17)
name old new delta delta_pct
AST.NumLoadedModules 30,053 30,053 0 0.0%
IRModule.NumIRBasicBlocks 5,449,988 5,449,988 0 0.0%
IRModule.NumIRFunctions 2,823,936 2,823,936 0 0.0%
IRModule.NumIRGlobals 3,183,499 3,183,499 0 0.0%
IRModule.NumIRInsts 53,499,760 53,499,760 0 0.0%
IRModule.NumIRValueSymbols 5,650,030 5,650,030 0 0.0%
LLVM.NumLLVMBytesOutput 1,924,890,852 1,924,893,644 2,792 0.0%
SILModule.NumSILGenFunctions 1,189,997 1,189,997 0 0.0%
SILModule.NumSILOptFunctions 1,733,256 1,733,256 0 0.0%
Sema.NumConformancesDeserialized 3,646,439 3,646,399 -40 -0.0%
Sema.NumConstraintScopes 27,313,772 27,313,772 0 0.0%
Sema.NumDeclsDeserialized 10,695,152 10,693,288 -1,864 -0.02%
Sema.NumFunctionsTypechecked 532,882 532,882 0 0.0%
Sema.NumGenericSignatureBuilders 281,445 281,445 0 0.0%
Sema.NumLazyIterableDeclContexts 1,318,580 1,317,414 -1,166 -0.09%
Sema.NumTypesDeserialized 5,410,862 5,410,184 -678 -0.01%
Sema.NumTypesValidated 826,777 826,777 0 0.0%

@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 5, 2020

Hm, these regressions are unacceptable. Time to add the secondary cache.

@CodaFi CodaFi mentioned this pull request Jan 7, 2020
Closed
@CodaFi
Copy link
Contributor Author

CodaFi commented Jan 7, 2020

Closing this out. It was a bad idea...

@CodaFi CodaFi closed this Jan 7, 2020
@CodaFi CodaFi deleted the globals-as-members-redo branch January 21, 2020 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@CodaFi @swift-ci