Skip to content

Refactor Direct Lookup #28845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 20, 2019
Merged

Refactor Direct Lookup #28845

merged 1 commit into from
Dec 20, 2019

Conversation

CodaFi
Copy link
Contributor

@CodaFi CodaFi commented Dec 18, 2019

The old name lookup would frequently try to flush and rebuild the name lookup cache. Instead, never flush the cache, and use the cache misses as an opportunity to load members and bring the lookup table up to date with any added extensions.

Whether this is an improvement over the status quo remains to be seen.

This is built on #28840 for a little extra fun... Only the last commit really matters.

@CodaFi CodaFi requested a review from slavapestov December 18, 2019 04:55
@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci test

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci test source Compatibility

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci test compiler performance

/// scope; it should be manipulated through \c isLookupTablePopulated()
/// and \c setLookupTablePopulated().
/// scope; it should be manipulated through \c isLookupTableDirty()
/// and \c setLookupTableDirty().
llvm::PointerIntPair<MemberLookupTable *, 1, bool> LookupTable;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: If this works out, make this a unique_ptr.

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 32364a3350fd5c23db433f8e8c5860b0e4cc8321

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

Oh boy, cascading infrastructure failures.

@swift-ci test

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci test source compatibility

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci test compiler performance

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 32364a3350fd5c23db433f8e8c5860b0e4cc8321

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 32364a3350fd5c23db433f8e8c5860b0e4cc8321

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 18, 2019

@swift-ci please smoke test

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 19, 2019

@swift-ci please smoke test

1 similar comment
@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 19, 2019

@swift-ci please smoke test

@CodaFi CodaFi changed the title Experiment: Refactor Direct Lookup Refactor Direct Lookup Dec 19, 2019
The old name lookup would frequently try to flush and rebuild the name lookup cache.  Instead, never flush the cache, and use the cache misses as an opportunity to load members and bring the lookup table up to date with any added extensions.
@swiftlang swiftlang deleted a comment from swift-ci Dec 19, 2019
@swiftlang swiftlang deleted a comment from swift-ci Dec 19, 2019
@swiftlang swiftlang deleted a comment from swift-ci Dec 19, 2019
@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 19, 2019

@swift-ci please test

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 19, 2019

@swift-ci please test source compatibility

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 19, 2019

@swift-ci please test compiler performance

@swift-ci
Copy link
Contributor

Summary for master full

Unexpected test results, excluded stats for RxCocoa, SwifterSwift, Base64CoderSwiftUI

Regressions found (see below)

Debug-batch

debug-batch brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 50,254,411,033,679 50,017,873,067,966 -236,537,965,713 -0.47%
LLVM.NumLLVMBytesOutput 1,799,867,596 1,799,860,342 -7,254 -0.0%
time.swift-driver.wall 5434.4s 5425.0s -9.4s -0.17%

debug-batch detailed

Regressed (2)
name old new delta delta_pct
Driver.NumDriverPipePolls 77,932 78,929 997 1.28% ⛔
Driver.NumDriverPipeReads 65,756 66,830 1,074 1.63% ⛔
Improved (6)
name old new delta delta_pct
Sema.AccessLevelRequest 12,288,410 12,147,238 -141,172 -1.15% ✅
Sema.CollectOverriddenDeclsRequest 7,311,313 7,195,499 -115,814 -1.58% ✅
Sema.NamedLazyMemberLoadFailureCount 17,789 17,249 -540 -3.04% ✅
Sema.NamedLazyMemberLoadSuccessCount 31,735,145 31,303,464 -431,681 -1.36% ✅
Sema.ProvideDefaultImplForRequest 7,311,313 7,195,499 -115,814 -1.58% ✅
Sema.USRGenerationRequest 8,702,096 8,583,502 -118,594 -1.36% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (198)
name old new delta delta_pct
AST.ImportSetCacheHit 1,796,565 1,796,073 -492 -0.03%
AST.ImportSetCacheMiss 1,089,979 1,089,911 -68 -0.01%
AST.ImportSetFoldHit 630,512 630,524 12 0.0%
AST.ImportSetFoldMiss 459,467 459,387 -80 -0.02%
AST.ModuleShadowCacheHit 1,517,335 1,507,516 -9,819 -0.65%
AST.ModuleShadowCacheMiss 13,331 13,330 -1 -0.01%
AST.ModuleVisibilityCacheHit 33,128 33,128 0 0.0%
AST.ModuleVisibilityCacheMiss 8,399 8,399 0 0.0%
AST.NumASTBytesAllocated 63,776,676,958 63,420,868,408 -355,808,550 -0.56%
AST.NumASTScopeLookups 4,245,985 4,245,809 -176 -0.0%
AST.NumBraceStmtASTScopeExpansions 624,966 624,966 0 0.0%
AST.NumBraceStmtASTScopes 624,966 624,966 0 0.0%
AST.NumDecls 139,809 139,809 0 0.0%
AST.NumDependencies 420,748 420,804 56 0.01%
AST.NumInfixOperators 53,667 53,667 0 0.0%
AST.NumIterableTypeBodyASTScopeExpansions 269,870 269,858 -12 -0.0%
AST.NumIterableTypeBodyASTScopes 352,095 352,016 -79 -0.02%
AST.NumLinkLibraries 0 0 0 0.0%
AST.NumLoadedModules 396,362 396,362 0 0.0%
AST.NumLocalTypeDecls 250 250 0 0.0%
AST.NumModuleLookupClassMember 6,927 6,927 0 0.0%
AST.NumModuleLookupValue 48,138,520 48,122,209 -16,311 -0.03%
AST.NumObjCMethods 24,240 24,240 0 0.0%
AST.NumPostfixOperators 49 49 0 0.0%
AST.NumPrecedenceGroups 25,836 25,836 0 0.0%
AST.NumPrefixOperators 99 99 0 0.0%
AST.NumReferencedDynamicNames 189 189 0 0.0%
AST.NumReferencedMemberNames 6,308,148 6,308,148 0 0.0%
AST.NumReferencedTopLevelNames 476,000 476,000 0 0.0%
AST.NumSourceBuffers 582,005 582,005 0 0.0%
AST.NumSourceLines 4,716,580 4,716,580 0 0.0%
AST.NumSourceLinesPerSecond 3,336,444 3,362,024 25,580 0.77%
AST.NumTotalClangImportedEntities 6,149,059 6,109,957 -39,102 -0.64%
Driver.ChildrenMaxRSS 242,868,109,312 242,728,331,264 -139,778,048 -0.06%
Driver.DriverDepCascadingDynamic 0 0 0 0.0%
Driver.DriverDepCascadingExternal 0 0 0 0.0%
Driver.DriverDepCascadingMember 0 0 0 0.0%
Driver.DriverDepCascadingNominal 0 0 0 0.0%
Driver.DriverDepCascadingTopLevel 0 0 0 0.0%
Driver.DriverDepDynamic 0 0 0 0.0%
Driver.DriverDepExternal 0 0 0 0.0%
Driver.DriverDepMember 0 0 0 0.0%
Driver.DriverDepNominal 0 0 0 0.0%
Driver.DriverDepTopLevel 0 0 0 0.0%
Driver.NumDriverJobsRun 28,026 28,026 0 0.0%
Driver.NumDriverJobsSkipped 0 0 0 0.0%
Driver.NumProcessFailures 0 0 0 0.0%
Frontend.MaxMallocUsage 1,217,799,705,768 1,214,242,319,616 -3,557,386,152 -0.29%
Frontend.NumInstructionsExecuted 50,254,411,033,679 50,017,873,067,966 -236,537,965,713 -0.47%
Frontend.NumProcessFailures 0 0 0 0.0%
IRModule.NumIRAliases 196,255 196,255 0 0.0%
IRModule.NumIRBasicBlocks 6,861,539 6,861,539 0 0.0%
IRModule.NumIRComdatSymbols 0 0 0 0.0%
IRModule.NumIRFunctions 3,361,071 3,361,071 0 0.0%
IRModule.NumIRGlobals 3,671,597 3,671,597 0 0.0%
IRModule.NumIRIFuncs 0 0 0 0.0%
IRModule.NumIRInsts 87,954,827 87,954,827 0 0.0%
IRModule.NumIRNamedMetaData 135,360 135,360 0 0.0%
IRModule.NumIRValueSymbols 6,377,583 6,377,583 0 0.0%
LLVM.NumLLVMBytesOutput 1,799,867,596 1,799,860,342 -7,254 -0.0%
Parse.NumFunctionsParsed 268,908 268,908 0 0.0%
Parse.NumIterableDeclContextParsed 922,436 922,411 -25 -0.0%
Parse.ParseAbstractFunctionBodyRequest 246,318 246,318 0 0.0%
Parse.ParseMembersRequest 756,509 756,484 -25 -0.0%
SILModule.NumSILGenDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILGenFunctions 1,710,885 1,710,885 0 0.0%
SILModule.NumSILGenGlobalVariables 53,546 53,546 0 0.0%
SILModule.NumSILGenVtables 18,627 18,627 0 0.0%
SILModule.NumSILGenWitnessTables 72,371 72,371 0 0.0%
SILModule.NumSILOptDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILOptFunctions 2,462,444 2,462,444 0 0.0%
SILModule.NumSILOptGlobalVariables 55,175 55,175 0 0.0%
SILModule.NumSILOptVtables 31,424 31,424 0 0.0%
SILModule.NumSILOptWitnessTables 158,535 158,535 0 0.0%
Sema.AbstractGenericSignatureRequest 34,614 34,610 -4 -0.01%
Sema.AnyObjectLookupRequest 281 281 0 0.0%
Sema.AreAllStoredPropertiesDefaultInitableRequest 23,226 23,226 0 0.0%
Sema.AttachedFunctionBuilderRequest 3 3 0 0.0%
Sema.AttachedPropertyWrapperTypeRequest 539,524 539,524 0 0.0%
Sema.AttachedPropertyWrappersRequest 2,209,166 2,209,167 1 0.0%
Sema.CallerSideDefaultArgExprRequest 78,549 78,549 0 0.0%
Sema.ClassAncestryFlagsRequest 109,163 109,163 0 0.0%
Sema.CompareDeclSpecializationRequest 469,800 468,617 -1,183 -0.25%
Sema.CursorInfoRequest 0 0 0 0.0%
Sema.CustomAttrNominalRequest 3 3 0 0.0%
Sema.DefaultAndMaxAccessLevelRequest 58,407 58,408 1 0.0%
Sema.DefaultArgumentExprRequest 38,177 38,177 0 0.0%
Sema.DefaultArgumentInitContextRequest 454 454 0 0.0%
Sema.DefaultDefinitionTypeRequest 8,168 8,168 0 0.0%
Sema.DefaultTypeRequest 455,892 455,892 0 0.0%
Sema.DynamicallyReplacedDeclRequest 1,076,530 1,076,530 0 0.0%
Sema.EmittedMembersRequest 27,969 27,969 0 0.0%
Sema.EnumRawTypeRequest 25,550 25,550 0 0.0%
Sema.EnumRawValuesRequest 10,581 10,581 0 0.0%
Sema.ExistentialConformsToSelfRequest 14,331 14,308 -23 -0.16%
Sema.ExistentialTypeSupportedRequest 17,717 17,717 0 0.0%
Sema.ExpandASTScopeRequest 7,624,666 7,624,501 -165 -0.0%
Sema.ExtendedNominalRequest 887,370 887,370 0 0.0%
Sema.ExtendedTypeRequest 83,370 83,366 -4 -0.0%
Sema.FunctionBuilderTypeRequest 3 3 0 0.0%
Sema.FunctionOperatorRequest 72,937 72,937 0 0.0%
Sema.GenericParamListRequest 12,713,367 12,620,590 -92,777 -0.73%
Sema.GenericSignatureRequest 5,602,394 5,594,033 -8,361 -0.15%
Sema.GetDestructorRequest 28,370 28,370 0 0.0%
Sema.HasCircularInheritanceRequest 23,846 23,846 0 0.0%
Sema.HasCircularInheritedProtocolsRequest 10,228 10,228 0 0.0%
Sema.HasCircularRawValueRequest 8,958 8,958 0 0.0%
Sema.HasDefaultInitRequest 56,819 56,819 0 0.0%
Sema.HasDynamicMemberLookupAttributeRequest 707,860 707,468 -392 -0.06%
Sema.HasMemberwiseInitRequest 20,343 20,343 0 0.0%
Sema.HasUserDefinedDesignatedInitRequest 56,829 56,829 0 0.0%
Sema.InferredGenericSignatureRequest 188,877 188,855 -22 -0.01%
Sema.InheritedDeclsReferencedRequest 6,883,996 6,855,683 -28,313 -0.41%
Sema.InheritedTypeRequest 313,216 313,118 -98 -0.03%
Sema.InheritsSuperclassInitializersRequest 29,949 29,929 -20 -0.07%
Sema.InitKindRequest 97,995 97,995 0 0.0%
Sema.InterfaceTypeRequest 16,869,539 16,773,014 -96,525 -0.57%
Sema.IsABICompatibleOverrideRequest 136,155 136,155 0 0.0%
Sema.IsAccessorTransparentRequest 319,258 319,258 0 0.0%
Sema.IsCallableNominalTypeRequest 2,089 2,089 0 0.0%
Sema.IsDeclApplicableRequest 0 0 0 0.0%
Sema.IsDynamicRequest 1,759,445 1,759,445 0 0.0%
Sema.IsFinalRequest 2,666,967 2,654,020 -12,947 -0.49%
Sema.IsGetterMutatingRequest 434,110 434,110 0 0.0%
Sema.IsImplicitlyUnwrappedOptionalRequest 2,518,596 2,517,280 -1,316 -0.05%
Sema.IsObjCRequest 1,593,768 1,592,739 -1,029 -0.06%
Sema.IsSetterMutatingRequest 342,734 342,734 0 0.0%
Sema.IsStaticRequest 3,708,867 3,689,583 -19,284 -0.52%
Sema.LazyStoragePropertyRequest 2,550 2,550 0 0.0%
Sema.LookupInModuleRequest 6,740,560 6,728,242 -12,318 -0.18%
Sema.LookupPrecedenceGroupRequest 175,927 175,927 0 0.0%
Sema.MangleLocalTypeDeclRequest 500 500 0 0.0%
Sema.ModuleQualifiedLookupRequest 2,971,324 2,959,166 -12,158 -0.41%
Sema.NamingPatternRequest 205,986 205,987 1 0.0%
Sema.NeedsNewVTableEntryRequest 694,555 694,555 0 0.0%
Sema.NominalTypeLookupDirectCount 34,595,999 34,509,980 -86,019 -0.25%
Sema.NumAccessorBodiesSynthesized 189,118 189,118 0 0.0%
Sema.NumAccessorsSynthesized 294,770 294,770 0 0.0%
Sema.NumConformancesDeserialized 9,538,510 9,482,555 -55,955 -0.59%
Sema.NumConstraintScopes 27,683,442 27,676,686 -6,756 -0.02%
Sema.NumConstraintsConsideredForEdgeContraction 92,801,030 92,799,438 -1,592 -0.0%
Sema.NumCyclicOneWayComponentsCollapsed 0 0 0 0.0%
Sema.NumDeclsDeserialized 76,148,983 75,729,749 -419,234 -0.55%
Sema.NumDeclsTypechecked 1,411,738 1,411,738 0 0.0%
Sema.NumFunctionsTypechecked 529,144 529,144 0 0.0%
Sema.NumGenericSignatureBuilders 1,447,823 1,444,295 -3,528 -0.24%
Sema.NumLazyIterableDeclContexts 9,703,786 9,685,281 -18,505 -0.19%
Sema.NumLazyRequirementSignatures 1,027,006 1,026,458 -548 -0.05%
Sema.NumLazyRequirementSignaturesLoaded 744,163 743,696 -467 -0.06%
Sema.NumLeafScopes 17,646,317 17,640,577 -5,740 -0.03%
Sema.NumTypesDeserialized 23,189,530 23,113,873 -75,657 -0.33%
Sema.NumTypesValidated 1,521,044 1,521,042 -2 -0.0%
Sema.NumUnloadedLazyIterableDeclContexts 6,240,000 6,245,866 5,866 0.09%
Sema.OpaqueReadOwnershipRequest 281,382 281,382 0 0.0%
Sema.OpaqueResultTypeRequest 0 0 0 0.0%
Sema.OperatorPrecedenceGroupRequest 747 747 0 0.0%
Sema.OverriddenDeclsRequest 3,033,002 3,016,685 -16,317 -0.54%
Sema.ParamSpecifierRequest 1,305,215 1,305,212 -3 -0.0%
Sema.PatternBindingEntryRequest 463,350 463,351 1 0.0%
Sema.PreCheckFunctionBuilderRequest 0 0 0 0.0%
Sema.PropertyWrapperBackingPropertyInfoRequest 534,155 534,155 0 0.0%
Sema.PropertyWrapperBackingPropertyTypeRequest 539,524 539,524 0 0.0%
Sema.PropertyWrapperMutabilityRequest 630,316 630,316 0 0.0%
Sema.PropertyWrapperTypeInfoRequest 1 1 0 0.0%
Sema.ProtocolRequiresClassRequest 75,575 75,401 -174 -0.23%
Sema.QualifiedLookupRequest 7,354,874 7,322,370 -32,504 -0.44%
Sema.RangeInfoRequest 0 0 0 0.0%
Sema.RequirementRequest 113,786 113,783 -3 -0.0%
Sema.RequirementSignatureRequest 822,852 822,285 -567 -0.07%
Sema.RequiresOpaqueAccessorsRequest 1,313,504 1,313,504 0 0.0%
Sema.RequiresOpaqueModifyCoroutineRequest 272,298 272,298 0 0.0%
Sema.ResilienceExpansionRequest 1,862,693 1,862,672 -21 -0.0%
Sema.ResolveImplicitMemberRequest 372,808 372,208 -600 -0.16%
Sema.ResolveProtocolNameRequest 0 0 0 0.0%
Sema.ResultTypeRequest 693,041 693,027 -14 -0.0%
Sema.RootAndResultTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.RootTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.SelfAccessKindRequest 3,140,966 3,137,131 -3,835 -0.12%
Sema.SelfBoundsFromWhereClauseRequest 1,798,856 1,794,259 -4,597 -0.26%
Sema.SetterAccessLevelRequest 145,155 145,155 0 0.0%
Sema.StorageImplInfoRequest 1,420,497 1,420,497 0 0.0%
Sema.StoredPropertiesAndMissingMembersRequest 32,387 32,387 0 0.0%
Sema.StoredPropertiesRequest 353,778 353,778 0 0.0%
Sema.StructuralTypeRequest 2,323 2,323 0 0.0%
Sema.SuperclassDeclRequest 497,309 496,738 -571 -0.11%
Sema.SuperclassTypeRequest 60,016 60,016 0 0.0%
Sema.SynthesizeAccessorRequest 294,770 294,770 0 0.0%
Sema.SynthesizeDefaultInitRequest 5,380 5,380 0 0.0%
Sema.SynthesizeMemberwiseInitRequest 3,197 3,197 0 0.0%
Sema.TypeCheckFunctionBodyUntilRequest 529,144 529,144 0 0.0%
Sema.TypeCheckSourceFileRequest 27,072 27,072 0 0.0%
Sema.TypeDeclsFromWhereClauseRequest 30,548 30,549 1 0.0%
Sema.TypeRelationCheckRequest 0 0 0 0.0%
Sema.TypeWitnessRequest 9,282 9,277 -5 -0.05%
Sema.UnderlyingTypeDeclsReferencedRequest 380,836 379,752 -1,084 -0.28%
Sema.UnderlyingTypeRequest 38,709 38,709 0 0.0%
Sema.UnqualifiedLookupRequest 4,390,775 4,390,599 -176 -0.0%
Sema.ValueWitnessRequest 68,497 68,497 0 0.0%

Release

release brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 50,446,661,254,840 50,416,685,648,642 -29,975,606,198 -0.06%
LLVM.NumLLVMBytesOutput 1,925,957,516 1,925,971,820 14,304 0.0%
time.swift-driver.wall 9357.7s 9356.8s -980.1ms -0.01%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (18)
name old new delta delta_pct
AST.NumLoadedModules 30,053 30,053 0 0.0%
AST.NumTotalClangImportedEntities 1,181,420 1,178,169 -3,251 -0.28%
IRModule.NumIRBasicBlocks 5,448,505 5,448,505 0 0.0%
IRModule.NumIRFunctions 2,823,716 2,823,716 0 0.0%
IRModule.NumIRGlobals 3,182,311 3,182,311 0 0.0%
IRModule.NumIRInsts 53,522,509 53,522,509 0 0.0%
IRModule.NumIRValueSymbols 5,648,642 5,648,642 0 0.0%
LLVM.NumLLVMBytesOutput 1,925,957,516 1,925,971,820 14,304 0.0%
SILModule.NumSILGenFunctions 1,193,344 1,193,344 0 0.0%
SILModule.NumSILOptFunctions 1,712,579 1,712,579 0 0.0%
Sema.NumConformancesDeserialized 3,603,444 3,603,444 0 0.0%
Sema.NumConstraintScopes 27,318,811 27,318,803 -8 -0.0%
Sema.NumDeclsDeserialized 10,374,163 10,373,275 -888 -0.01%
Sema.NumFunctionsTypechecked 532,766 532,766 0 0.0%
Sema.NumGenericSignatureBuilders 280,409 280,409 0 0.0%
Sema.NumLazyIterableDeclContexts 1,313,748 1,313,651 -97 -0.01%
Sema.NumTypesDeserialized 5,260,737 5,260,390 -347 -0.01%
Sema.NumTypesValidated 826,549 826,549 0 0.0%

@CodaFi
Copy link
Contributor Author

CodaFi commented Dec 20, 2019

⛵️

@CodaFi CodaFi merged commit 4a943ab into swiftlang:master Dec 20, 2019
@CodaFi CodaFi deleted the a-little-looksie branch December 20, 2019 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants