Skip to content

Add profitability check to array-property-opt #29236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 3, 2020

Conversation

meg-gupta
Copy link
Contributor

This change adds additional heuristics to array-property-opt to avoid code size increase in cases where the optimization may not be beneficial. With this change, we do not specialize a loop if it has any opaque function calls or the instruction count computed recursively exceeds a predefined threshold.

@meg-gupta
Copy link
Contributor Author

@swift-ci Please Test Source Compatibility

@meg-gupta
Copy link
Contributor Author

@swift-ci Please benchmark

@gottesmm
Copy link
Contributor

@meg-gupta you also want to do the perf suite. Can give you code-size #s

@gottesmm
Copy link
Contributor

@swift-ci test compiler performance

@swift-ci
Copy link
Contributor

Performance: -O

Regression OLD NEW DELTA RATIO
MapReduceNSDecimalNumber 96 264 +175.0% 0.36x
MapReduceNSDecimalNumberShort 247 414 +67.6% 0.60x
Set.isSubset.Seq.Box0 1037 1334 +28.6% 0.78x
Set.isSubset.Seq.Box25 1174 1470 +25.2% 0.80x
Set.intersection.Seq.Box0 328 388 +18.3% 0.85x
Set.intersection.Seq.Box25 446 506 +13.5% 0.88x (?)
Set.subtracting.Empty.Box 13 14 +7.7% 0.93x (?)

Code size: -O

Improvement OLD NEW DELTA RATIO
SetTests.o 142053 135781 -4.4% 1.05x

Performance: -Osize

Regression OLD NEW DELTA RATIO
ObjectiveCBridgeStubFromArrayOfNSString2 2990 3230 +8.0% 0.93x (?)

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB

@eeckstein
Copy link
Contributor

Forgot to mention: can you add tests? E.g. including the case where a recursive function is called within the loop.
For better testing you could make the MaxInstThreshold configurable with a command line option and set it to a very low limit just for the test.

@swift-ci
Copy link
Contributor

Summary for master full

Unexpected test results, excluded stats for RxCocoa, Base64CoderSwiftUI, SwifterSwift

Regressions found (see below)

Debug-batch

debug-batch brief

Regressed (0)
name old new delta delta_pct
Improved (1)
name old new delta delta_pct
time.swift-driver.wall 3058.5s 3025.4s -33.1s -1.08% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (2)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 39,288,307,809,947 39,160,834,494,016 -127,473,315,931 -0.32%
LLVM.NumLLVMBytesOutput 1,788,440,340 1,788,435,354 -4,986 -0.0%

debug-batch detailed

Regressed (3)
name old new delta delta_pct
AST.NumSourceLinesPerSecond 4,790,769 4,875,592 84,823 1.77% ⛔
Driver.NumDriverPipePolls 104,180 107,803 3,623 3.48% ⛔
Driver.NumDriverPipeReads 95,374 98,926 3,552 3.72% ⛔
Improved (4)
name old new delta delta_pct
Sema.AccessLevelRequest 10,843,583 10,702,554 -141,029 -1.3% ✅
Sema.CollectOverriddenDeclsRequest 6,431,544 6,349,420 -82,124 -1.28% ✅
Sema.ProvideDefaultImplForRequest 6,431,544 6,349,420 -82,124 -1.28% ✅
Sema.USRGenerationRequest 7,722,162 7,637,781 -84,381 -1.09% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (201)
name old new delta delta_pct
AST.ImportSetCacheHit 1,653,563 1,653,796 233 0.01%
AST.ImportSetCacheMiss 636,724 636,768 44 0.01%
AST.ImportSetFoldHit 315,398 315,412 14 0.0%
AST.ImportSetFoldMiss 321,326 321,356 30 0.01%
AST.ModuleShadowCacheHit 2,013 2,013 0 0.0%
AST.ModuleShadowCacheMiss 1,512 1,512 0 0.0%
AST.ModuleVisibilityCacheHit 28,905 28,905 0 0.0%
AST.ModuleVisibilityCacheMiss 6,638 6,638 0 0.0%
AST.NumASTBytesAllocated 51,180,927,671 50,774,072,772 -406,854,899 -0.79%
AST.NumASTScopeLookups 3,528,626 3,528,932 306 0.01%
AST.NumBraceStmtASTScopeExpansions 619,337 619,337 0 0.0%
AST.NumBraceStmtASTScopes 619,337 619,337 0 0.0%
AST.NumDecls 139,853 139,853 0 0.0%
AST.NumDependencies 301,338 301,381 43 0.01%
AST.NumInfixOperators 53,695 53,695 0 0.0%
AST.NumIterableTypeBodyASTScopeExpansions 227,922 227,922 0 0.0%
AST.NumIterableTypeBodyASTScopes 279,407 279,516 109 0.04%
AST.NumLinkLibraries 0 0 0 0.0%
AST.NumLoadedModules 281,092 281,092 0 0.0%
AST.NumLocalTypeDecls 250 250 0 0.0%
AST.NumModuleLookupClassMember 6,927 6,927 0 0.0%
AST.NumModuleLookupValue 41,342,808 41,348,965 6,157 0.01%
AST.NumObjCMethods 24,240 24,240 0 0.0%
AST.NumPostfixOperators 49 49 0 0.0%
AST.NumPrecedenceGroups 25,840 25,840 0 0.0%
AST.NumPrefixOperators 99 99 0 0.0%
AST.NumReferencedDynamicNames 189 189 0 0.0%
AST.NumReferencedMemberNames 6,309,721 6,309,721 0 0.0%
AST.NumReferencedTopLevelNames 476,247 476,247 0 0.0%
AST.NumSourceBuffers 338,426 338,426 0 0.0%
AST.NumSourceLines 4,717,722 4,717,722 0 0.0%
AST.NumTotalClangImportedEntities 4,771,188 4,769,735 -1,453 -0.03%
Driver.ChildrenMaxRSS 217,788,876,800 217,087,539,200 -701,337,600 -0.32%
Driver.DriverDepCascadingDynamic 0 0 0 0.0%
Driver.DriverDepCascadingExternal 0 0 0 0.0%
Driver.DriverDepCascadingMember 0 0 0 0.0%
Driver.DriverDepCascadingNominal 0 0 0 0.0%
Driver.DriverDepCascadingTopLevel 0 0 0 0.0%
Driver.DriverDepDynamic 0 0 0 0.0%
Driver.DriverDepExternal 0 0 0 0.0%
Driver.DriverDepMember 0 0 0 0.0%
Driver.DriverDepNominal 0 0 0 0.0%
Driver.DriverDepTopLevel 0 0 0 0.0%
Driver.NumDriverJobsRun 28,030 28,030 0 0.0%
Driver.NumDriverJobsSkipped 0 0 0 0.0%
Driver.NumProcessFailures 0 0 0 0.0%
Frontend.MaxMallocUsage 813,153,487,264 811,407,310,312 -1,746,176,952 -0.21%
Frontend.NumInstructionsExecuted 39,288,307,809,947 39,160,834,494,016 -127,473,315,931 -0.32%
Frontend.NumProcessFailures 0 0 0 0.0%
IRModule.NumIRAliases 196,319 196,319 0 0.0%
IRModule.NumIRBasicBlocks 6,879,394 6,879,344 -50 -0.0%
IRModule.NumIRComdatSymbols 0 0 0 0.0%
IRModule.NumIRFunctions 3,372,375 3,372,375 0 0.0%
IRModule.NumIRGlobals 3,666,296 3,666,296 0 0.0%
IRModule.NumIRIFuncs 0 0 0 0.0%
IRModule.NumIRInsts 87,111,031 87,110,660 -371 -0.0%
IRModule.NumIRNamedMetaData 135,380 135,380 0 0.0%
IRModule.NumIRValueSymbols 6,383,540 6,383,540 0 0.0%
LLVM.NumLLVMBytesOutput 1,788,440,340 1,788,435,354 -4,986 -0.0%
Parse.NumFunctionsParsed 268,976 268,976 0 0.0%
Parse.NumIterableDeclContextParsed 617,825 617,841 16 0.0%
Parse.ParseAbstractFunctionBodyRequest 246,113 246,113 0 0.0%
Parse.ParseMembersRequest 485,975 485,991 16 0.0%
SILModule.NumSILGenDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILGenFunctions 1,701,345 1,701,345 0 0.0%
SILModule.NumSILGenGlobalVariables 53,554 53,554 0 0.0%
SILModule.NumSILGenVtables 18,627 18,627 0 0.0%
SILModule.NumSILGenWitnessTables 72,395 72,395 0 0.0%
SILModule.NumSILOptDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILOptFunctions 2,520,421 2,520,421 0 0.0%
SILModule.NumSILOptGlobalVariables 55,187 55,187 0 0.0%
SILModule.NumSILOptVtables 31,428 31,428 0 0.0%
SILModule.NumSILOptWitnessTables 169,282 169,282 0 0.0%
Sema.AbstractGenericSignatureRequest 30,630 30,630 0 0.0%
Sema.AnyObjectLookupRequest 281 281 0 0.0%
Sema.AreAllStoredPropertiesDefaultInitableRequest 21,023 21,023 0 0.0%
Sema.AttachedFunctionBuilderRequest 3 3 0 0.0%
Sema.AttachedPropertyWrapperTypeRequest 538,630 538,630 0 0.0%
Sema.AttachedPropertyWrappersRequest 2,286,436 2,286,434 -2 -0.0%
Sema.CallerSideDefaultArgExprRequest 78,637 78,637 0 0.0%
Sema.ClassAncestryFlagsRequest 96,510 96,510 0 0.0%
Sema.CompareDeclSpecializationRequest 458,338 457,889 -449 -0.1%
Sema.CursorInfoRequest 0 0 0 0.0%
Sema.CustomAttrNominalRequest 3 3 0 0.0%
Sema.DefaultAndMaxAccessLevelRequest 53,531 53,536 5 0.01%
Sema.DefaultArgumentExprRequest 38,177 38,177 0 0.0%
Sema.DefaultArgumentInitContextRequest 454 454 0 0.0%
Sema.DefaultDefinitionTypeRequest 7,790 7,790 0 0.0%
Sema.DefaultTypeRequest 454,010 454,010 0 0.0%
Sema.DynamicallyReplacedDeclRequest 1,013,155 1,013,155 0 0.0%
Sema.EmittedMembersRequest 25,899 25,899 0 0.0%
Sema.EnumRawTypeRequest 21,816 21,816 0 0.0%
Sema.EnumRawValuesRequest 10,526 10,526 0 0.0%
Sema.ExistentialConformsToSelfRequest 12,551 12,533 -18 -0.14%
Sema.ExistentialTypeSupportedRequest 15,806 15,806 0 0.0%
Sema.ExpandASTScopeRequest 6,428,603 6,428,705 102 0.0%
Sema.ExtendedNominalRequest 516,280 516,280 0 0.0%
Sema.ExtendedTypeRequest 75,641 75,641 0 0.0%
Sema.FunctionBuilderTypeRequest 3 3 0 0.0%
Sema.FunctionOperatorRequest 51,330 51,330 0 0.0%
Sema.GenericParamListRequest 8,788,914 8,709,354 -79,560 -0.91%
Sema.GenericSignatureRequest 2,690,195 2,689,892 -303 -0.01%
Sema.GetDestructorRequest 26,389 26,389 0 0.0%
Sema.HasCircularInheritanceRequest 22,567 22,567 0 0.0%
Sema.HasCircularInheritedProtocolsRequest 10,021 10,021 0 0.0%
Sema.HasCircularRawValueRequest 8,958 8,958 0 0.0%
Sema.HasDefaultInitRequest 51,660 51,660 0 0.0%
Sema.HasDynamicMemberLookupAttributeRequest 617,731 617,374 -357 -0.06%
Sema.HasMemberwiseInitRequest 19,272 19,272 0 0.0%
Sema.HasMissingDesignatedInitializersRequest 24,835 24,835 0 0.0%
Sema.HasUserDefinedDesignatedInitRequest 51,670 51,670 0 0.0%
Sema.InferredGenericSignatureRequest 163,450 163,450 0 0.0%
Sema.InheritedDeclsReferencedRequest 5,668,508 5,653,543 -14,965 -0.26%
Sema.InheritedTypeRequest 277,544 277,569 25 0.01%
Sema.InheritsSuperclassInitializersRequest 26,229 26,229 0 0.0%
Sema.InitKindRequest 91,279 91,279 0 0.0%
Sema.InterfaceTypeRequest 12,741,941 12,665,875 -76,066 -0.6%
Sema.IsABICompatibleOverrideRequest 135,234 135,234 0 0.0%
Sema.IsAccessorTransparentRequest 309,073 309,073 0 0.0%
Sema.IsCallableNominalTypeRequest 2,089 2,089 0 0.0%
Sema.IsDeclApplicableRequest 0 0 0 0.0%
Sema.IsDynamicRequest 1,686,288 1,686,288 0 0.0%
Sema.IsFinalRequest 2,432,601 2,421,137 -11,464 -0.47%
Sema.IsGetterMutatingRequest 415,704 415,704 0 0.0%
Sema.IsImplicitlyUnwrappedOptionalRequest 2,367,218 2,367,692 474 0.02%
Sema.IsObjCRequest 1,503,779 1,501,805 -1,974 -0.13%
Sema.IsSetterMutatingRequest 336,831 336,831 0 0.0%
Sema.IsStaticRequest 2,941,034 2,942,411 1,377 0.05%
Sema.LazyStoragePropertyRequest 2,197 2,197 0 0.0%
Sema.LookupInModuleRequest 5,620,673 5,624,341 3,668 0.07%
Sema.LookupPrecedenceGroupRequest 175,916 175,916 0 0.0%
Sema.MangleLocalTypeDeclRequest 500 500 0 0.0%
Sema.ModuleQualifiedLookupRequest 2,543,658 2,547,019 3,361 0.13%
Sema.NamedLazyMemberLoadFailureCount 12,830 12,839 9 0.07%
Sema.NamedLazyMemberLoadSuccessCount 11,152,963 11,147,405 -5,558 -0.05%
Sema.NamingPatternRequest 177,242 177,240 -2 -0.0%
Sema.NeedsNewVTableEntryRequest 661,583 661,583 0 0.0%
Sema.NominalTypeLookupDirectCount 32,954,696 32,905,048 -49,648 -0.15%
Sema.NumAccessorBodiesSynthesized 189,102 189,102 0 0.0%
Sema.NumAccessorsSynthesized 277,639 277,639 0 0.0%
Sema.NumConformancesDeserialized 8,226,687 8,170,291 -56,396 -0.69%
Sema.NumConstraintScopes 27,506,584 27,500,712 -5,872 -0.02%
Sema.NumConstraintsConsideredForEdgeContraction 92,131,284 92,130,930 -354 -0.0%
Sema.NumCyclicOneWayComponentsCollapsed 0 0 0 0.0%
Sema.NumDeclsDeserialized 63,375,641 63,024,886 -350,755 -0.55%
Sema.NumDeclsTypechecked 1,412,191 1,412,191 0 0.0%
Sema.NumFunctionsTypechecked 529,260 529,260 0 0.0%
Sema.NumGenericSignatureBuilders 1,193,223 1,188,366 -4,857 -0.41%
Sema.NumLazyIterableDeclContexts 7,579,688 7,560,530 -19,158 -0.25%
Sema.NumLazyRequirementSignatures 761,845 760,877 -968 -0.13%
Sema.NumLazyRequirementSignaturesLoaded 568,391 567,678 -713 -0.13%
Sema.NumLeafScopes 17,517,203 17,511,997 -5,206 -0.03%
Sema.NumTypesDeserialized 19,472,301 19,394,287 -78,014 -0.4%
Sema.NumTypesValidated 1,373,618 1,373,617 -1 -0.0%
Sema.NumUnloadedLazyIterableDeclContexts 4,623,113 4,629,784 6,671 0.14%
Sema.OpaqueReadOwnershipRequest 262,684 262,684 0 0.0%
Sema.OpaqueResultTypeRequest 0 0 0 0.0%
Sema.OperatorPrecedenceGroupRequest 724 724 0 0.0%
Sema.OverriddenDeclsRequest 2,264,436 2,252,192 -12,244 -0.54%
Sema.ParamSpecifierRequest 1,190,801 1,190,802 1 0.0%
Sema.PatternBindingEntryRequest 433,181 433,179 -2 -0.0%
Sema.PatternTypeRequest 554,490 554,488 -2 -0.0%
Sema.PreCheckFunctionBuilderRequest 0 0 0 0.0%
Sema.PropertyWrapperBackingPropertyInfoRequest 533,608 533,608 0 0.0%
Sema.PropertyWrapperBackingPropertyTypeRequest 538,630 538,630 0 0.0%
Sema.PropertyWrapperMutabilityRequest 610,627 610,627 0 0.0%
Sema.PropertyWrapperTypeInfoRequest 1 1 0 0.0%
Sema.ProtocolRequiresClassRequest 59,021 59,048 27 0.05%
Sema.QualifiedLookupRequest 6,605,037 6,594,719 -10,318 -0.16%
Sema.RangeInfoRequest 0 0 0 0.0%
Sema.RequirementRequest 107,348 107,348 0 0.0%
Sema.RequirementSignatureRequest 628,722 628,054 -668 -0.11%
Sema.RequiresOpaqueAccessorsRequest 1,286,368 1,286,368 0 0.0%
Sema.RequiresOpaqueModifyCoroutineRequest 253,458 253,458 0 0.0%
Sema.ResilienceExpansionRequest 1,697,057 1,697,058 1 0.0%
Sema.ResolveImplicitMemberRequest 354,531 353,596 -935 -0.26%
Sema.ResolveProtocolNameRequest 0 0 0 0.0%
Sema.ResultTypeRequest 633,311 633,313 2 0.0%
Sema.RootAndResultTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.RootTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.SelfAccessKindRequest 779,711 779,975 264 0.03%
Sema.SelfBoundsFromWhereClauseRequest 1,489,661 1,491,561 1,900 0.13%
Sema.SetterAccessLevelRequest 141,907 141,907 0 0.0%
Sema.StorageImplInfoRequest 1,375,308 1,375,308 0 0.0%
Sema.StoredPropertiesAndMissingMembersRequest 29,897 29,897 0 0.0%
Sema.StoredPropertiesRequest 305,308 305,308 0 0.0%
Sema.StructuralTypeRequest 2,078 2,078 0 0.0%
Sema.SuperclassDeclRequest 395,962 395,422 -540 -0.14%
Sema.SuperclassTypeRequest 52,537 52,537 0 0.0%
Sema.SynthesizeAccessorRequest 277,639 277,639 0 0.0%
Sema.SynthesizeDefaultInitRequest 4,810 4,810 0 0.0%
Sema.SynthesizeMemberwiseInitRequest 3,061 3,061 0 0.0%
Sema.TypeCheckFunctionBodyUntilRequest 529,260 529,260 0 0.0%
Sema.TypeCheckSourceFileRequest 27,076 27,076 0 0.0%
Sema.TypeDeclsFromWhereClauseRequest 28,699 28,704 5 0.02%
Sema.TypeRelationCheckRequest 0 0 0 0.0%
Sema.TypeWitnessRequest 8,337 8,337 0 0.0%
Sema.UnderlyingTypeDeclsReferencedRequest 307,381 307,179 -202 -0.07%
Sema.UnderlyingTypeRequest 33,849 33,849 0 0.0%
Sema.UnqualifiedLookupRequest 3,671,569 3,671,875 306 0.01%
Sema.ValueWitnessRequest 63,137 63,137 0 0.0%

Release

release brief

Regressed (0)
name old new delta delta_pct
Improved (1)
name old new delta delta_pct
time.swift-driver.wall 5647.2s 5565.0s -82.2s -1.45% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (2)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 48,383,368,842,806 48,251,824,792,952 -131,544,049,854 -0.27%
LLVM.NumLLVMBytesOutput 1,875,996,990 1,869,919,470 -6,077,520 -0.32%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (18)
name old new delta delta_pct
AST.NumLoadedModules 30,053 30,053 0 0.0%
AST.NumTotalClangImportedEntities 1,126,674 1,126,674 0 0.0%
IRModule.NumIRBasicBlocks 5,262,333 5,241,579 -20,754 -0.39%
IRModule.NumIRFunctions 2,784,170 2,783,000 -1,170 -0.04%
IRModule.NumIRGlobals 3,180,336 3,179,067 -1,269 -0.04%
IRModule.NumIRInsts 52,180,843 52,013,931 -166,912 -0.32%
IRModule.NumIRValueSymbols 5,607,174 5,604,738 -2,436 -0.04%
LLVM.NumLLVMBytesOutput 1,875,996,990 1,869,919,470 -6,077,520 -0.32%
SILModule.NumSILGenFunctions 1,189,889 1,189,889 0 0.0%
SILModule.NumSILOptFunctions 1,626,955 1,627,086 131 0.01%
Sema.NumConformancesDeserialized 3,474,227 3,474,227 0 0.0%
Sema.NumConstraintScopes 27,217,762 27,217,762 0 0.0%
Sema.NumDeclsDeserialized 10,605,506 10,605,506 0 0.0%
Sema.NumFunctionsTypechecked 532,882 532,882 0 0.0%
Sema.NumGenericSignatureBuilders 278,724 278,724 0 0.0%
Sema.NumLazyIterableDeclContexts 1,313,977 1,313,977 0 0.0%
Sema.NumTypesDeserialized 5,181,056 5,181,056 0 0.0%
Sema.NumTypesValidated 826,803 826,803 0 0.0%

@meg-gupta
Copy link
Contributor Author

Leaving this PR on hold for a bit. I am investigating ways we improve analyses for LICM, which can avoid the need for array-property-opt in some cases.

@airspeedswift
Copy link
Member

@swift-ci test compiler performance

Meghana Gupta and others added 2 commits January 29, 2020 12:57
This change adds additional heuristics to array-property-opt to avoid
code size increase in cases where the optimization may not be
beneficial. With this change, we do not specialize a loop if it has any
opaque function calls or the instruction count computed recursively
exceeds a predefined threshold.
Add debugging statements

Increase threshold for deeper loop nests
@meg-gupta meg-gupta changed the title [WIP] Add profitability check to array-property-opt Add profitability check to array-property-opt Jan 31, 2020
@meg-gupta
Copy link
Contributor Author

@swift-ci Test Source Compatibility
@swift-ci please benchmark
@swift-ci test compiler performance

@theblixguy
Copy link
Collaborator

Unfortunately passing multiple CI commands in a single comment doesn’t work, so you need to pass one at a time.

@meg-gupta
Copy link
Contributor Author

@swift-ci Test Source Compatibility

@meg-gupta
Copy link
Contributor Author

@swift-ci please benchmark

@meg-gupta
Copy link
Contributor Author

@swift-ci test compiler performance

@swift-ci
Copy link
Contributor

Performance: -O

Regression OLD NEW DELTA RATIO
MapReduceNSDecimalNumber 114 291 +155.3% 0.39x
MapReduceNSDecimalNumberShort 279 458 +64.2% 0.61x
Set.isSubset.Seq.Box0 1152 1481 +28.6% 0.78x
Set.isSubset.Seq.Box25 1306 1633 +25.0% 0.80x
Set.intersection.Seq.Box0 365 432 +18.4% 0.84x
Set.intersection.Seq.Box25 496 562 +13.3% 0.88x
 
Improvement OLD NEW DELTA RATIO
FlattenListFlatMap 9344 7290 -22.0% 1.28x (?)
DataCopyBytesMedium 480 402 -16.2% 1.19x (?)
PrefixAnySeqCRangeIter 43 38 -11.6% 1.13x (?)
FlattenListLoop 4811 4276 -11.1% 1.13x (?)

Code size: -O

Improvement OLD NEW DELTA RATIO
SetTests.o 142021 135749 -4.4% 1.05x
MapReduce.o 29095 28727 -1.3% 1.01x

Performance: -Osize

Regression OLD NEW DELTA RATIO
UTF8Decode_InitFromData_ascii_as_ascii 619 691 +11.6% 0.90x (?)
 
Improvement OLD NEW DELTA RATIO
FlattenListLoop 4766 4107 -13.8% 1.16x (?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6020 5260 -12.6% 1.14x (?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 48300 44500 -7.9% 1.09x (?)
DataReplaceMediumBuffer 10100 9400 -6.9% 1.07x (?)

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

@meg-gupta
Copy link
Contributor Author

Regression OLD NEW DELTA RATIO
MapReduceNSDecimalNumber 114 291 +155.3% 0.39x
MapReduceNSDecimalNumberShort 279 458 +64.2% 0.61x
Set.isSubset.Seq.Box0 1152 1481 +28.6% 0.78x
Set.isSubset.Seq.Box25 1306 1633 +25.0% 0.80x
Set.intersection.Seq.Box0 365 432 +18.4% 0.84x
Set.intersection.Seq.Box25

MapReduce test regressions are because the control flow pattern changes limit some jump threading which in turn limit arc optimizations. The regression is due to additional obj.retain/release calls in the loop.

SetTest regressions are because it sees a library function Set.Variant.insert which has a class_method that cannot be devirtualized, even otherwise the AnalysisThreshold will exceed scanning the library function call chain.

@atrick atrick self-requested a review February 1, 2020 00:36
Copy link
Contributor

@atrick atrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no concerns with checking this in if it meets our immediate needs. The implementation looks good. It's clear how it could be extended and generalized in the future if that becomes worthwhile.

If you're not immediately fixing those regressions, then please file a bug to follow up!

@meg-gupta
Copy link
Contributor Author

@swift-ci smoke test please

@swift-ci
Copy link
Contributor

swift-ci commented Feb 1, 2020

Summary for master full

Unexpected test results, excluded stats for RxCocoa, SwifterSwift, Base64CoderSwiftUI

No regressions above thresholds

Debug-batch

debug-batch brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 80,068,151,727,931 80,169,868,045,173 101,716,317,242 0.13%
LLVM.NumLLVMBytesOutput 1,822,638,608 1,822,630,834 -7,774 -0.0%
time.swift-driver.wall 6257.9s 6260.3s 2.5s 0.04%

debug-batch detailed

Regressed (0)
name old new delta delta_pct
Improved (2)
name old new delta delta_pct
Driver.NumDriverPipePolls 80,933 79,496 -1,437 -1.78% ✅
Driver.NumDriverPipeReads 68,396 66,925 -1,471 -2.15% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (208)
name old new delta delta_pct
AST.ImportSetCacheHit 2,170,260 2,170,248 -12 -0.0%
AST.ImportSetCacheMiss 1,044,801 1,044,744 -57 -0.01%
AST.ImportSetFoldHit 563,450 563,444 -6 -0.0%
AST.ImportSetFoldMiss 481,350 481,300 -50 -0.01%
AST.ModuleShadowCacheHit 3,602 3,602 0 0.0%
AST.ModuleShadowCacheMiss 1,913 1,913 0 0.0%
AST.ModuleVisibilityCacheHit 35,063 35,063 0 0.0%
AST.ModuleVisibilityCacheMiss 9,157 9,157 0 0.0%
AST.NumASTBytesAllocated 72,159,799,317 72,218,664,665 58,865,348 0.08%
AST.NumASTScopeLookups 4,401,446 4,400,990 -456 -0.01%
AST.NumBraceStmtASTScopeExpansions 639,722 639,722 0 0.0%
AST.NumBraceStmtASTScopes 639,722 639,722 0 0.0%
AST.NumDecls 143,587 143,587 0 0.0%
AST.NumDependencies 448,573 448,496 -77 -0.02%
AST.NumInfixOperators 54,567 54,567 0 0.0%
AST.NumIterableTypeBodyASTScopeExpansions 278,365 278,355 -10 -0.0%
AST.NumIterableTypeBodyASTScopes 365,201 365,213 12 0.0%
AST.NumLinkLibraries 0 0 0 0.0%
AST.NumLoadedModules 414,192 414,192 0 0.0%
AST.NumLocalTypeDecls 254 254 0 0.0%
AST.NumModuleLookupClassMember 6,927 6,927 0 0.0%
AST.NumModuleLookupValue 54,994,687 54,995,116 429 0.0%
AST.NumObjCMethods 24,747 24,747 0 0.0%
AST.NumPostfixOperators 49 49 0 0.0%
AST.NumPrecedenceGroups 26,329 26,329 0 0.0%
AST.NumPrefixOperators 99 99 0 0.0%
AST.NumReferencedDynamicNames 189 189 0 0.0%
AST.NumReferencedMemberNames 6,469,647 6,469,647 0 0.0%
AST.NumReferencedTopLevelNames 565,430 565,430 0 0.0%
AST.NumSourceBuffers 1,033,788 1,033,788 0 0.0%
AST.NumSourceLines 4,795,108 4,795,108 0 0.0%
AST.NumSourceLinesPerSecond 3,256,927 3,246,228 -10,699 -0.33%
AST.NumTotalClangImportedEntities 6,367,798 6,367,941 143 0.0%
Driver.ChildrenMaxRSS 252,045,721,600 252,944,359,424 898,637,824 0.36%
Driver.DriverDepCascadingDynamic 0 0 0 0.0%
Driver.DriverDepCascadingExternal 0 0 0 0.0%
Driver.DriverDepCascadingMember 0 0 0 0.0%
Driver.DriverDepCascadingNominal 0 0 0 0.0%
Driver.DriverDepCascadingTopLevel 0 0 0 0.0%
Driver.DriverDepDynamic 0 0 0 0.0%
Driver.DriverDepExternal 0 0 0 0.0%
Driver.DriverDepMember 0 0 0 0.0%
Driver.DriverDepNominal 0 0 0 0.0%
Driver.DriverDepTopLevel 0 0 0 0.0%
Driver.NumDriverJobsRun 28,508 28,508 0 0.0%
Driver.NumDriverJobsSkipped 0 0 0 0.0%
Driver.NumProcessFailures 0 0 0 0.0%
Frontend.MaxMallocUsage 1,494,874,532,152 1,496,369,546,992 1,495,014,840 0.1%
Frontend.NumInstructionsExecuted 80,068,151,727,931 80,169,868,045,173 101,716,317,242 0.13%
Frontend.NumProcessFailures 0 0 0 0.0%
IRModule.NumIRAliases 199,469 199,469 0 0.0%
IRModule.NumIRBasicBlocks 6,985,707 6,985,687 -20 -0.0%
IRModule.NumIRComdatSymbols 0 0 0 0.0%
IRModule.NumIRFunctions 3,437,768 3,437,768 0 0.0%
IRModule.NumIRGlobals 3,735,700 3,735,700 0 0.0%
IRModule.NumIRIFuncs 0 0 0 0.0%
IRModule.NumIRInsts 88,378,438 88,378,166 -272 -0.0%
IRModule.NumIRNamedMetaData 137,660 137,660 0 0.0%
IRModule.NumIRValueSymbols 6,508,017 6,508,017 0 0.0%
LLVM.NumLLVMBytesOutput 1,822,638,608 1,822,630,834 -7,774 -0.0%
Parse.NumFunctionsParsed 273,969 273,969 0 0.0%
Parse.NumIterableDeclContextParsed 963,585 963,568 -17 -0.0%
Parse.ParseAbstractFunctionBodyRequest 250,031 250,031 0 0.0%
Parse.ParseMembersRequest 792,390 792,373 -17 -0.0%
SILGen.GenerateSILRequest 28,497 28,497 0 0.0%
SILModule.NumSILGenDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILGenFunctions 1,733,138 1,733,138 0 0.0%
SILModule.NumSILGenGlobalVariables 54,724 54,724 0 0.0%
SILModule.NumSILGenVtables 18,938 18,938 0 0.0%
SILModule.NumSILGenWitnessTables 73,908 73,908 0 0.0%
SILModule.NumSILOptDefaultWitnessTables 0 0 0 0.0%
SILModule.NumSILOptFunctions 2,558,124 2,558,124 0 0.0%
SILModule.NumSILOptGlobalVariables 56,390 56,390 0 0.0%
SILModule.NumSILOptVtables 32,041 32,041 0 0.0%
SILModule.NumSILOptWitnessTables 171,962 171,962 0 0.0%
Sema.AbstractGenericSignatureRequest 35,189 35,189 0 0.0%
Sema.AccessLevelRequest 14,725,281 14,743,912 18,631 0.13%
Sema.AnyObjectLookupRequest 281 281 0 0.0%
Sema.AreAllStoredPropertiesDefaultInitableRequest 23,540 23,540 0 0.0%
Sema.AttachedFunctionBuilderRequest 1 1 0 0.0%
Sema.AttachedPropertyWrapperTypeRequest 550,960 550,960 0 0.0%
Sema.AttachedPropertyWrappersRequest 2,423,722 2,423,702 -20 -0.0%
Sema.CallerSideDefaultArgExprRequest 78,971 78,971 0 0.0%
Sema.ClassAncestryFlagsRequest 113,887 113,887 0 0.0%
Sema.CollectOverriddenDeclsRequest 9,102,774 9,109,958 7,184 0.08%
Sema.CompareDeclSpecializationRequest 489,474 489,395 -79 -0.02%
Sema.CursorInfoRequest 0 0 0 0.0%
Sema.CustomAttrNominalRequest 1 1 0 0.0%
Sema.DefaultAndMaxAccessLevelRequest 59,765 59,768 3 0.01%
Sema.DefaultArgumentExprRequest 39,319 39,319 0 0.0%
Sema.DefaultArgumentInitContextRequest 458 458 0 0.0%
Sema.DefaultDefinitionTypeRequest 8,411 8,411 0 0.0%
Sema.DefaultTypeRequest 485,948 485,948 0 0.0%
Sema.DifferentiableAttributeTypeCheckRequest 0 0 0 0.0%
Sema.DirectLookupRequest 35,499,400 35,503,155 3,755 0.01%
Sema.DynamicallyReplacedDeclRequest 1,089,887 1,089,887 0 0.0%
Sema.EmittedMembersRequest 28,512 28,512 0 0.0%
Sema.EnumRawTypeRequest 26,380 26,380 0 0.0%
Sema.EnumRawValuesRequest 10,748 10,748 0 0.0%
Sema.ExistentialConformsToSelfRequest 14,515 14,527 12 0.08%
Sema.ExistentialTypeSupportedRequest 18,355 18,355 0 0.0%
Sema.ExpandASTScopeRequest 7,850,518 7,850,413 -105 -0.0%
Sema.ExtendedNominalRequest 942,165 942,165 0 0.0%
Sema.ExtendedTypeRequest 86,341 86,341 0 0.0%
Sema.FunctionBuilderTypeRequest 1 1 0 0.0%
Sema.FunctionOperatorRequest 74,104 74,104 0 0.0%
Sema.GenericParamListRequest 11,885,825 11,895,224 9,399 0.08%
Sema.GenericSignatureRequest 3,359,574 3,359,288 -286 -0.01%
Sema.GetDestructorRequest 28,919 28,919 0 0.0%
Sema.HasCircularInheritanceRequest 24,294 24,294 0 0.0%
Sema.HasCircularInheritedProtocolsRequest 10,466 10,466 0 0.0%
Sema.HasCircularRawValueRequest 9,127 9,127 0 0.0%
Sema.HasDefaultInitRequest 58,039 58,039 0 0.0%
Sema.HasDynamicMemberLookupAttributeRequest 677,340 677,363 23 0.0%
Sema.HasMemberwiseInitRequest 20,721 20,721 0 0.0%
Sema.HasMissingDesignatedInitializersRequest 27,324 27,324 0 0.0%
Sema.HasUserDefinedDesignatedInitRequest 58,050 58,050 0 0.0%
Sema.InferredGenericSignatureRequest 194,168 194,158 -10 -0.01%
Sema.InheritedDeclsReferencedRequest 7,489,860 7,487,220 -2,640 -0.04%
Sema.InheritedTypeRequest 324,930 324,925 -5 -0.0%
Sema.InheritsSuperclassInitializersRequest 30,498 30,498 0 0.0%
Sema.InitKindRequest 100,173 100,173 0 0.0%
Sema.InterfaceTypeRequest 20,562,265 20,571,799 9,534 0.05%
Sema.IsABICompatibleOverrideRequest 138,436 138,436 0 0.0%
Sema.IsAccessorTransparentRequest 324,533 324,533 0 0.0%
Sema.IsCallableNominalTypeRequest 2,132 2,132 0 0.0%
Sema.IsDeclApplicableRequest 0 0 0 0.0%
Sema.IsDynamicRequest 1,786,419 1,786,419 0 0.0%
Sema.IsFinalRequest 2,888,982 2,891,044 2,062 0.07%
Sema.IsGetterMutatingRequest 443,653 443,653 0 0.0%
Sema.IsImplicitlyUnwrappedOptionalRequest 2,581,151 2,581,201 50 0.0%
Sema.IsObjCRequest 1,701,165 1,701,580 415 0.02%
Sema.IsSetterMutatingRequest 349,588 349,588 0 0.0%
Sema.IsStaticRequest 3,759,191 3,758,563 -628 -0.02%
Sema.LazyStoragePropertyRequest 2,570 2,570 0 0.0%
Sema.LookupInModuleRequest 7,302,325 7,303,226 901 0.01%
Sema.LookupPrecedenceGroupRequest 179,608 179,608 0 0.0%
Sema.MangleLocalTypeDeclRequest 508 508 0 0.0%
Sema.ModuleQualifiedLookupRequest 3,169,048 3,170,386 1,338 0.04%
Sema.NamedLazyMemberLoadFailureCount 15,476 15,469 -7 -0.05%
Sema.NamedLazyMemberLoadSuccessCount 13,005,964 13,005,224 -740 -0.01%
Sema.NamingPatternRequest 212,950 212,930 -20 -0.01%
Sema.NeedsNewVTableEntryRequest 708,793 708,793 0 0.0%
Sema.NumAccessorBodiesSynthesized 191,878 191,878 0 0.0%
Sema.NumAccessorsSynthesized 300,881 300,881 0 0.0%
Sema.NumConformancesDeserialized 10,762,798 10,767,132 4,334 0.04%
Sema.NumConstraintScopes 28,918,025 28,918,434 409 0.0%
Sema.NumConstraintsConsideredForEdgeContraction 98,541,823 98,541,531 -292 -0.0%
Sema.NumCyclicOneWayComponentsCollapsed 0 0 0 0.0%
Sema.NumDeclsDeserialized 85,145,116 85,182,201 37,085 0.04%
Sema.NumDeclsTypechecked 1,437,915 1,437,915 0 0.0%
Sema.NumFunctionsTypechecked 538,730 538,730 0 0.0%
Sema.NumGenericSignatureBuilders 1,639,565 1,640,064 499 0.03%
Sema.NumLazyIterableDeclContexts 10,358,119 10,359,108 989 0.01%
Sema.NumLazyRequirementSignatures 1,080,802 1,080,988 186 0.02%
Sema.NumLazyRequirementSignaturesLoaded 799,493 799,672 179 0.02%
Sema.NumLeafScopes 18,611,644 18,612,044 400 0.0%
Sema.NumTypesDeserialized 25,426,554 25,437,026 10,472 0.04%
Sema.NumTypesValidated 1,556,651 1,556,631 -20 -0.0%
Sema.NumUnloadedLazyIterableDeclContexts 6,402,035 6,399,902 -2,133 -0.03%
Sema.OpaqueReadOwnershipRequest 287,502 287,502 0 0.0%
Sema.OpaqueResultTypeRequest 0 0 0 0.0%
Sema.OperatorPrecedenceGroupRequest 747 747 0 0.0%
Sema.OverriddenDeclsRequest 2,726,029 2,728,061 2,032 0.07%
Sema.ParamSpecifierRequest 1,333,468 1,333,468 0 0.0%
Sema.PatternBindingEntryRequest 474,577 474,557 -20 -0.0%
Sema.PatternTypeRequest 598,454 598,434 -20 -0.0%
Sema.PreCheckFunctionBuilderRequest 0 0 0 0.0%
Sema.PropertyWrapperBackingPropertyInfoRequest 545,513 545,513 0 0.0%
Sema.PropertyWrapperBackingPropertyTypeRequest 550,960 550,960 0 0.0%
Sema.PropertyWrapperMutabilityRequest 643,552 643,552 0 0.0%
Sema.PropertyWrapperTypeInfoRequest 7 7 0 0.0%
Sema.ProtocolRequiresClassRequest 77,556 77,529 -27 -0.03%
Sema.ProvideDefaultImplForRequest 9,102,774 9,109,958 7,184 0.08%
Sema.QualifiedLookupRequest 7,569,844 7,570,703 859 0.01%
Sema.RangeInfoRequest 0 0 0 0.0%
Sema.RequirementRequest 120,118 120,118 0 0.0%
Sema.RequirementSignatureRequest 881,173 881,304 131 0.01%
Sema.RequiresOpaqueAccessorsRequest 1,341,327 1,341,327 0 0.0%
Sema.RequiresOpaqueModifyCoroutineRequest 278,072 278,072 0 0.0%
Sema.ResilienceExpansionRequest 1,904,109 1,904,099 -10 -0.0%
Sema.ResolveImplicitMemberRequest 407,456 407,679 223 0.05%
Sema.ResolveProtocolNameRequest 0 0 0 0.0%
Sema.ResultTypeRequest 708,715 708,715 0 0.0%
Sema.RootAndResultTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.RootTypeOfKeypathDynamicMemberRequest 0 0 0 0.0%
Sema.SelfAccessKindRequest 936,979 936,622 -357 -0.04%
Sema.SelfBoundsFromWhereClauseRequest 1,910,396 1,908,320 -2,076 -0.11%
Sema.SetterAccessLevelRequest 147,874 147,874 0 0.0%
Sema.StorageImplInfoRequest 1,453,715 1,453,715 0 0.0%
Sema.StoredPropertiesAndMissingMembersRequest 33,206 33,206 0 0.0%
Sema.StoredPropertiesRequest 378,155 378,155 0 0.0%
Sema.StructuralTypeRequest 2,336 2,336 0 0.0%
Sema.SuperclassDeclRequest 520,464 520,530 66 0.01%
Sema.SuperclassTypeRequest 61,701 61,701 0 0.0%
Sema.SynthesizeAccessorRequest 300,881 300,881 0 0.0%
Sema.SynthesizeDefaultInitRequest 5,421 5,421 0 0.0%
Sema.SynthesizeMemberwiseInitRequest 3,247 3,247 0 0.0%
Sema.TypeCheckFunctionBodyUntilRequest 538,730 538,730 0 0.0%
Sema.TypeCheckSourceFileRequest 27,532 27,532 0 0.0%
Sema.TypeDeclsFromWhereClauseRequest 31,218 31,221 3 0.01%
Sema.TypeRelationCheckRequest 0 0 0 0.0%
Sema.TypeWitnessRequest 9,906 9,906 0 0.0%
Sema.USRGenerationRequest 29,415,069 29,444,885 29,816 0.1%
Sema.UnderlyingTypeDeclsReferencedRequest 406,637 406,877 240 0.06%
Sema.UnderlyingTypeRequest 39,330 39,330 0 0.0%
Sema.UnqualifiedLookupRequest 4,548,581 4,548,125 -456 -0.01%
Sema.ValueWitnessRequest 70,316 70,316 0 0.0%

Release

release brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 49,354,888,644,392 49,262,143,438,784 -92,745,205,608 -0.19%
LLVM.NumLLVMBytesOutput 1,862,723,524 1,858,596,162 -4,127,362 -0.22%
time.swift-driver.wall 9072.4s 9067.8s -4.7s -0.05%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (18)
name old new delta delta_pct
AST.NumLoadedModules 30,691 30,691 0 0.0%
AST.NumTotalClangImportedEntities 1,155,245 1,155,245 0 0.0%
IRModule.NumIRBasicBlocks 5,035,260 5,018,408 -16,852 -0.33%
IRModule.NumIRFunctions 2,828,694 2,827,930 -764 -0.03%
IRModule.NumIRGlobals 3,256,565 3,255,492 -1,073 -0.03%
IRModule.NumIRInsts 50,642,167 50,528,191 -113,976 -0.23%
IRModule.NumIRValueSymbols 5,720,660 5,718,840 -1,820 -0.03%
LLVM.NumLLVMBytesOutput 1,862,723,524 1,858,596,162 -4,127,362 -0.22%
SILModule.NumSILGenFunctions 1,212,481 1,212,481 0 0.0%
SILModule.NumSILOptFunctions 1,027,129 1,027,217 88 0.01%
Sema.NumConformancesDeserialized 3,528,955 3,528,955 0 0.0%
Sema.NumConstraintScopes 28,450,539 28,450,546 7 0.0%
Sema.NumDeclsDeserialized 10,872,555 10,872,555 0 0.0%
Sema.NumFunctionsTypechecked 542,352 542,352 0 0.0%
Sema.NumGenericSignatureBuilders 285,906 285,906 0 0.0%
Sema.NumLazyIterableDeclContexts 1,345,984 1,345,984 0 0.0%
Sema.NumTypesDeserialized 5,270,754 5,270,754 0 0.0%
Sema.NumTypesValidated 841,795 841,795 0 0.0%

@meg-gupta
Copy link
Contributor Author

@swift-ci Please smoke test

Copy link
Contributor

@eeckstein eeckstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@meg-gupta meg-gupta merged commit fbc8457 into swiftlang:master Feb 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants