Skip to content

[Serialization] Drop GenericEnvironmentID for GenericSignatureID #26862

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 28, 2019

Conversation

jrose-apple
Copy link
Contributor

@jrose-apple jrose-apple commented Aug 27, 2019

Built on top of #26813 for now (only the last commit is relevant). Prompted by this FIXME I noticed recently (added in #26800).


A generic environment is always serialized as a GenericSignature with a lazily-recreated environment, though sometimes it has to include extra info specifically for generic environments used by SIL. The code that was doing this claimed a bit for disambiguating between the two, shrinking the permitted size of a compiled module from 2^31 bits to 2^30. (The code isn't just needlessly complicated; GenericEnvironments used to be serialized with more information.)

Rather than have two representations for GenericEnvironmentID, this commit serializes generic signatures and "special SIL generic environments" in the same list, automatically picking the most compact representation, and makes sure the right type always comes out on the deserialization side. Rather than have two representations for GenericEnvironmentID, this commit just drops it altogether in favor of referencing GenericSignatures directly. This causes a negligible file size shrinkage for swiftmodules in addition to eliminating the problematic disambiguation bit.

For now, the Deserialization logic will continue to cache GenericEnvironments that are used directly by Deserialization, but really that should probably be done at the AST level. Then we can simplify further to ModuleFile tracking a plain list of GenericSignatures.

@jrose-apple
Copy link
Contributor Author

@swift-ci Please test

@jrose-apple
Copy link
Contributor Author

@swift-ci Please test source compatibility

@jrose-apple
Copy link
Contributor Author

@swift-ci Please smoke test compiler performance

@swift-ci
Copy link
Contributor

Summary for master smoketest

Unexpected test results, excluded stats for ReactiveCocoa

No regressions above thresholds

Debug

debug brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 104,741,423,560 104,741,136,116 -287,444 -0.0%
LLVM.NumLLVMBytesOutput 6,145,872 6,145,872 0 0.0%
time.swift-driver.wall 8.5s 8.4s -64.9ms -0.76%

debug detailed

Regressed (0)
name old new delta delta_pct
Improved (2)
name old new delta delta_pct
Sema.NumLazyGenericEnvironments 22,698 21,870 -828 -3.65% ✅
Sema.NumLazyGenericEnvironmentsLoaded 1,942 1,666 -276 -14.21% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (19)
name old new delta delta_pct
AST.NumLoadedModules 660 660 0 0.0%
AST.NumTotalClangImportedEntities 3,768 3,768 0 0.0%
IRModule.NumIRBasicBlocks 17,905 17,905 0 0.0%
IRModule.NumIRFunctions 10,613 10,613 0 0.0%
IRModule.NumIRGlobals 8,320 8,320 0 0.0%
IRModule.NumIRInsts 308,861 308,861 0 0.0%
IRModule.NumIRValueSymbols 17,957 17,957 0 0.0%
LLVM.NumLLVMBytesOutput 6,145,872 6,145,872 0 0.0%
SILModule.NumSILGenFunctions 5,372 5,372 0 0.0%
SILModule.NumSILOptFunctions 7,270 7,270 0 0.0%
Sema.NumConformancesDeserialized 11,810 11,810 0 0.0%
Sema.NumConstraintScopes 38,426 38,426 0 0.0%
Sema.NumDeclsDeserialized 113,362 113,362 0 0.0%
Sema.NumDeclsValidated 9,600 9,600 0 0.0%
Sema.NumFunctionsTypechecked 2,034 2,034 0 0.0%
Sema.NumGenericSignatureBuilders 4,058 4,058 0 0.0%
Sema.NumLazyIterableDeclContexts 18,455 18,455 0 0.0%
Sema.NumTypesDeserialized 44,105 44,105 0 0.0%
Sema.NumTypesValidated 7,546 7,546 0 0.0%

Release

release brief

Regressed (0)
name old new delta delta_pct
Improved (0)
name old new delta delta_pct
Unchanged (delta < 1.0% or delta < 100.0ms) (3)
name old new delta delta_pct
Frontend.NumInstructionsExecuted 142,900,106,137 142,933,087,098 32,980,961 0.02%
LLVM.NumLLVMBytesOutput 7,050,348 7,050,348 0 0.0%
time.swift-driver.wall 16.4s 16.5s 138.9ms 0.85%

release detailed

Regressed (0)
name old new delta delta_pct
Improved (2)
name old new delta delta_pct
Sema.NumLazyGenericEnvironments 6,238 6,070 -168 -2.69% ✅
Sema.NumLazyGenericEnvironmentsLoaded 130 128 -2 -1.54% ✅
Unchanged (delta < 1.0% or delta < 100.0ms) (19)
name old new delta delta_pct
AST.NumLoadedModules 76 76 0 0.0%
AST.NumTotalClangImportedEntities 2,206 2,206 0 0.0%
IRModule.NumIRBasicBlocks 18,890 18,890 0 0.0%
IRModule.NumIRFunctions 10,142 10,142 0 0.0%
IRModule.NumIRGlobals 8,346 8,346 0 0.0%
IRModule.NumIRInsts 209,410 209,410 0 0.0%
IRModule.NumIRValueSymbols 17,668 17,668 0 0.0%
LLVM.NumLLVMBytesOutput 7,050,348 7,050,348 0 0.0%
SILModule.NumSILGenFunctions 4,178 4,178 0 0.0%
SILModule.NumSILOptFunctions 6,240 6,240 0 0.0%
Sema.NumConformancesDeserialized 13,419 13,419 0 0.0%
Sema.NumConstraintScopes 38,252 38,252 0 0.0%
Sema.NumDeclsDeserialized 31,851 31,851 0 0.0%
Sema.NumDeclsValidated 7,720 7,720 0 0.0%
Sema.NumFunctionsTypechecked 2,034 2,034 0 0.0%
Sema.NumGenericSignatureBuilders 1,559 1,559 0 0.0%
Sema.NumLazyIterableDeclContexts 4,273 4,273 0 0.0%
Sema.NumTypesDeserialized 18,335 18,335 0 0.0%
Sema.NumTypesValidated 4,808 4,808 0 0.0%

@jrose-apple
Copy link
Contributor Author

Mm, does that mean that the environments were deserialized eagerly or that they were de-duplicated better? :-/

@jrose-apple jrose-apple changed the title [Serialization] Combine GenericSignatureID and GenericEnvironmentID [Serialization] Drop GenericEnvironmentID for GenericSignatureID Aug 27, 2019
@jrose-apple
Copy link
Contributor Author

@varungandhi-apple The new code makes a lot more sense, check it out.

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 3f5d2fcb72eca1dc66d81ea273d66249abf7c0b3

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 3f5d2fcb72eca1dc66d81ea273d66249abf7c0b3

@jrose-apple
Copy link
Contributor Author

@swift-ci Please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 27514cfd14481d4854d8e72b360488a4496a6e5d

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 27514cfd14481d4854d8e72b360488a4496a6e5d

@jrose-apple jrose-apple force-pushed the a.ham branch 2 times, most recently from 4065988 to 1556c06 Compare August 27, 2019 23:23
@jrose-apple
Copy link
Contributor Author

@swift-ci Please test macOS

Copy link
Contributor

@varungandhi-apple varungandhi-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

A generic environment is always serialized as a GenericSignature with
a lazily-recreated environment, though sometimes it has to include
extra info specifically for generic environments used by SIL. The code
that was doing this claimed a bit for disambiguating between the two,
shrinking the permitted size of a compiled module from 2^31 bits to
2^30. (The code isn't just needlessly complicated; GenericEnvironments
used to be serialized with more information.)

Rather than have two representations for GenericEnvironmentID, this
commit just drops it altogether in favor of referencing
GenericSignatures directly.  This causes a negligible file size
shrinkage for swiftmodules in addition to eliminating the problematic
disambiguation bit.

For now, the Deserialization logic will continue to cache
GenericEnvironments that are used directly by Deserialization, but
really that should probably be done at the AST level. Then we can
simplify further to ModuleFile tracking a plain list of
GenericSignatures.
@jrose-apple
Copy link
Contributor Author

Passed full tests already before the merge conflicts.

@swift-ci Please smoke test

@jrose-apple jrose-apple merged commit 62f947d into swiftlang:master Aug 28, 2019
@jrose-apple jrose-apple deleted the a.ham branch August 28, 2019 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants