Skip to content

[PackageCMO] Use InstructionVisitor to check inst serializability. #76832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 16, 2024

Conversation

elsh
Copy link
Contributor

@elsh elsh commented Oct 2, 2024

Currently, InstructionVisitor is only used during the serialization pass,
that comes after a pass that checks if instructions are serializable. This
causes inconsistencies as some types bypass the first pass but are processed
in the second, leading to assert fails.

This PR uses InstructionVisitor in the first pass to be exhaustive and to
be consistent with the coverage in the second pass. The downside is the visitor
is SILCloner, which introduces overhead of cloning and discarding insts per pass
-- a potential area for future refactoring.

Resolves rdar://130788545.

@elsh elsh requested a review from eeckstein as a code owner October 2, 2024 19:20
@elsh elsh requested review from aschwaighofer, slavapestov and eeckstein and removed request for eeckstein October 2, 2024 19:20
@nkcsgexi nkcsgexi self-requested a review October 2, 2024 22:44
Copy link
Contributor

@eeckstein eeckstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the one hand, this is the right thing to do. On the other hand it means that we are (potentially) cloning the whole SIL in this pass. It's really unfortunate that the visit logic is so much baked into the Cloner that it's not really possible to separate them.

@elsh elsh force-pushed the elsh/pcmo-refactor branch from 88c2f55 to 70c3414 Compare October 10, 2024 22:11
@elsh elsh changed the title [PackageCMO] Use InstructionVisitor to remap types during canSerialize checks. [PackageCMO] Use InstructionVisitor to check inst serializability. Oct 10, 2024
@elsh
Copy link
Contributor Author

elsh commented Oct 10, 2024

@swift-ci smoke test

bool isInstSerializable = true;
// Tracks whether an instruction can be serialized by
// checking its contained types and fields.
llvm::DenseMap<SILInstruction *, bool> instToSerializeMap;
Copy link
Contributor

@aschwaighofer aschwaighofer Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this dense map is needed.

Instead the InstructionVisitor instance should be discarded after/scoped for each SILInstruction that is visited and the isInstSerializable queried.

i.e something like:

for (auto inst : block) {

   InstructionVisitor visitor(...);
   visitor.visit(inst);
   if (!visitor.isInstSerializableSet())
     return false; // can't serialize inst
   ...
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The visitor was created per function instead of per inst to match how it was created in the serialization pass; https://github.com/swiftlang/swift/blame/3d40a8c9f2660294234f08ac47680544f547202d/lib/SILOptimizer/IPO/CrossModuleOptimization.cpp#L733 cc @slavapestov

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a need of this here? There is no state that needs to be carried across one iteration of the loop but instead the bigger scope across multiple iterations forces the use of the map (seems to force, see next comment).

... thinking about this some more. I think you can get rid of the map and leave the scope of the visitor (since we exist early once the isSerializable state changes) as is:

The purpose of this loop is to find the first instruction that has a type that can't be serialized. You start of with isSerializable = true. You are iterating over instructions collecting their types, checking if any of the type you encounter is not serializable. Once you encounter the first one that is not serializable (isSerializable = false) you exist the loop and don't "look at the visitor again". No need for the map and you can leave the Visitor in the scope it currently sits in because we won't look at it again after we found an instruction that does not serialize.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree per-instruction visit won't require a map, but I think the reason for creating visitor per function during the serialization pass was so that unresolved local archetypes can be handled at https://github.com/swiftlang/swift/blame/3d40a8c9f2660294234f08ac47680544f547202d/lib/SILOptimizer/IPO/CrossModuleOptimization.cpp#L742 in recursive calls, and the new pass that checks serializability should also follow the same path. Let me know if that's a wrong assumption.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: removed the map

@elsh
Copy link
Contributor Author

elsh commented Oct 11, 2024

@swift-ci smoke test

1 similar comment
@elsh
Copy link
Contributor Author

elsh commented Oct 13, 2024

@swift-ci smoke test

@elsh elsh force-pushed the elsh/pcmo-refactor branch from 708e2e2 to b1eb89f Compare October 14, 2024 00:24
@elsh
Copy link
Contributor Author

elsh commented Oct 14, 2024

@swift-ci smoke test

@elsh elsh force-pushed the elsh/pcmo-refactor branch from d4d70d2 to 45857fc Compare October 14, 2024 05:19
@elsh
Copy link
Contributor Author

elsh commented Oct 14, 2024

@swift-ci smoke test

@elsh elsh force-pushed the elsh/pcmo-refactor branch from 45857fc to 85c2728 Compare October 14, 2024 07:56
@elsh
Copy link
Contributor Author

elsh commented Oct 14, 2024

@swift-ci smoke test

1 similar comment
@elsh
Copy link
Contributor Author

elsh commented Oct 14, 2024

@swift-ci smoke test

Currently, InstructionVisitor is only used during the serialization pass,
that comes after a pass that checks if instructions are serializable. This
causes inconsistencies as some types bypass the first pass but are processed
in the second, leading to assert fails.

This PR uses InstructionVisitor in the first pass to be exhaustive and to
be consistent with the coverage in the second pass. The downside is the visitor
is SILCloner, which introduces overhead of cloning and discarding insts per pass
-- a potential area for future refactoring.

Resolves rdar://130788545.
@elsh elsh force-pushed the elsh/pcmo-refactor branch from 02dfa01 to f87c5ba Compare October 15, 2024 00:34
@elsh
Copy link
Contributor Author

elsh commented Oct 15, 2024

@swift-ci smoke test

@elsh
Copy link
Contributor Author

elsh commented Oct 15, 2024

@swift-ci smoke test

Copy link
Contributor

@aschwaighofer aschwaighofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@elsh elsh merged commit 6b5df52 into main Oct 16, 2024
3 checks passed
@elsh elsh deleted the elsh/pcmo-refactor branch October 16, 2024 19:00
elsh added a commit that referenced this pull request Oct 18, 2024
shahmishal added a commit that referenced this pull request Oct 18, 2024
Revert "Merge pull request #76832 [PackageCMO] Use InstructionVisitor to check inst serializability."
elsh added a commit that referenced this pull request Oct 18, 2024
elsh added a commit that referenced this pull request Oct 28, 2024
elsh added a commit that referenced this pull request Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants