Skip to content

Commit ea2b77a

Browse files
alexey-bataevsvkeerthy
authored andcommitted
[SLP]Check if the user node has instructions, used only outside
Gather nodes with parents, which scalar instructions are used only outside, are generated before the whole tree vectorization. Need to teach isGatherShuffledSingleRegisterEntry to check that such nodes are emitted first and they cannot depend on other nodes, which are emitted later. Fixes #141628
1 parent a859735 commit ea2b77a

File tree

2 files changed

+532
-0
lines changed

2 files changed

+532
-0
lines changed

llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15317,6 +15317,20 @@ BoUpSLP::isGatherShuffledSingleRegisterEntry(
1531715317
UserPHI ? UserPHI->getIncomingBlock(UseEI.EdgeIdx)->getTerminator()
1531815318
: &getLastInstructionInBundle(UseEI.UserTE);
1531915319
if (TEInsertPt == InsertPt) {
15320+
// Check nodes, which might be emitted first.
15321+
if (TEUseEI.UserTE->State == TreeEntry::Vectorize &&
15322+
(TEUseEI.UserTE->getOpcode() != Instruction::PHI ||
15323+
TEUseEI.UserTE->isAltShuffle()) &&
15324+
all_of(TEUseEI.UserTE->Scalars,
15325+
[](Value *V) { return isUsedOutsideBlock(V); })) {
15326+
if (UseEI.UserTE->State != TreeEntry::Vectorize ||
15327+
(UseEI.UserTE->getOpcode() == Instruction::PHI &&
15328+
!UseEI.UserTE->isAltShuffle()) ||
15329+
any_of(UseEI.UserTE->Scalars,
15330+
[](Value *V) { return !isUsedOutsideBlock(V); }))
15331+
continue;
15332+
}
15333+
1532015334
// If the schedulable insertion point is used in multiple entries - just
1532115335
// exit, no known ordering at this point, available only after real
1532215336
// scheduling.

0 commit comments

Comments
 (0)