Skip to content

Commit cb648ba

Browse files
committed
[SLP]Check if the user node has instructions, used only outside
Gather nodes with parents, which scalar instructions are used only outside, are generated before the whole tree vectorization. Need to teach isGatherShuffledSingleRegisterEntry to check that such nodes are emitted first and they cannot depend on other nodes, which are emitted later. Fixes #141628
1 parent 8a49db3 commit cb648ba

File tree

2 files changed

+532
-0
lines changed

2 files changed

+532
-0
lines changed

llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15317,6 +15317,20 @@ BoUpSLP::isGatherShuffledSingleRegisterEntry(
1531715317
UserPHI ? UserPHI->getIncomingBlock(UseEI.EdgeIdx)->getTerminator()
1531815318
: &getLastInstructionInBundle(UseEI.UserTE);
1531915319
if (TEInsertPt == InsertPt) {
15320+
// Check nodes, which might be emitted first.
15321+
if (TEUseEI.UserTE->State == TreeEntry::Vectorize &&
15322+
(TEUseEI.UserTE->getOpcode() != Instruction::PHI ||
15323+
TEUseEI.UserTE->isAltShuffle()) &&
15324+
all_of(TEUseEI.UserTE->Scalars,
15325+
[](Value *V) { return isUsedOutsideBlock(V); })) {
15326+
if (UseEI.UserTE->State != TreeEntry::Vectorize ||
15327+
(UseEI.UserTE->getOpcode() == Instruction::PHI &&
15328+
!UseEI.UserTE->isAltShuffle()) ||
15329+
any_of(UseEI.UserTE->Scalars,
15330+
[](Value *V) { return !isUsedOutsideBlock(V); }))
15331+
continue;
15332+
}
15333+
1532015334
// If the schedulable insertion point is used in multiple entries - just
1532115335
// exit, no known ordering at this point, available only after real
1532215336
// scheduling.

0 commit comments

Comments
 (0)