Skip to content

Commit d8c2e12

Browse files
committed
[LoopVectorize] In LoopVectorize start using getSymbolicMaxBackedgeTakenCount
LoopVectorizationLegality currently only treats a loop as legal to vectorise if PredicatedScalarEvolution::getBackedgeTakenCount returns a valid SCEV, or more precisely that the loop must have an exact backedge taken count. Therefore, in LoopVectorize.cpp we can safely replace all calls to getBackedgeTakenCount with calls to getSymbolicMaxBackedgeTakenCount, since the result is the same. This also helps prepare the loop vectoriser for PR #88385.
1 parent 587eaef commit d8c2e12

File tree

2 files changed

+15
-5
lines changed

2 files changed

+15
-5
lines changed

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4054,7 +4054,12 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
40544054
unsigned MaxVFtimesIC =
40554055
UserIC ? *MaxPowerOf2RuntimeVF * UserIC : *MaxPowerOf2RuntimeVF;
40564056
ScalarEvolution *SE = PSE.getSE();
4057-
const SCEV *BackedgeTakenCount = PSE.getBackedgeTakenCount();
4057+
// Currently only loops with countable exits are vectorized so it's safe to
4058+
// use getSymbolicMaxBackedgeTakenCount as it should give the same result
4059+
// as getBackedgeTakenCount.
4060+
const SCEV *BackedgeTakenCount = PSE.getSymbolicMaxBackedgeTakenCount();
4061+
assert(!isa<SCEVCouldNotCompute>(BackedgeTakenCount) &&
4062+
"Invalid loop count");
40584063
const SCEV *ExitCount = SE->getAddExpr(
40594064
BackedgeTakenCount, SE->getOne(BackedgeTakenCount->getType()));
40604065
const SCEV *Rem = SE->getURemExpr(

llvm/lib/Transforms/Vectorize/VPlan.cpp

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -880,11 +880,16 @@ VPlanPtr VPlan::createInitialVPlan(Type *InductionTy,
880880
auto Plan = std::make_unique<VPlan>(Entry, VecPreheader);
881881

882882
// Create SCEV and VPValue for the trip count.
883-
const SCEV *BackedgeTakenCount = PSE.getBackedgeTakenCount();
884-
assert(!isa<SCEVCouldNotCompute>(BackedgeTakenCount) && "Invalid loop count");
883+
884+
// Using getSymbolicMaxBackedgeTakenCount instead of getBackedgeTakenCount,
885+
// since they should be identical as we currently only vectorize loops when
886+
// all exits are countable.
887+
const SCEV *BackedgeTakenCountSCEV = PSE.getSymbolicMaxBackedgeTakenCount();
888+
assert(!isa<SCEVCouldNotCompute>(BackedgeTakenCountSCEV) &&
889+
"Invalid loop count");
885890
ScalarEvolution &SE = *PSE.getSE();
886-
const SCEV *TripCount =
887-
SE.getTripCountFromExitCount(BackedgeTakenCount, InductionTy, TheLoop);
891+
const SCEV *TripCount = SE.getTripCountFromExitCount(BackedgeTakenCountSCEV,
892+
InductionTy, TheLoop);
888893
Plan->TripCount =
889894
vputils::getOrCreateVPValueForSCEVExpr(*Plan, TripCount, SE);
890895

0 commit comments

Comments
 (0)