Skip to content

[indvars] Allow widenWithVariantUse to succeed without extend users #71557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[indvars] Allow widenWithVariantUse to succeed without extend users #71557

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 27 additions & 31 deletions llvm/lib/Transforms/Utils/SimplifyIndVar.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1611,37 +1611,33 @@ bool WidenIV::widenWithVariantUse(WidenIV::NarrowIVDefUse DU) {
return false;
ExtUsers.push_back(User);
}
if (ExtUsers.empty()) {
DeadInsts.emplace_back(NarrowUse);
return true;
}

// We'll prove some facts that should be true in the context of ext users. If
// there is no users, we are done now. If there are some, pick their common
// dominator as context.
const Instruction *CtxI = findCommonDominator(ExtUsers, *DT);

if (!CanSignExtend && !CanZeroExtend) {
// Because InstCombine turns 'sub nuw' to 'add' losing the no-wrap flag, we
// will most likely not see it. Let's try to prove it.
if (OpCode != Instruction::Add)
return false;
if (ExtKind != ExtendKind::Zero)
return false;
const SCEV *LHS = SE->getSCEV(OBO->getOperand(0));
const SCEV *RHS = SE->getSCEV(OBO->getOperand(1));
// TODO: Support case for NarrowDef = NarrowUse->getOperand(1).
if (NarrowUse->getOperand(0) != NarrowDef)
return false;
if (!SE->isKnownNegative(RHS))
return false;
bool ProvedSubNUW = SE->isKnownPredicateAt(ICmpInst::ICMP_UGE, LHS,
SE->getNegativeSCEV(RHS), CtxI);
if (!ProvedSubNUW)
return false;
// In fact, our 'add' is 'sub nuw'. We will need to widen the 2nd operand as
// neg(zext(neg(op))), which is basically sext(op).
AnotherOpExtKind = ExtendKind::Sign;
if (!ExtUsers.empty()) {
// We'll prove some facts that should be true in the context of ext users.
// Pick their common dominator as context.
const Instruction *CtxI = findCommonDominator(ExtUsers, *DT);

if (!CanSignExtend && !CanZeroExtend) {
// Because InstCombine turns 'sub nuw' to 'add' losing the no-wrap flag, we
// will most likely not see it. Let's try to prove it.
if (OpCode != Instruction::Add)
return false;
if (ExtKind != ExtendKind::Zero)
return false;
const SCEV *LHS = SE->getSCEV(OBO->getOperand(0));
const SCEV *RHS = SE->getSCEV(OBO->getOperand(1));
// TODO: Support case for NarrowDef = NarrowUse->getOperand(1).
if (NarrowUse->getOperand(0) != NarrowDef)
return false;
if (!SE->isKnownNegative(RHS))
return false;
bool ProvedSubNUW = SE->isKnownPredicateAt(ICmpInst::ICMP_UGE, LHS,
SE->getNegativeSCEV(RHS), CtxI);
if (!ProvedSubNUW)
return false;
// In fact, our 'add' is 'sub nuw'. We will need to widen the 2nd operand as
// neg(zext(neg(op))), which is basically sext(op).
AnotherOpExtKind = ExtendKind::Sign;
}
}

// Verifying that Defining operand is an AddRec
Expand Down
5 changes: 3 additions & 2 deletions llvm/test/Transforms/IndVarSimplify/X86/pr57187.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@ define void @test(i32 %start) {
; CHECK-NEXT: br label [[LOOP]]
; CHECK: loop:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[BACKEDGE:%.*]] ], [ [[TMP1]], [[ENTRY:%.*]] ]
; CHECK-NEXT: [[TMP2:%.*]] = add nuw nsw i64 [[INDVARS_IV]], 4294967295
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: [[INDVARS:%.*]] = trunc i64 [[INDVARS_IV_NEXT]] to i32
; CHECK-NEXT: [[LOOP_EXIT_COND:%.*]] = icmp slt i32 [[TMP0]], 11
; CHECK-NEXT: br i1 [[LOOP_EXIT_COND]], label [[EXIT:%.*]], label [[STUCK_PREHEADER:%.*]]
; CHECK: stuck.preheader:
; CHECK-NEXT: br label [[STUCK:%.*]]
; CHECK: exit:
; CHECK-NEXT: [[IV_NEXT_LCSSA:%.*]] = phi i32 [ [[INDVARS]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT_LCSSA_WIDE:%.*]] = phi i64 [ [[TMP2]], [[LOOP]] ]
; CHECK-NEXT: [[TMP3:%.*]] = trunc i64 [[IV_NEXT_LCSSA_WIDE]] to i32
; CHECK-NEXT: ret void
; CHECK: stuck:
; CHECK-NEXT: br i1 false, label [[BACKEDGE]], label [[STUCK]]
Expand Down
21 changes: 9 additions & 12 deletions llvm/test/Transforms/IndVarSimplify/widen-nonnegative-countdown.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,12 @@ define void @zext_postinc(ptr %A, i32 %start) {
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[TMP0]], [[FOR_BODY_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[J_016_US:%.*]] = phi i32 [ [[INC_US:%.*]], [[FOR_BODY]] ], [ [[START]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[ARRAYIDX_US:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
; CHECK-NEXT: tail call void @use_ptr(ptr [[ARRAYIDX_US]])
; CHECK-NEXT: [[INC_US]] = add nsw i32 [[J_016_US]], -1
; CHECK-NEXT: [[CMP2_US:%.*]] = icmp ugt i32 [[INC_US]], 6
; CHECK-NEXT: [[TMP1:%.*]] = add nsw i64 [[INDVARS_IV]], 4294967295
; CHECK-NEXT: [[CMP2_US_WIDE:%.*]] = icmp ugt i64 [[TMP1]], 6
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: br i1 [[CMP2_US]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK-NEXT: br i1 [[CMP2_US_WIDE]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK: exit.loopexit:
; CHECK-NEXT: br label [[EXIT]]
; CHECK: exit:
Expand Down Expand Up @@ -365,16 +364,15 @@ define void @zext_postinc_offset_constant_one(ptr %A, i32 %start) {
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[TMP0]], [[FOR_BODY_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[J_016_US:%.*]] = phi i32 [ [[INC_US:%.*]], [[FOR_BODY]] ], [ [[START]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[TMP1:%.*]] = trunc i64 [[INDVARS_IV]] to i32
; CHECK-NEXT: [[ADD_US:%.*]] = add i32 [[TMP1]], 1
; CHECK-NEXT: [[IDXPROM_US:%.*]] = zext i32 [[ADD_US]] to i64
; CHECK-NEXT: [[ARRAYIDX_US:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[IDXPROM_US]]
; CHECK-NEXT: tail call void @use_ptr(ptr [[ARRAYIDX_US]])
; CHECK-NEXT: [[INC_US]] = add nsw i32 [[J_016_US]], -1
; CHECK-NEXT: [[CMP2_US:%.*]] = icmp ugt i32 [[INC_US]], 6
; CHECK-NEXT: [[TMP2:%.*]] = add nsw i64 [[INDVARS_IV]], 4294967295
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really get why this transform is correct. It looks like indvars.iv here is a normal {%start,+,-1} addrec, but then the exit condition here becomes {%start+4294967295,+,-1} ugt 6? That's not the same as {%start-1,+,-1} ugt 6 in a smaller bit width.

Am I missing something here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key fact is that we have a loop entry condition which proves both that start is non-negative, and that start -1 is still positive. Given this precondition trunc(zext(start)+4294967295) == start-1. Given we know we exit without the IV every becoming negative (in either 32 or 64), this should be the same.

In general, this transform relies on the high bits of the addrec being don't care except for the identified users it visits and proves that the result of the user is unchanged by the possible change in the high bits.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, hold on. You're right and there's something funky going on here. We have a 64 bit IV which enumerates the same values as the 32 bit IV, but then the actual comparison uses a 64 bit offset (without truncation), and the high bits change the exit condition.

I need to investigate this further, thanks for spotting.

; CHECK-NEXT: [[CMP2_US_WIDE:%.*]] = icmp ugt i64 [[TMP2]], 6
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: br i1 [[CMP2_US]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK-NEXT: br i1 [[CMP2_US_WIDE]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK: exit.loopexit:
; CHECK-NEXT: br label [[EXIT]]
; CHECK: exit:
Expand Down Expand Up @@ -664,14 +662,13 @@ define void @zext_postinc_offset_constant_minus_one(ptr %A, i32 %start) {
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[TMP0]], [[FOR_BODY_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[J_016_US:%.*]] = phi i32 [ [[INC_US:%.*]], [[FOR_BODY]] ], [ [[START]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[TMP1:%.*]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: [[ARRAYIDX_US:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[TMP1]]
; CHECK-NEXT: tail call void @use_ptr(ptr [[ARRAYIDX_US]])
; CHECK-NEXT: [[INC_US]] = add nsw i32 [[J_016_US]], -1
; CHECK-NEXT: [[CMP2_US:%.*]] = icmp ugt i32 [[INC_US]], 6
; CHECK-NEXT: [[TMP2:%.*]] = add nsw i64 [[INDVARS_IV]], 4294967295
; CHECK-NEXT: [[CMP2_US_WIDE:%.*]] = icmp ugt i64 [[TMP2]], 6
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: br i1 [[CMP2_US]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK-NEXT: br i1 [[CMP2_US_WIDE]], label [[FOR_BODY]], label [[EXIT_LOOPEXIT:%.*]]
; CHECK: exit.loopexit:
; CHECK-NEXT: br label [[EXIT]]
; CHECK: exit:
Expand Down
10 changes: 6 additions & 4 deletions llvm/test/Transforms/LoopFlatten/widen-iv3.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@ define i16 @foo() {
; CHECK-NEXT: [[J_011:%.*]] = phi i16 [ 0, [[FOR_COND1_PREHEADER]] ], [ [[INC:%.*]], [[FOR_BODY4]] ]
; CHECK-NEXT: [[SUM_110:%.*]] = phi i16 [ [[SUM_012]], [[FOR_COND1_PREHEADER]] ], [ [[ADD5]], [[FOR_BODY4]] ]
; CHECK-NEXT: [[TMP2:%.*]] = add nuw nsw i32 [[INDVAR]], [[TMP0]]
; CHECK-NEXT: [[TMP3:%.*]] = sext i16 [[J_011]] to i32
; CHECK-NEXT: [[TMP4:%.*]] = add nuw nsw i32 [[TMP3]], [[TMP0]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These instructions look dead. Is that the fault of LoopFlatten or IndVars?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopFlatten. It doesn't call up after calling the widening routines. I added adce to at least one of the LoopFlatten test previously, can do so for this one as well. (They're already invoking multiple passes.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, LoopFlatten ignores the DeadInsts parameter. It would be best to make it drop the dead insts directly instead of adding adce to tests.

; CHECK-NEXT: [[ADD:%.*]] = add nuw nsw i16 [[J_011]], [[MUL]]
; CHECK-NEXT: [[TMP3:%.*]] = trunc i32 [[TMP2]] to i16
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds [64 x i16], ptr @v, i16 0, i16 [[TMP3]]
; CHECK-NEXT: [[TMP4:%.*]] = load i16, ptr [[ARRAYIDX]], align 1
; CHECK-NEXT: [[ADD5]] = add nsw i16 [[TMP4]], [[SUM_110]]
; CHECK-NEXT: [[TMP5:%.*]] = trunc i32 [[TMP2]] to i16
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds [64 x i16], ptr @v, i16 0, i16 [[TMP5]]
; CHECK-NEXT: [[TMP6:%.*]] = load i16, ptr [[ARRAYIDX]], align 1
; CHECK-NEXT: [[ADD5]] = add nsw i16 [[TMP6]], [[SUM_110]]
; CHECK-NEXT: [[INDVAR_NEXT]] = add i32 [[INDVAR]], 1
; CHECK-NEXT: [[INC]] = add nuw nsw i16 [[J_011]], 1
; CHECK-NEXT: [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INDVAR_NEXT]], 16
Expand Down