Skip to content

[SimplifyCFG] Add support for hoisting commutative instructions #104805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 20, 2024
Merged

[SimplifyCFG] Add support for hoisting commutative instructions #104805

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion llvm/lib/Transforms/Utils/SimplifyCFG.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1583,6 +1583,26 @@ static void hoistLockstepIdenticalDbgVariableRecords(
}
}

static bool areIdenticalUpToCommutativity(const Instruction *I1,
const Instruction *I2) {
if (I1->isIdenticalToWhenDefined(I2))
return true;

if (auto *Cmp1 = dyn_cast<CmpInst>(I1))
if (auto *Cmp2 = dyn_cast<CmpInst>(I2))
return Cmp1->getPredicate() == Cmp2->getSwappedPredicate() &&
Cmp1->getOperand(0) == Cmp2->getOperand(1) &&
Cmp1->getOperand(1) == Cmp2->getOperand(0);

if (I1->isCommutative() && I1->isSameOperationAs(I2)) {
return I1->getOperand(0) == I2->getOperand(1) &&
I1->getOperand(1) == I2->getOperand(0) &&
equal(drop_begin(I1->operands(), 2), drop_begin(I2->operands(), 2));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Isn't the drop_begin(..., 2) always empty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For intrinsics, "commutative" means that the first two operands are commutative, but there may be more after that. An example are FMA intrinsics.

}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the operand compares be recursive? I.e (add a, (xor (b, c)) vs (add (xor c, b), a)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be handled by hoisting the two xors first, at which point the add is no longer recursive.


return false;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although unrelated to the complexity canonicalization case, should we also handle select here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean where the select operands are swapped with an inverted icmp? I don't think we can easily handle this here, because we hoist instructions one by one, so the icmp needs to be hoistable independently. It's not impossible, but a lot more complex than just changing the comparison logic.


/// Hoist any common code in the successor blocks up into the block. This
/// function guarantees that BB dominates all successors. If EqTermsOnly is
/// given, only perform hoisting in case both blocks only contain a terminator.
Expand Down Expand Up @@ -1676,7 +1696,7 @@ bool SimplifyCFGOpt::hoistCommonCodeFromSuccessors(BasicBlock *BB,
for (auto &SuccIter : OtherSuccIterRange) {
Instruction *I2 = &*SuccIter;
HasTerminator |= I2->isTerminator();
if (AllInstsAreIdentical && (!I1->isIdenticalToWhenDefined(I2) ||
if (AllInstsAreIdentical && (!areIdenticalUpToCommutativity(I1, I2) ||
MMRAMetadata(*I1) != MMRAMetadata(*I2)))
AllInstsAreIdentical = false;
}
Expand Down
35 changes: 10 additions & 25 deletions llvm/test/Transforms/SimplifyCFG/hoist-common-code.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,16 +159,14 @@ declare void @foo()

define i1 @test_icmp_simple(i1 %c, i32 %a, i32 %b) {
; CHECK-LABEL: @test_icmp_simple(
; CHECK-NEXT: [[CMP1:%.*]] = icmp ult i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: common.ret:
; CHECK-NEXT: [[COMMON_RET_OP:%.*]] = phi i1 [ [[CMP1:%.*]], [[IF]] ], [ [[CMP2:%.*]], [[ELSE]] ]
; CHECK-NEXT: ret i1 [[COMMON_RET_OP]]
; CHECK-NEXT: ret i1 [[CMP1]]
; CHECK: if:
; CHECK-NEXT: [[CMP1]] = icmp ult i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: call void @foo()
; CHECK-NEXT: br label [[COMMON_RET:%.*]]
; CHECK: else:
; CHECK-NEXT: [[CMP2]] = icmp ugt i32 [[B]], [[A]]
; CHECK-NEXT: call void @bar()
; CHECK-NEXT: br label [[COMMON_RET]]
;
Expand All @@ -187,13 +185,8 @@ else:

define void @test_icmp_complex(i1 %c, i32 %a, i32 %b) {
; CHECK-LABEL: @test_icmp_complex(
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: if:
; CHECK-NEXT: [[CMP1:%.*]] = icmp ult i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: br i1 [[CMP1]], label [[IF2:%.*]], label [[ELSE2:%.*]]
; CHECK: else:
; CHECK-NEXT: [[CMP2:%.*]] = icmp ugt i32 [[B]], [[A]]
; CHECK-NEXT: br i1 [[CMP2]], label [[IF2]], label [[ELSE2]]
; CHECK: common.ret:
; CHECK-NEXT: ret void
; CHECK: if2:
Expand Down Expand Up @@ -280,16 +273,14 @@ else:

define i32 @test_binop(i1 %c, i32 %a, i32 %b) {
; CHECK-LABEL: @test_binop(
; CHECK-NEXT: [[OP1:%.*]] = add i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: common.ret:
; CHECK-NEXT: [[COMMON_RET_OP:%.*]] = phi i32 [ [[OP1:%.*]], [[IF]] ], [ [[OP2:%.*]], [[ELSE]] ]
; CHECK-NEXT: ret i32 [[COMMON_RET_OP]]
; CHECK-NEXT: ret i32 [[OP1]]
; CHECK: if:
; CHECK-NEXT: [[OP1]] = add i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: call void @foo()
; CHECK-NEXT: br label [[COMMON_RET:%.*]]
; CHECK: else:
; CHECK-NEXT: [[OP2]] = add i32 [[B]], [[A]]
; CHECK-NEXT: call void @bar()
; CHECK-NEXT: br label [[COMMON_RET]]
;
Expand All @@ -308,16 +299,14 @@ else:

define i32 @test_binop_flags(i1 %c, i32 %a, i32 %b) {
; CHECK-LABEL: @test_binop_flags(
; CHECK-NEXT: [[OP1:%.*]] = add nsw i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: common.ret:
; CHECK-NEXT: [[COMMON_RET_OP:%.*]] = phi i32 [ [[OP1:%.*]], [[IF]] ], [ [[OP2:%.*]], [[ELSE]] ]
; CHECK-NEXT: ret i32 [[COMMON_RET_OP]]
; CHECK-NEXT: ret i32 [[OP1]]
; CHECK: if:
; CHECK-NEXT: [[OP1]] = add nuw nsw i32 [[A:%.*]], [[B:%.*]]
; CHECK-NEXT: call void @foo()
; CHECK-NEXT: br label [[COMMON_RET:%.*]]
; CHECK: else:
; CHECK-NEXT: [[OP2]] = add nsw i32 [[B]], [[A]]
; CHECK-NEXT: call void @bar()
; CHECK-NEXT: br label [[COMMON_RET]]
;
Expand Down Expand Up @@ -392,16 +381,14 @@ else:

define i32 @test_intrin(i1 %c, i32 %a, i32 %b) {
; CHECK-LABEL: @test_intrin(
; CHECK-NEXT: [[OP1:%.*]] = call i32 @llvm.umin.i32(i32 [[A:%.*]], i32 [[B:%.*]])
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: common.ret:
; CHECK-NEXT: [[COMMON_RET_OP:%.*]] = phi i32 [ [[OP1:%.*]], [[IF]] ], [ [[OP2:%.*]], [[ELSE]] ]
; CHECK-NEXT: ret i32 [[COMMON_RET_OP]]
; CHECK-NEXT: ret i32 [[OP1]]
; CHECK: if:
; CHECK-NEXT: [[OP1]] = call i32 @llvm.umin.i32(i32 [[A:%.*]], i32 [[B:%.*]])
; CHECK-NEXT: call void @foo()
; CHECK-NEXT: br label [[COMMON_RET:%.*]]
; CHECK: else:
; CHECK-NEXT: [[OP2]] = call i32 @llvm.umin.i32(i32 [[B]], i32 [[A]])
; CHECK-NEXT: call void @bar()
; CHECK-NEXT: br label [[COMMON_RET]]
;
Expand Down Expand Up @@ -448,16 +435,14 @@ else:

define float @test_intrin_3arg(i1 %c, float %a, float %b, float %d) {
; CHECK-LABEL: @test_intrin_3arg(
; CHECK-NEXT: [[OP1:%.*]] = call float @llvm.fma.f32(float [[A:%.*]], float [[B:%.*]], float [[D:%.*]])
; CHECK-NEXT: br i1 [[C:%.*]], label [[IF:%.*]], label [[ELSE:%.*]]
; CHECK: common.ret:
; CHECK-NEXT: [[COMMON_RET_OP:%.*]] = phi float [ [[OP1:%.*]], [[IF]] ], [ [[OP2:%.*]], [[ELSE]] ]
; CHECK-NEXT: ret float [[COMMON_RET_OP]]
; CHECK-NEXT: ret float [[OP1]]
; CHECK: if:
; CHECK-NEXT: [[OP1]] = call float @llvm.fma.f32(float [[A:%.*]], float [[B:%.*]], float [[D:%.*]])
; CHECK-NEXT: call void @foo()
; CHECK-NEXT: br label [[COMMON_RET:%.*]]
; CHECK: else:
; CHECK-NEXT: [[OP2]] = call float @llvm.fma.f32(float [[B]], float [[A]], float [[D]])
; CHECK-NEXT: call void @bar()
; CHECK-NEXT: br label [[COMMON_RET]]
;
Expand Down
Loading