-
Notifications
You must be signed in to change notification settings - Fork 14.2k
[SimplifyCFG] Replace unreachable switch lookup table holes with poison #94990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@llvm/pr-subscribers-llvm-transforms Author: DaPorkchop_ (DaMatrix) ChangesAs discussed in #94468, this causes switch lookup table entries which are unreachable to be poison instead of filling them with a value from one of the reachable cases. Full diff: https://github.com/llvm/llvm-project/pull/94990.diff 3 Files Affected:
diff --git a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
index 292739b6c5fda..50ef68ec02cc6 100644
--- a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
+++ b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
@@ -6266,7 +6266,8 @@ SwitchLookupTable::SwitchLookupTable(
assert(Values.size() && "Can't build lookup table without values!");
assert(TableSize >= Values.size() && "Can't fit values in table!");
- // If all values in the table are equal, this is that value.
+ // If all values in the table are equal, ignoring any values that are poison,
+ // this is that value.
SingleValue = Values.begin()->second;
Type *ValueType = Values.begin()->second->getType();
@@ -6281,8 +6282,15 @@ SwitchLookupTable::SwitchLookupTable(
uint64_t Idx = (CaseVal->getValue() - Offset->getValue()).getLimitedValue();
TableContents[Idx] = CaseRes;
- if (CaseRes != SingleValue)
- SingleValue = nullptr;
+ if (CaseRes != SingleValue) {
+ if (SingleValue && isa<PoisonValue>(SingleValue)) {
+ // All of the switch cases until now have returned poison; ignore them
+ // and use this value as the single constant value.
+ SingleValue = CaseRes;
+ } else if (!isa<PoisonValue>(CaseRes)) {
+ SingleValue = nullptr;
+ }
+ }
}
// Fill in any holes in the table with the default result.
@@ -6295,7 +6303,10 @@ SwitchLookupTable::SwitchLookupTable(
TableContents[I] = DefaultValue;
}
- if (DefaultValue != SingleValue)
+ // If the default value is poison, all the holes are poison.
+ bool DefaultValueIsPoison = isa<PoisonValue>(DefaultValue);
+
+ if (DefaultValue != SingleValue && !DefaultValueIsPoison)
SingleValue = nullptr;
}
@@ -6322,6 +6333,9 @@ SwitchLookupTable::SwitchLookupTable(
if (!ConstVal) {
// This is an undef. We could deal with it, but undefs in lookup tables
// are very seldom. It's probably not worth the additional complexity.
+ // TODO: In switches with holes and an unreachable default branch, this
+ // will actually occur every time, as the holes will be filled
+ // with poison.
LinearMappingPossible = false;
break;
}
@@ -6752,8 +6766,8 @@ static bool SwitchToLookupTable(SwitchInst *SI, IRBuilder<> &Builder,
// If the table has holes but the default destination doesn't produce any
// constant results, the lookup table entries corresponding to the holes will
- // contain undefined values.
- bool AllHolesAreUndefined = TableHasHoles && !HasDefaultResults;
+ // contain poison.
+ bool AllHolesArePoison = TableHasHoles && !HasDefaultResults;
// If the default destination doesn't produce a constant result but is still
// reachable, and the lookup table has holes, we need to use a mask to
@@ -6761,7 +6775,7 @@ static bool SwitchToLookupTable(SwitchInst *SI, IRBuilder<> &Builder,
// to the default case.
// The mask is unnecessary if the table has holes but the default destination
// is unreachable, as in that case the holes must also be unreachable.
- bool NeedMask = AllHolesAreUndefined && DefaultIsReachable;
+ bool NeedMask = AllHolesArePoison && DefaultIsReachable;
if (NeedMask) {
// As an extra penalty for the validity test we require more cases.
if (SI->getNumCases() < 4) // FIXME: Find best threshold value (benchmark).
@@ -6906,9 +6920,11 @@ static bool SwitchToLookupTable(SwitchInst *SI, IRBuilder<> &Builder,
for (PHINode *PHI : PHIs) {
const ResultListTy &ResultList = ResultLists[PHI];
+ Type *ResultType = ResultList.begin()->second->getType();
+
// Use any value to fill the lookup table holes.
Constant *DV =
- AllHolesAreUndefined ? ResultLists[PHI][0].second : DefaultResults[PHI];
+ AllHolesArePoison ? PoisonValue::get(ResultType) : DefaultResults[PHI];
StringRef FuncName = Fn->getName();
SwitchLookupTable Table(Mod, TableSize, TableIndexOffset, ResultList, DV,
DL, FuncName);
diff --git a/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table.ll b/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table.ll
index 845c5008e3837..345476481f570 100644
--- a/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table.ll
+++ b/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table.ll
@@ -34,11 +34,12 @@ target triple = "x86_64-unknown-linux-gnu"
; CHECK: @switch.table.unreachable_case = private unnamed_addr constant [9 x i32] [i32 0, i32 0, i32 0, i32 2, i32 -1, i32 1, i32 1, i32 1, i32 1], align 4
; CHECK: @switch.table.unreachable_default = private unnamed_addr constant [4 x i32] [i32 42, i32 52, i32 1, i32 2], align 4
; CHECK: @switch.table.nodefaultnoholes = private unnamed_addr constant [4 x i32] [i32 55, i32 123, i32 0, i32 -1], align 4
-; CHECK: @switch.table.nodefaultwithholes = private unnamed_addr constant [6 x i32] [i32 55, i32 123, i32 0, i32 -1, i32 55, i32 -1], align 4
+; CHECK: @switch.table.nodefaultwithholes = private unnamed_addr constant [6 x i32] [i32 55, i32 123, i32 0, i32 -1, i32 poison, i32 -1], align 4
; CHECK: @switch.table.threecases = private unnamed_addr constant [3 x i32] [i32 10, i32 7, i32 5], align 4
-; CHECK: @switch.table.covered_switch_with_bit_tests = private unnamed_addr constant [8 x i32] [i32 2, i32 2, i32 2, i32 2, i32 2, i32 2, i32 1, i32 1], align 4
+; CHECK: @switch.table.covered_switch_with_bit_tests = private unnamed_addr constant [8 x i32] [i32 2, i32 2, i32 poison, i32 poison, i32 poison, i32 poison, i32 1, i32 1], align 4
; CHECK: @switch.table.signed_overflow1 = private unnamed_addr constant [4 x i32] [i32 3333, i32 4444, i32 1111, i32 2222], align 4
-; CHECK: @switch.table.signed_overflow2 = private unnamed_addr constant [4 x i32] [i32 3333, i32 4444, i32 2222, i32 2222], align 4
+; CHECK: @switch.table.signed_overflow2 = private unnamed_addr constant [4 x i32] [i32 3333, i32 4444, i32 poison, i32 2222], align 4
+; CHECK: @switch.table.linearmap_hole_unreachable_default = private unnamed_addr constant [5 x i32] [i32 1, i32 poison, i32 5, i32 7, i32 9], align 4
;.
define i32 @f(i32 %c) {
; CHECK-LABEL: @f(
@@ -2151,3 +2152,130 @@ return: ; preds = %sw.default, %entry,
%retval.0 = phi { i8, i8 } [ undef, %entry ], [ undef, %entry ], [ undef, %entry ], [ %1, %sw.default ]
ret { i8, i8 } %retval.0
}
+
+; The switch has a hole which falls through to an unreachable default case, but it can still be optimized into a constant load because
+; the poison value used for the hole is ignored.
+define i32 @constant_hole_unreachable_default(i32 %x) {
+; CHECK-LABEL: @constant_hole_unreachable_default(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: ret i32 1
+;
+entry:
+ switch i32 %x, label %sw.default [
+ i32 0, label %bb0
+ i32 2, label %bb0
+ i32 3, label %bb0
+ i32 4, label %bb0
+ ]
+
+sw.default: unreachable
+bb0: br label %return
+
+return:
+ %res = phi i32 [ 1, %bb0 ]
+ ret i32 %res
+}
+
+; The switch has a hole which falls through to an unreachable default case and the first case explicitly returns poison, but it can still
+; be optimized into a constant load because the poison values are ignored.
+define i32 @constant_hole_unreachable_default_firstpoison(i32 %x) {
+; CHECK-LABEL: @constant_hole_unreachable_default_firstpoison(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: ret i32 1
+;
+entry:
+ switch i32 %x, label %sw.default [
+ i32 0, label %bb.poison
+ i32 2, label %bb0
+ i32 3, label %bb0
+ i32 4, label %bb0
+ ]
+
+sw.default: unreachable
+bb.poison: br label %return
+bb0: br label %return
+
+return:
+ %res = phi i32 [ poison, %bb.poison ], [ 1, %bb0 ]
+ ret i32 %res
+}
+
+; The switch has a hole which falls through to an unreachable default case and the first case explicitly returns poison, but it can still
+; be optimized into a constant load because the poison values are ignored.
+define i32 @constant_hole_unreachable_default_lastpoison(i32 %x) {
+; CHECK-LABEL: @constant_hole_unreachable_default_lastpoison(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: ret i32 1
+;
+entry:
+ switch i32 %x, label %sw.default [
+ i32 0, label %bb0
+ i32 2, label %bb0
+ i32 3, label %bb0
+ i32 4, label %bb.poison
+ ]
+
+sw.default: unreachable
+bb.poison: br label %return
+bb0: br label %return
+
+return:
+ %res = phi i32 [ poison, %bb.poison ], [ 1, %bb0 ]
+ ret i32 %res
+}
+
+; The switch has a hole which falls through to an unreachable default case, which prevents it from being optimized into a linear mapping 2*x+1.
+; TODO: We should add support for this, at least in certain cases.
+define i32 @linearmap_hole_unreachable_default(i32 %x) {
+; CHECK-LABEL: @linearmap_hole_unreachable_default(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[SWITCH_GEP:%.*]] = getelementptr inbounds [5 x i32], ptr @switch.table.linearmap_hole_unreachable_default, i32 0, i32 [[X:%.*]]
+; CHECK-NEXT: [[SWITCH_LOAD:%.*]] = load i32, ptr [[SWITCH_GEP]], align 4
+; CHECK-NEXT: ret i32 [[SWITCH_LOAD]]
+;
+entry:
+ switch i32 %x, label %sw.default [
+ i32 0, label %bb0
+ i32 2, label %bb2
+ i32 3, label %bb3
+ i32 4, label %bb4
+ ]
+
+sw.default: unreachable
+bb0: br label %return
+bb2: br label %return
+bb3: br label %return
+bb4: br label %return
+
+return:
+ %res = phi i32 [ 1, %bb0 ], [ 5, %bb2 ], [ 7, %bb3 ], [ 9, %bb4 ]
+ ret i32 %res
+}
+
+; The switch has a hole which falls through to an unreachable default case, but it can still be optimized into a bitmask extraction because
+; the poison value used for the hole is simply replaced with zero.
+define i1 @bitset_hole_unreachable_default(i32 %x) {
+; CHECK-LABEL: @bitset_hole_unreachable_default(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[SWITCH_CAST:%.*]] = trunc i32 [[X:%.*]] to i5
+; CHECK-NEXT: [[SWITCH_SHIFTAMT:%.*]] = mul nuw nsw i5 [[SWITCH_CAST]], 1
+; CHECK-NEXT: [[SWITCH_DOWNSHIFT:%.*]] = lshr i5 8, [[SWITCH_SHIFTAMT]]
+; CHECK-NEXT: [[SWITCH_MASKED:%.*]] = trunc i5 [[SWITCH_DOWNSHIFT]] to i1
+; CHECK-NEXT: ret i1 [[SWITCH_MASKED]]
+;
+entry:
+ switch i32 %x, label %sw.default [
+ i32 0, label %bb0
+ i32 2, label %bb0
+ i32 3, label %bb1
+ i32 4, label %bb0
+ ]
+
+sw.default: unreachable
+bb0: br label %return
+bb1: br label %return
+
+return:
+ %res = phi i1 [ 0, %bb0 ], [ 1, %bb1 ]
+ ret i1 %res
+}
diff --git a/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table_big.ll b/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table_big.ll
index 7988e3057a2c2..4ebf09ae3b127 100644
--- a/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table_big.ll
+++ b/llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table_big.ll
@@ -7,11 +7,11 @@ target triple = "i386-pc-linux-gnu"
;.
; CHECK: @switch.table.reachable_default_dense_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1], align 4
; CHECK: @switch.table.unreachable_default_dense_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1], align 4
-; CHECK: @switch.table.reachable_default_holes_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 0, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 0, i32 2, i32 1, i32 0, i32 7, i32 0, i32 5, i32 4, i32 3, i32 2, i32 0, i32 0, i32 7, i32 6, i32 5, i32 0, i32 3, i32 2, i32 1], align 4
-; CHECK: @switch.table.unreachable_default_holes_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 0, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 0, i32 2, i32 1, i32 0, i32 7, i32 0, i32 5, i32 4, i32 3, i32 2, i32 0, i32 0, i32 7, i32 6, i32 5, i32 0, i32 3, i32 2, i32 1], align 4
+; CHECK: @switch.table.reachable_default_holes_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 poison, i32 4, i32 3, i32 2, i32 1, i32 poison, i32 7, i32 6, i32 5, i32 4, i32 poison, i32 2, i32 1, i32 0, i32 7, i32 poison, i32 5, i32 4, i32 3, i32 2, i32 poison, i32 0, i32 7, i32 6, i32 5, i32 poison, i32 3, i32 2, i32 1], align 4
+; CHECK: @switch.table.unreachable_default_holes_0to31 = private unnamed_addr constant [32 x i32] [i32 0, i32 7, i32 6, i32 poison, i32 4, i32 3, i32 2, i32 1, i32 poison, i32 7, i32 6, i32 5, i32 4, i32 poison, i32 2, i32 1, i32 0, i32 7, i32 poison, i32 5, i32 4, i32 3, i32 2, i32 poison, i32 0, i32 7, i32 6, i32 5, i32 poison, i32 3, i32 2, i32 1], align 4
; CHECK: @switch.table.reachable_default_dense_0to32 = private unnamed_addr constant [33 x i32] [i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0], align 4
; CHECK: @switch.table.unreachable_default_dense_0to32 = private unnamed_addr constant [33 x i32] [i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0], align 4
-; CHECK: @switch.table.unreachable_default_holes_0to32 = private unnamed_addr constant [33 x i32] [i32 0, i32 7, i32 6, i32 0, i32 4, i32 3, i32 2, i32 1, i32 0, i32 7, i32 6, i32 5, i32 4, i32 0, i32 2, i32 1, i32 0, i32 7, i32 0, i32 5, i32 4, i32 3, i32 2, i32 0, i32 0, i32 7, i32 6, i32 5, i32 0, i32 3, i32 2, i32 1, i32 0], align 4
+; CHECK: @switch.table.unreachable_default_holes_0to32 = private unnamed_addr constant [33 x i32] [i32 0, i32 7, i32 6, i32 poison, i32 4, i32 3, i32 2, i32 1, i32 poison, i32 7, i32 6, i32 5, i32 4, i32 poison, i32 2, i32 1, i32 0, i32 7, i32 poison, i32 5, i32 4, i32 3, i32 2, i32 poison, i32 0, i32 7, i32 6, i32 5, i32 poison, i32 3, i32 2, i32 1, i32 0], align 4
;.
define i32 @reachable_default_dense_0to31(i32 %x, i32 %y) {
; CHECK-LABEL: @reachable_default_dense_0to31(
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll review this PR again over the weekend.
@@ -6322,6 +6333,9 @@ SwitchLookupTable::SwitchLookupTable( | |||
if (!ConstVal) { | |||
// This is an undef. We could deal with it, but undefs in lookup tables | |||
// are very seldom. It's probably not worth the additional complexity. | |||
// TODO: In switches with holes and an unreachable default branch, this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you want to finish this TODO in this PR? If not, could you change the poison values back to the first value? I'm concerned this might cause some regressions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't planning on fixing it in this PR, since it seemed like too big of a change. I've got another PR lined up which fixes this TODO, which I was planning to open as soon as this gets merged. Would you prefer I made that part of this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's fine with me, but I hope you can restore the unhandled cases to their previous values in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've changed this as well. Is the new version what you had in mind?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can set it to the first value here:
TableContents[Idx] = CaseRes; |
undef
/poison
, so this might not cause significant regressions. Others might have different opinions on this, so I'll request comments from them after you address the poison
and undef
mixing issue.
95f0f20
to
e30a331
Compare
e30a331
to
d6098d6
Compare
Rebasing to a much more recent commit to hopefully prevent GitHub Actions from timing out. |
d6098d6
to
4858b9c
Compare
You can test this locally with the following command:git diff -U0 --pickaxe-regex -S '([^a-zA-Z0-9#_-]undef[^a-zA-Z0-9_-]|UndefValue::get)' f8d270474c14c6705c77971494505dbe4b6d55ae 659c67692818151c2f7796ce52325aef7fd6cdef llvm/lib/Transforms/Utils/SimplifyCFG.cpp llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table.ll llvm/test/Transforms/SimplifyCFG/X86/switch_to_lookup_table_big.ll The following files introduce new uses of undef:
Undef is now deprecated and should only be used in the rare cases where no replacement is possible. For example, a load of uninitialized memory yields In tests, avoid using For example, this is considered a bad practice: define void @fn() {
...
br i1 undef, ...
} Please use the following instead: define void @fn(i1 %cond) {
...
br i1 %cond, ...
} Please refer to the Undefined Behavior Manual for more information. |
Oh... that's annoying, how am I supposed to differentiate |
I think you can only check if it is a poison. Please keep these test cases to prevent mis-compile. |
Okay, so I should remove all the logic for handling combinations of |
4858b9c
to
f9d54a5
Compare
Okay, this new commit will still fail that check as there are (necessarily) new occurrences of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. If no one else comments, I'll merge it next Tuesday.
; CHECK: @switch.table.signed_overflow2 = private unnamed_addr constant [4 x i32] [i32 3333, i32 4444, i32 2222, i32 2222], align 4 | ||
; CHECK: @switch.table.signed_overflow2 = private unnamed_addr constant [4 x i32] [i32 3333, i32 4444, i32 poison, i32 2222], align 4 | ||
; CHECK: @switch.table.constant_hole_unreachable_default_firstundef = private unnamed_addr constant [5 x i32] [i32 undef, i32 poison, i32 1, i32 1, i32 1], align 4 | ||
; CHECK: @switch.table.constant_hole_unreachable_default_lastundef = private unnamed_addr constant [5 x i32] [i32 1, i32 poison, i32 1, i32 1, i32 undef], align 4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dtcxzyw I don't know what impact this has on real-world code. Do you think it's worth running llvm-opt-benchmark?
✅ With the latest revision this PR passed the C/C++ code formatter. |
Co-authored-by: DianQK <[email protected]>
65dd653
to
659c676
Compare
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/17/builds/4896 Here is the relevant piece of the build log for the reference
|
@nikic @dtcxzyw @RalfJung |
Local branch amd-gfx 0756140 Merged main:5712e293fb31 into amd-gfx:5f712560902a Remote branch main cea738b [SimplifyCFG] Replace unreachable switch lookup table holes with poison (llvm#94990) Change-Id: If1d143783b7f74b6d146934b0cf9fc3fe20ac6b4
As discussed in #94468, this causes switch lookup table entries which are unreachable to be poison instead of filling them with a value from one of the reachable cases.