Add minimum count threshold for indirect call promotion #145282
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a new command-line option -icp-minimum-count-threshold to prevent promotion of indirect call candidates with very low absolute counts, even if they meet the percentage-based profitability thresholds. Currently, candidates with counts as low as 1-2 calls can be promoted if they represent a significant percentage of the call site. This can lead to unprofitable micro-optimizations that increase code size without meaningful performance benefits. The new threshold defaults to 0 to maintain backward compatibility but can be set to values like 5-20 for production use to filter out statistically insignificant candidates. Added comprehensive test coverage for the new threshold including negative tests (blocking low-count promotions), positive tests (allowing high-count promotions), and edge cases (threshold exactly at count value).
llvmbot
added
PGO
|
@llvm/pr-subscribers-pgo @llvm/pr-subscribers-llvm-transforms Author: Snehasish Kumar (snehasish) ChangesAllow users to set the minimum absolute count for indirect call promotion. This is primarily meant to be control indirect call promotion for synthetic vp metadata introduced in #141164 for use by MemProf. Full diff: https://github.com/llvm/llvm-project/pull/145282.diff 2 Files Affected:
diff --git a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
index 90065e053b629..861dc0d364b84 100644
--- a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
+++ b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
@@ -37,6 +37,12 @@ static cl::opt<unsigned>
cl::desc("The percentage threshold against total "
"count for the promotion"));
+// Set the minimum absolute count threshold for indirect call promotion.
+// Candidates with counts below this threshold will not be promoted.
+static cl::opt<unsigned> ICPMinimumCountThreshold(
+ "icp-minimum-count-threshold", cl::init(0), cl::Hidden,
+ cl::desc("Minimum absolute count for promotion candidate"));
+
// Set the maximum number of targets to promote for a single indirect-call
// callsite.
static cl::opt<unsigned>
@@ -51,7 +57,8 @@ cl::opt<unsigned> MaxNumVTableAnnotations(
bool ICallPromotionAnalysis::isPromotionProfitable(uint64_t Count,
uint64_t TotalCount,
uint64_t RemainingCount) {
- return Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
+ return Count >= ICPMinimumCountThreshold &&
+ Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
Count * 100 >= ICPTotalPercentThreshold * TotalCount;
}
diff --git a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
index 3b0e3250cd6df..7f1fea9d1386a 100644
--- a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
+++ b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
@@ -1,6 +1,12 @@
; RUN: opt < %s -passes=pgo-icall-prom -S -icp-total-percent-threshold=50 | FileCheck %s --check-prefix=ICALL-PROM
; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS-REMARK
; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=20 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS2-REMARK
+; Test minimum count threshold - should prevent func1 promotion (count 10 < threshold 15)
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-BLOCK
+; Test minimum count threshold - should allow func4 promotion (count 1030 > threshold 15)
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-ALLOW
+; Test edge case - threshold exactly at count value
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=10 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-EDGE
; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
@@ -12,6 +18,21 @@
; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func3
; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-BLOCK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-ALLOW-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func1 with count 10 out of 10
+
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"
|
|
@llvm/pr-subscribers-llvm-analysis Author: Snehasish Kumar (snehasish) ChangesAllow users to set the minimum absolute count for indirect call promotion. This is primarily meant to be control indirect call promotion for synthetic vp metadata introduced in #141164 for use by MemProf. Full diff: https://github.com/llvm/llvm-project/pull/145282.diff 2 Files Affected:
diff --git a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
index 90065e053b629..861dc0d364b84 100644
--- a/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
+++ b/llvm/lib/Analysis/IndirectCallPromotionAnalysis.cpp
@@ -37,6 +37,12 @@ static cl::opt<unsigned>
cl::desc("The percentage threshold against total "
"count for the promotion"));
+// Set the minimum absolute count threshold for indirect call promotion.
+// Candidates with counts below this threshold will not be promoted.
+static cl::opt<unsigned> ICPMinimumCountThreshold(
+ "icp-minimum-count-threshold", cl::init(0), cl::Hidden,
+ cl::desc("Minimum absolute count for promotion candidate"));
+
// Set the maximum number of targets to promote for a single indirect-call
// callsite.
static cl::opt<unsigned>
@@ -51,7 +57,8 @@ cl::opt<unsigned> MaxNumVTableAnnotations(
bool ICallPromotionAnalysis::isPromotionProfitable(uint64_t Count,
uint64_t TotalCount,
uint64_t RemainingCount) {
- return Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
+ return Count >= ICPMinimumCountThreshold &&
+ Count * 100 >= ICPRemainingPercentThreshold * RemainingCount &&
Count * 100 >= ICPTotalPercentThreshold * TotalCount;
}
diff --git a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
index 3b0e3250cd6df..7f1fea9d1386a 100644
--- a/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
+++ b/llvm/test/Transforms/PGOProfile/indirect_call_promotion.ll
@@ -1,6 +1,12 @@
; RUN: opt < %s -passes=pgo-icall-prom -S -icp-total-percent-threshold=50 | FileCheck %s --check-prefix=ICALL-PROM
; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS-REMARK
; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=20 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS2-REMARK
+; Test minimum count threshold - should prevent func1 promotion (count 10 < threshold 15)
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-BLOCK
+; Test minimum count threshold - should allow func4 promotion (count 1030 > threshold 15)
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-ALLOW
+; Test edge case - threshold exactly at count value
+; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=10 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-EDGE
; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
; PASS-REMARK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
@@ -12,6 +18,21 @@
; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func3
; PASS2-REMARK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-BLOCK: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-BLOCK-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-ALLOW: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-ALLOW-NOT: remark: <unknown>:0:0: Promote indirect call to func1
+
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func4 with count 1030 out of 1600
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func2 with count 410 out of 570
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func3 with count 150 out of 160
+; MIN-COUNT-EDGE: remark: <unknown>:0:0: Promote indirect call to func1 with count 10 out of 10
+
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"
|
snehasish
force-pushed
the
users/snehasish/06-22-add_minimum_count_threshold_for_indirect_call_promotion
branch
from
cb1d8ec to
5b0b5ff
Compare
mingmingl-llvm
approved these changes
Jun 26, 2025
teresajohnson
approved these changes
Jun 26, 2025
| @@ -37,6 +37,12 @@ static cl::opt<unsigned> | |||
| cl::desc("The percentage threshold against total " | |||
| "count for the promotion")); | |||
|
|
|||
| // Set the minimum absolute count threshold for indirect call promotion. | |||
| // Candidates with counts below this threshold will not be promoted. | |||
| static cl::opt<unsigned> ICPMinimumCountThreshold( | |||
There was a problem hiding this comment.
Should this be uint64_t to match the type of the Count it is being compared against?
| @@ -1,6 +1,10 @@ | |||
| ; RUN: opt < %s -passes=pgo-icall-prom -S -icp-total-percent-threshold=50 | FileCheck %s --check-prefix=ICALL-PROM | |||
| ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS-REMARK | |||
| ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=20 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=PASS2-REMARK | |||
| ; Test minimum count threshold - should prevent func1 promotion (count 10 < threshold 15) | |||
| ; RUN: opt < %s -passes=pgo-icall-prom -S -pass-remarks=pgo-icall-prom -icp-minimum-count-threshold=15 -icp-remaining-percent-threshold=0 -icp-total-percent-threshold=0 -icp-max-prom=4 2>&1 | FileCheck %s --check-prefix=MIN-COUNT-ALLOW | |||
There was a problem hiding this comment.
Nit: The check prefix names for the new invocations are unclear (e.g. ALLOW means allow what?). Maybe just explicitly make them "MIN-COUNT-15" and "MIN-COUNT-10".
There was a problem hiding this comment.
Sure, updated the suffix.
Merge activity
|
snehasish
deleted the
users/snehasish/06-22-add_minimum_count_threshold_for_indirect_call_promotion
branch
anthonyhatran
pushed a commit
to anthonyhatran/llvm-project
that referenced
this pull request
Jun 26, 2025
Allow users to set the minimum absolute count for indirect call promotion. This is primarily meant to be control indirect call promotion for synthetic vp metadata introduced in llvm#141164 for use by MemProf.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow users to set the minimum absolute count for indirect call promotion. This is primarily meant to be control indirect call promotion for synthetic vp metadata introduced in #141164 for use by MemProf.