Skip to content

[LAA] Keep pointer checks on partial analysis #139719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 4, 2025
Merged

[LAA] Keep pointer checks on partial analysis #139719

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
babdff1
[LoopAccessAnalysis] Keep pointer checks on partial analysis
john-brawn-arm Mar 26, 2025
6dc3239
Add extra line to LoopAccessInfo::print when we generate some but not…
john-brawn-arm May 15, 2025
686b462
Merge branch 'main' into loop_access_analysis_partial_result
john-brawn-arm May 19, 2025
d78421a
Add HasCompletePtrRtChecking
john-brawn-arm May 20, 2025
be7173d
Merge branch 'main' into loop_access_analysis_partial_result
john-brawn-arm May 21, 2025
5da2cc9
Merge branch 'main' into HEAD
john-brawn-arm May 30, 2025
a77c6f9
Add an option to control whether partial results are allowed
john-brawn-arm May 21, 2025
32dcdac
Adjust based on review comments.
john-brawn-arm Jun 4, 2025
2b02a5b
Add an extra comment
john-brawn-arm Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions llvm/include/llvm/Analysis/LoopAccessAnalysis.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,8 @@ class MemoryDepChecker {
/// of a write access.
void addAccess(LoadInst *LI);

/// Check whether the dependencies between the accesses are safe.
/// Check whether the dependencies between the accesses are safe, and records
/// the dependence information in Dependences if so.
///
/// Only checks sets with elements in \p CheckDeps.
bool areDepsSafe(const DepCandidates &AccessSets,
Expand Down Expand Up @@ -779,10 +780,14 @@ class LoopAccessInfo {

/// We need to check that all of the pointers in this list are disjoint
/// at runtime. Using std::unique_ptr to make using move ctor simpler.
/// This list may contain only partial information when we've failed to
/// analyze all the memory accesses in the loop (i.e. CanVecMem is false).
std::unique_ptr<RuntimePointerChecking> PtrRtChecking;

/// the Memory Dependence Checker which can determine the
/// The Memory Dependence Checker which can determine the
/// loop-independent and loop-carried dependences between memory accesses.
/// This will be empty if we've failed to analyze all the memory access in the
/// loop (i.e. CanVecMem is false).
std::unique_ptr<MemoryDepChecker> DepChecker;

Loop *TheLoop;
Expand Down
19 changes: 9 additions & 10 deletions llvm/lib/Analysis/LoopAccessAnalysis.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,8 +529,10 @@ void RuntimePointerChecking::groupChecks(
// equivalence class, the iteration order is deterministic.
for (auto M : DepCands.members(Access)) {
auto PointerI = PositionMap.find(M.getPointer());
assert(PointerI != PositionMap.end() &&
"pointer in equivalence class not found in PositionMap");
// If we can't find the pointer in PositionMap that means we can't
// generate a memcheck for it.
if (PointerI == PositionMap.end())
continue;
for (unsigned Pointer : PointerI->second) {
bool Merged = false;
// Mark this pointer as seen.
Expand Down Expand Up @@ -682,7 +684,9 @@ class AccessAnalysis {
/// non-intersection.
///
/// Returns true if we need no check or if we do and we can generate them
/// (i.e. the pointers have computable bounds).
/// (i.e. the pointers have computable bounds). A return value of false means
/// we couldn't analyze and generate runtime checks for all pointers in the
/// loop, but we will have checks for those pointers we could analyze.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... if \p AllowPartial is set?

bool canCheckPtrAtRT(RuntimePointerChecking &RtCheck, ScalarEvolution *SE,
Loop *TheLoop,
const DenseMap<Value *, const SCEV *> &Strides,
Expand Down Expand Up @@ -1273,7 +1277,6 @@ bool AccessAnalysis::canCheckPtrAtRT(
/*Assume=*/true)) {
CanDoAliasSetRT = false;
UncomputablePtr = Access.getPointer();
break;
}
}
}
Expand Down Expand Up @@ -1313,7 +1316,7 @@ bool AccessAnalysis::canCheckPtrAtRT(
}
}

if (MayNeedRTCheck && CanDoRT)
if (MayNeedRTCheck)
RtCheck.generateChecks(DepCands, IsDepCheckNeeded);

LLVM_DEBUG(dbgs() << "LAA: We need to do " << RtCheck.getNumberOfChecks()
Expand All @@ -1323,11 +1326,7 @@ bool AccessAnalysis::canCheckPtrAtRT(
// are needed. This can happen when all pointers point to the same underlying
// object for example.
RtCheck.Need = CanDoRT ? RtCheck.getNumberOfChecks() != 0 : MayNeedRTCheck;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stray newline strip?

bool CanDoRTIfNeeded = !RtCheck.Need || CanDoRT;
if (!CanDoRTIfNeeded)
RtCheck.reset();
return CanDoRTIfNeeded;
return !RtCheck.Need || CanDoRT;
}

void AccessAnalysis::processMemAccesses() {
Expand Down
307 changes: 254 additions & 53 deletions llvm/test/Analysis/LoopAccessAnalysis/forked-pointers.ll
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 12 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/non-constant-strides-backward.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,9 @@ define void @different_non_constant_strides_known_backward_via_assume(ptr %A, i6
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -271,6 +274,9 @@ define void @different_non_constant_strides_known_backward_via_assume_distance_l
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A)<nuw>)
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -306,6 +312,9 @@ define void @different_non_constant_strides_known_backward_via_assume_min_distan
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP3:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -341,6 +350,9 @@ define void @different_non_constant_strides_not_known_backward(ptr %A, i64 %scal
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP4:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
9 changes: 9 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/non-constant-strides-forward.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ define void @different_non_constant_strides_known_forward_via_assume(ptr %A, i64
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -119,6 +122,9 @@ define void @different_non_constant_strides_known_forward_via_assume_min_distanc
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -154,6 +160,9 @@ define void @different_non_constant_strides_not_known_forward(ptr %A, i64 %scale
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP3:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
21 changes: 12 additions & 9 deletions llvm/test/Analysis/LoopAccessAnalysis/pointer-phis.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,9 @@ define i32 @store_with_pointer_phi_in_same_bb_use_other_phi(ptr %A, ptr %B, ptr
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP10:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (256000 + %A))
; CHECK-NEXT: Member: {%A,+,8}<nuw><%loop.header>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -503,36 +506,36 @@ define void @phi_load_store_memdep_check(i1 %c, ptr %A, ptr %B, ptr %C) {
; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Check 0:
; CHECK-NEXT: Comparing group ([[GRP10:0x[0-9a-f]+]]):
; CHECK-NEXT: Comparing group ([[GRP11:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %A
; CHECK-NEXT: ptr %A
; CHECK-NEXT: Against group ([[GRP11:0x[0-9a-f]+]]):
; CHECK-NEXT: Against group ([[GRP12:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %C
; CHECK-NEXT: ptr %C
; CHECK-NEXT: Check 1:
; CHECK-NEXT: Comparing group ([[GRP10]]):
; CHECK-NEXT: Comparing group ([[GRP11]]):
; CHECK-NEXT: ptr %A
; CHECK-NEXT: ptr %A
; CHECK-NEXT: Against group ([[GRP12:0x[0-9a-f]+]]):
; CHECK-NEXT: Against group ([[GRP13:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %B
; CHECK-NEXT: ptr %B
; CHECK-NEXT: Check 2:
; CHECK-NEXT: Comparing group ([[GRP11]]):
; CHECK-NEXT: Comparing group ([[GRP12]]):
; CHECK-NEXT: ptr %C
; CHECK-NEXT: ptr %C
; CHECK-NEXT: Against group ([[GRP12]]):
; CHECK-NEXT: Against group ([[GRP13]]):
; CHECK-NEXT: ptr %B
; CHECK-NEXT: ptr %B
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP10]]:
; CHECK-NEXT: Group [[GRP11]]:
; CHECK-NEXT: (Low: %A High: (2 + %A))
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Group [[GRP11]]:
; CHECK-NEXT: Group [[GRP12]]:
; CHECK-NEXT: (Low: %C High: (2 + %C))
; CHECK-NEXT: Member: %C
; CHECK-NEXT: Member: %C
; CHECK-NEXT: Group [[GRP12]]:
; CHECK-NEXT: Group [[GRP13]]:
; CHECK-NEXT: (Low: %B High: (2 + %B))
; CHECK-NEXT: Member: %B
; CHECK-NEXT: Member: %B
Expand Down
14 changes: 14 additions & 0 deletions ...ysis/LoopAccessAnalysis/retry-runtime-checks-after-dependence-analysis-forked-pointers.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,13 @@ define void @dependency_check_and_runtime_checks_needed_select_of_ptr_add_recs_m
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP9:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: ((4 * %offset) + %a) High: ((4 * %offset) + (4 * %n) + %a))
; CHECK-NEXT: Member: {((4 * %offset) + %a),+,4}<%loop>
; CHECK-NEXT: Group [[GRP10:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %a High: ((4 * %n) + %a))
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -208,6 +215,13 @@ define void @dependency_check_and_runtime_checks_needed_select_of_ptr_add_recs_m
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP11:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: ((4 * %offset) + %a) High: ((4 * %offset) + (4 * %n) + %a))
; CHECK-NEXT: Member: {((4 * %offset) + %a),+,4}<%loop>
; CHECK-NEXT: Group [[GRP12:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %a High: ((4 * %n) + %a))
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
14 changes: 14 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/underlying-object-loop-varying-phi.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@ define void @indirect_ptr_recurrences_read_write_may_alias_no_tbaa(ptr %A, ptr %
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: (8 + %B)<nuw> High: (40 + %B))
; CHECK-NEXT: Member: {(8 + %B)<nuw>,+,8}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -111,7 +114,18 @@ define void @indirect_ptr_recurrences_read_write_may_alias_different_obj(ptr %A,
; CHECK-NEXT: Report: cannot identify array bounds
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Check 0:
; CHECK-NEXT: Comparing group ([[GRP2:0x[0-9a-f]+]]):
; CHECK-NEXT: %gep.C = getelementptr inbounds ptr, ptr %C, i64 %iv
; CHECK-NEXT: Against group ([[GRP3:0x[0-9a-f]+]]):
; CHECK-NEXT: %gep.B = getelementptr inbounds ptr, ptr %B, i64 %iv
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2]]:
; CHECK-NEXT: (Low: (8 + %C)<nuw> High: (36 + %C))
; CHECK-NEXT: Member: {(8 + %C)<nuw>,+,8}<nuw><%loop>
; CHECK-NEXT: Group [[GRP3]]:
; CHECK-NEXT: (Low: (8 + %B)<nuw> High: (40 + %B))
; CHECK-NEXT: Member: {(8 + %B)<nuw>,+,8}<nuw><%loop>
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
Loading