[SandboxVec][DAG] Avoid unnecessary dependency scan and improve description #112057
+63
−26
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-vectorizers @llvm/pr-subscribers-llvm-transforms Author: vporpo (vporpo) ChangesWhen NewInterval is below DAGInterval we used to revisit instructions already visited. This patch fixes this by separating the scan in two:
This is further explained in the new description. Full diff: https://github.com/llvm/llvm-project/pull/112057.diff 1 Files Affected:
diff --git a/llvm/lib/Transforms/Vectorize/SandboxVectorizer/DependencyGraph.cpp b/llvm/lib/Transforms/Vectorize/SandboxVectorizer/DependencyGraph.cpp
index a8e362571837cc..1043c85f78e141 100644
--- a/llvm/lib/Transforms/Vectorize/SandboxVectorizer/DependencyGraph.cpp
+++ b/llvm/lib/Transforms/Vectorize/SandboxVectorizer/DependencyGraph.cpp
@@ -276,30 +276,44 @@ Interval<Instruction> DependencyGraph::extend(ArrayRef<Instruction *> Instrs) {
// Create the dependencies.
//
- // 1. DAGInterval empty 2. New is below Old 3. New is above old
- // ------------------------ ------------------- -------------------
- // Scan: DstN: Scan:
- // +---+ -ScanTopN +---+DstTopN -ScanTopN
- // | | | |New| |
- // |Old| | +---+ -ScanBotN
- // | | | +---+
- // DstN: Scan: +---+DstN: | | |
- // +---+DstTopN -ScanTopN +---+DstTopN | |Old|
- // |New| | |New| | | |
- // +---+DstBotN -ScanBotN +---+DstBotN -ScanBotN +---+DstBotN
-
- // 1. This is a new DAG.
- if (DAGInterval.empty()) {
- assert(NewInterval == InstrsInterval && "Expected empty DAGInterval!");
- auto DstRange = MemDGNodeIntervalBuilder::make(NewInterval, *this);
+ // 1. This is a new DAG, DAGInterval is empty. Fully scan the whole interval.
+ // +---+ - -
+ // | | SrcN | |
+ // | | | | SrcRange |
+ // |New| v | | DstRange
+ // | | DstN - |
+ // | | |
+ // +---+ -
+ // We are scanning for deps with destination in NewInterval and sources in
+ // NewInterval until DstN, for each DstN.
+ auto FullScan = [this](const Interval<Instruction> Intvl) {
+ auto DstRange = MemDGNodeIntervalBuilder::make(Intvl, *this);
if (!DstRange.empty()) {
for (MemDGNode &DstN : drop_begin(DstRange)) {
auto SrcRange = Interval<MemDGNode>(DstRange.top(), DstN.getPrevNode());
scanAndAddDeps(DstN, SrcRange);
}
}
+ };
+ if (DAGInterval.empty()) {
+ assert(NewInterval == InstrsInterval && "Expected empty DAGInterval!");
+ FullScan(NewInterval);
}
// 2. The new section is below the old section.
+ // +---+ -
+ // | | |
+ // |Old| SrcN |
+ // | | | |
+ // +---+ | | SrcRange
+ // +---+ | | -
+ // | | | | |
+ // |New| v | | DstRange
+ // | | DstN - |
+ // | | |
+ // +---+ -
+ // We are scanning for deps with destination in NewInterval because the deps
+ // in DAGInterval have already been computed. We consider sources in the whole
+ // range including both NewInterval and DAGInterval until DstN, for each DstN.
else if (DAGInterval.bottom()->comesBefore(NewInterval.top())) {
auto DstRange = MemDGNodeIntervalBuilder::make(NewInterval, *this);
auto SrcRangeFull = MemDGNodeIntervalBuilder::make(
@@ -312,16 +326,40 @@ Interval<Instruction> DependencyGraph::extend(ArrayRef<Instruction *> Instrs) {
}
// 3. The new section is above the old section.
else if (NewInterval.bottom()->comesBefore(DAGInterval.top())) {
- auto DstRange = MemDGNodeIntervalBuilder::make(
- NewInterval.getUnionInterval(DAGInterval), *this);
- auto SrcRangeFull = MemDGNodeIntervalBuilder::make(NewInterval, *this);
- if (!DstRange.empty()) {
- for (MemDGNode &DstN : drop_begin(DstRange)) {
- auto SrcRange =
- Interval<MemDGNode>(SrcRangeFull.top(), DstN.getPrevNode());
- scanAndAddDeps(DstN, SrcRange);
- }
- }
+ // +---+ - -
+ // | | SrcN | |
+ // |New| | | SrcRange | DstRange
+ // | | v | |
+ // | | DstN - |
+ // | | |
+ // +---+ -
+ // +---+
+ // |Old|
+ // | |
+ // +---+
+ // When scanning for deps with destination in NewInterval we need to
+ // consider all sources from the top of NewInterval until DstN, because
+ // this interval is new and contains no existing memory deps.
+ FullScan(NewInterval);
+
+ // +---+ -
+ // | | |
+ // |New| SrcN | SrcRange
+ // | | | |
+ // | | | |
+ // | | | |
+ // +---+ | -
+ // +---+ | -
+ // |Old| v | DstRange
+ // | | DstN |
+ // +---+ -
+ // When scanning for deps with destination in DAGInterval we need to
+ // consider sources from the NewInterval only, because all intra-DAGInterval
+ // dependencies have already been created.
+ auto DstRangeOld = MemDGNodeIntervalBuilder::make(DAGInterval, *this);
+ auto SrcRange = MemDGNodeIntervalBuilder::make(NewInterval, *this);
+ for (MemDGNode &DstN : DstRangeOld)
+ scanAndAddDeps(DstN, SrcRange);
} else {
llvm_unreachable("We don't expect extending in both directions!");
}
|
…iption When NewInterval is below DAGInterval we used to revisit instructions already visited. This patch fixes this by separating the scan in two: 1. The full scan of the NewInterval, and 2. The cross-interval scan for DAGInterval. This is further explained in the new description.
|
Updated the description of 3rd case . We are now calling |
tmsri
approved these changes
Oct 14, 2024
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/30/builds/8219 Here is the relevant piece of the build log for the reference |
DanielCChen
pushed a commit
to DanielCChen/llvm-project
that referenced
this pull request
Oct 16, 2024
…iption (llvm#112057) When NewInterval is below DAGInterval we used to revisit instructions already visited. This patch fixes this by separating the scan in two: 1. The full scan of the NewInterval, and 2. The cross-interval scan for DAGInterval. This is further explained in the new description.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When NewInterval is below DAGInterval we used to revisit instructions already visited. This patch fixes this by separating the scan in two:
This is further explained in the new description.