Skip to content

[LV][NFC]Introduce isScalableVectorizationAllowed() to refactor getMaxLegalScalableVF(). #98916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 17, 2024
Merged

[LV][NFC]Introduce isScalableVectorizationAllowed() to refactor getMaxLegalScalableVF(). #98916

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e2c6fc2
[𝘀𝗽𝗿] initial version
alexey-bataev Jul 15, 2024
0538ddc
Rebase, address comments
alexey-bataev Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 39 additions & 10 deletions llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1628,6 +1628,10 @@ class LoopVectorizationCostModel {
ElementCount MaxSafeVF,
bool FoldTailByMasking);

/// Checks if scalable vectorization is supported and enabled. Caches the
/// result to avoid repeated debug dumps for repeated queries.
bool isScalableVectorizationAllowed();

/// \return the maximum legal scalable VF, based on the safe max number
/// of elements.
ElementCount getMaxLegalScalableVF(unsigned MaxSafeElements);
Expand Down Expand Up @@ -1692,6 +1696,9 @@ class LoopVectorizationCostModel {
std::optional<std::pair<TailFoldingStyle, TailFoldingStyle>>
ChosenTailFoldingStyle;

/// true if scalable vectorization is supported and enabled.
std::optional<bool> IsScalableVectorizationAllowed;

/// A map holding scalar costs for different vectorization factors. The
/// presence of a cost for an instruction in the mapping indicates that the
/// instruction will be scalarized when vectorizing with the associated
Expand Down Expand Up @@ -4144,15 +4151,18 @@ bool LoopVectorizationCostModel::runtimeChecksRequired() {
return false;
}

ElementCount
LoopVectorizationCostModel::getMaxLegalScalableVF(unsigned MaxSafeElements) {
bool LoopVectorizationCostModel::isScalableVectorizationAllowed() {
if (IsScalableVectorizationAllowed)
return *IsScalableVectorizationAllowed;

IsScalableVectorizationAllowed = false;
if (!TTI.supportsScalableVectors() && !ForceTargetSupportsScalableVectors)
return ElementCount::getScalable(0);
return false;

if (Hints->isScalableVectorizationDisabled()) {
reportVectorizationInfo("Scalable vectorization is explicitly disabled",
"ScalableVectorizationDisabled", ORE, TheLoop);
return ElementCount::getScalable(0);
return false;
}

LLVM_DEBUG(dbgs() << "LV: Scalable vectorization is available\n");
Expand All @@ -4172,7 +4182,7 @@ LoopVectorizationCostModel::getMaxLegalScalableVF(unsigned MaxSafeElements) {
"Scalable vectorization not supported for the reduction "
"operations found in this loop.",
"ScalableVFUnfeasible", ORE, TheLoop);
return ElementCount::getScalable(0);
return false;
}

// Disable scalable vectorization if the loop contains any instructions
Expand All @@ -4184,17 +4194,36 @@ LoopVectorizationCostModel::getMaxLegalScalableVF(unsigned MaxSafeElements) {
reportVectorizationInfo("Scalable vectorization is not supported "
"for all element types found in this loop.",
"ScalableVFUnfeasible", ORE, TheLoop);
return ElementCount::getScalable(0);
return false;
}

if (!Legal->isSafeForAnyVectorWidth()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-commit: this stems from the original order of checks, but could the check if isSafeForAnyVectorWidth be dropped? Sounds redundant, or good to indicate why otherwise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It affects the vectorizations. Need to check the scale limit only if !Legal->isSafeForAnyVectorWidth()

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so be it. The error message could perhaps be more accurate then, as lack of maximum vscale alone may not imply that scalable VF is unfeasible. And perhaps suffice to check if (!Legal->isSafeForAnyVectorWidth() && !getMaxVScale()).

std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
if (!MaxVScale) {
reportVectorizationInfo(
"The target does not provide maximum vscale value.",
"ScalableVFUnfeasible", ORE, TheLoop);
return false;
}
}

IsScalableVectorizationAllowed = true;
return true;
}

ElementCount
LoopVectorizationCostModel::getMaxLegalScalableVF(unsigned MaxSafeElements) {
if (!isScalableVectorizationAllowed())
return ElementCount::getScalable(0);

auto MaxScalableVF = ElementCount::getScalable(
std::numeric_limits<ElementCount::ScalarTy>::max());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, preserving current behavior. Would be good if some assert could be added, independently, to make sure that MaxSafeElements corresponds to MaxScalableVF if isSafeForAnyVectorWidth(), as the former is ignored in this case, which returns the latter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said before, this requires looking into the internals of LAA. No idea how safely do it here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, probably we can just use auto MaxScalableVF = ElementCount::getScalable( MaxSafeElements); here

if (Legal->isSafeForAnyVectorWidth())
return MaxScalableVF;

std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
// Limit MaxScalableVF by the maximum safe dependence distance.
if (std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI))
MaxScalableVF = ElementCount::getScalable(MaxSafeElements / *MaxVScale);
else
MaxScalableVF = ElementCount::getScalable(0);
MaxScalableVF = ElementCount::getScalable(MaxSafeElements / *MaxVScale);

if (!MaxScalableVF)
reportVectorizationInfo(
Expand Down
Loading