Skip to content

Commit 22f7897

Browse files
authored
[RISCV] Use vmv.v.x for any rv32 e64 splat with equal halves (#130530)
The prior logic was reasoning in terms of vsetivli immediates, but using the vmv.v.x is strongly profitable for high LMUL cases. The key difference is that the vmv.v.x form is rematerializeable during register allocation, and the vsle form is not. This change uses the vlmax form of the vsetvli for all cases where the 2 x size can't be encoded as a vsetivli. This has the effect of increasing VL more than necessary across the vmv.v.x, which could in theory be problematic performance-wise on some hardware. We can revisit (or add a tune flag) if this turns out to be noteworthy.
1 parent aa38754 commit 22f7897

File tree

5 files changed

+1644
-2599
lines changed

5 files changed

+1644
-2599
lines changed

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4361,27 +4361,19 @@ static SDValue splatPartsI64WithVL(const SDLoc &DL, MVT VT, SDValue Passthru,
43614361
if ((LoC >> 31) == HiC)
43624362
return DAG.getNode(RISCVISD::VMV_V_X_VL, DL, VT, Passthru, Lo, VL);
43634363

4364-
// If vl is equal to VLMAX or fits in 4 bits and Hi constant is equal to Lo,
4365-
// we could use vmv.v.x whose EEW = 32 to lower it. This allows us to use
4366-
// vlmax vsetvli or vsetivli to change the VL.
4367-
// FIXME: Support larger constants?
4368-
// FIXME: Support non-constant VLs by saturating?
4364+
// Use vmv.v.x with EEW=32. Use either a vsetivli or vsetvli to change
4365+
// VL. This can temporarily increase VL if VL less than VLMAX.
43694366
if (LoC == HiC) {
43704367
SDValue NewVL;
4371-
if (isAllOnesConstant(VL) ||
4372-
(isa<RegisterSDNode>(VL) &&
4373-
cast<RegisterSDNode>(VL)->getReg() == RISCV::X0))
4374-
NewVL = DAG.getRegister(RISCV::X0, MVT::i32);
4375-
else if (isa<ConstantSDNode>(VL) && isUInt<4>(VL->getAsZExtVal()))
4368+
if (isa<ConstantSDNode>(VL) && isUInt<4>(VL->getAsZExtVal()))
43764369
NewVL = DAG.getNode(ISD::ADD, DL, VL.getValueType(), VL, VL);
4377-
4378-
if (NewVL) {
4379-
MVT InterVT =
4380-
MVT::getVectorVT(MVT::i32, VT.getVectorElementCount() * 2);
4381-
auto InterVec = DAG.getNode(RISCVISD::VMV_V_X_VL, DL, InterVT,
4382-
DAG.getUNDEF(InterVT), Lo, NewVL);
4383-
return DAG.getNode(ISD::BITCAST, DL, VT, InterVec);
4384-
}
4370+
else
4371+
NewVL = DAG.getRegister(RISCV::X0, MVT::i32);
4372+
MVT InterVT =
4373+
MVT::getVectorVT(MVT::i32, VT.getVectorElementCount() * 2);
4374+
auto InterVec = DAG.getNode(RISCVISD::VMV_V_X_VL, DL, InterVT,
4375+
DAG.getUNDEF(InterVT), Lo, NewVL);
4376+
return DAG.getNode(ISD::BITCAST, DL, VT, InterVec);
43854377
}
43864378
}
43874379

0 commit comments

Comments
 (0)