Skip to content

[RISCV] Fold PseudoVMV_V_V with undef passthru, handling policy #106943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 5, 2024
Merged

[RISCV] Fold PseudoVMV_V_V with undef passthru, handling policy #106943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1620005
[RISCV] Fold PseudoVMV_V_V with undef passthru, handling policy
lukel97 Sep 4, 2024
854d924
Address comments, also don't attempt further folds if MI was deleted
lukel97 Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions llvm/lib/Target/RISCV/RISCVVectorPeephole.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ class RISCVVectorPeephole : public MachineFunctionPass {
bool convertToWholeRegister(MachineInstr &MI) const;
bool convertToUnmasked(MachineInstr &MI) const;
bool convertVMergeToVMv(MachineInstr &MI) const;
bool foldUndefPassthruVMV_V_V(MachineInstr &MI);
bool foldVMV_V_V(MachineInstr &MI);

bool isAllOnesMask(const MachineInstr *MaskDef) const;
Expand Down Expand Up @@ -472,6 +473,38 @@ bool RISCVVectorPeephole::ensureDominates(const MachineOperand &MO,
return true;
}

/// If a PseudoVMV_V_V's passthru is undef then we can replace it with its input
bool RISCVVectorPeephole::foldUndefPassthruVMV_V_V(MachineInstr &MI) {
if (RISCV::getRVVMCOpcode(MI.getOpcode()) != RISCV::VMV_V_V)
return false;
if (MI.getOperand(1).getReg() != RISCV::NoRegister)
return false;

// If the input was a pseudo with a policy operand, we can give it a tail
// agnostic policy if MI's undef tail subsumes the input's.
MachineInstr *Src = MRI->getVRegDef(MI.getOperand(2).getReg());
if (Src && !Src->hasUnmodeledSideEffects() &&
MRI->hasOneUse(MI.getOperand(2).getReg()) &&
RISCVII::hasVLOp(Src->getDesc().TSFlags) &&
RISCVII::hasVecPolicyOp(Src->getDesc().TSFlags) &&
getSEWLMULRatio(MI) == getSEWLMULRatio(*Src)) {
const MachineOperand &MIVL = MI.getOperand(3);
const MachineOperand &SrcVL =
Src->getOperand(RISCVII::getVLOpNum(Src->getDesc()));

MachineOperand &SrcPolicy =
Src->getOperand(RISCVII::getVecPolicyOpNum(Src->getDesc()));

if (isVLKnownLE(MIVL, SrcVL))
SrcPolicy.setImm(SrcPolicy.getImm() | RISCVII::TAIL_AGNOSTIC);
}

MRI->replaceRegWith(MI.getOperand(0).getReg(), MI.getOperand(2).getReg());
MI.eraseFromParent();
V0Defs.erase(&MI);
return true;
}

/// If a PseudoVMV_V_V is the only user of its input, fold its passthru and VL
/// into it.
///
Expand Down Expand Up @@ -531,9 +564,8 @@ bool RISCVVectorPeephole::foldVMV_V_V(MachineInstr &MI) {

// If MI was tail agnostic and the VL didn't increase, preserve it.
int64_t Policy = RISCVII::TAIL_UNDISTURBED_MASK_UNDISTURBED;
bool TailAgnostic = (MI.getOperand(5).getImm() & RISCVII::TAIL_AGNOSTIC) ||
Passthru.getReg() == RISCV::NoRegister;
if (TailAgnostic && isVLKnownLE(MI.getOperand(3), SrcVL))
if ((MI.getOperand(5).getImm() & RISCVII::TAIL_AGNOSTIC) &&
isVLKnownLE(MI.getOperand(3), SrcVL))
Policy |= RISCVII::TAIL_AGNOSTIC;
Src->getOperand(RISCVII::getVecPolicyOpNum(Src->getDesc())).setImm(Policy);

Expand Down Expand Up @@ -584,6 +616,10 @@ bool RISCVVectorPeephole::runOnMachineFunction(MachineFunction &MF) {
Changed |= convertToUnmasked(MI);
Changed |= convertToWholeRegister(MI);
Changed |= convertVMergeToVMv(MI);
if (foldUndefPassthruVMV_V_V(MI)) {
Changed |= true;
continue; // MI is erased
}
Changed |= foldVMV_V_V(MI);
}
}
Expand Down
1 change: 0 additions & 1 deletion llvm/test/CodeGen/RISCV/rvv/rvv-peephole-vmerge-to-vmv.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ body: |
; CHECK-NEXT: %avl:gprnox0 = COPY $x1
; CHECK-NEXT: %mask:vmv0 = PseudoVMSET_M_B8 %avl, 5 /* e32 */
; CHECK-NEXT: $v0 = COPY %mask
; CHECK-NEXT: %x:vr = PseudoVMV_V_V_M1 $noreg, %true, %avl, 5 /* e32 */, 0 /* tu, mu */
%false:vr = COPY $v8
%true:vr = COPY $v9
%avl:gprnox0 = COPY $x1
Expand Down
4 changes: 1 addition & 3 deletions llvm/test/CodeGen/RISCV/rvv/vmv.v.v-peephole.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,10 +199,8 @@ define <vscale x 2 x i32> @unfoldable_mismatched_sew(<vscale x 2 x i32> %passthr
define <vscale x 1 x i64> @undef_passthru(<vscale x 1 x i64> %passthru, <vscale x 1 x i64> %x, <vscale x 1 x i64> %y, iXLen %avl) {
; CHECK-LABEL: undef_passthru:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetvli zero, a0, e64, m1, tu, ma
; CHECK-NEXT: vsetvli zero, a0, e64, m1, ta, ma
; CHECK-NEXT: vadd.vv v8, v9, v10
; CHECK-NEXT: vsetvli zero, zero, e64, m1, ta, ma
; CHECK-NEXT: vmv.v.v v8, v8
; CHECK-NEXT: ret
%a = call <vscale x 1 x i64> @llvm.riscv.vadd.nxv1i64.nxv1i64(<vscale x 1 x i64> %passthru, <vscale x 1 x i64> %x, <vscale x 1 x i64> %y, iXLen %avl)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually hold? Consider:
SrcVL = 2, so elements 0, 1 come from either x or y, but 2 onwards come from passthru
MIVL = 4, so elements 0, 1 come from either x or y, elements 2, 3 come from passthru, and elements 4+ are undefined

Oh, I see it. You're not changing passthru or VL on Src, you're basically just dropping MI.

I think this might be cleaner if expressed as an early continue here. This is arguably a different combine entirely. You're basically just using properties of the vmv.v.v itself to prove it can be elided entirely. Actually, now that I say that, yeah, this really doesn't depend on Src at all does it? I don't think you even need the single use condition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, that's exactly what I did in #106840. I added a new peephole that doesn't check Src, but I closed it because I thought it was too similar to foldVMV_V_V.

However I've since realised that if the vmv.v.v passthru is undef, then we can still relax Src's policy to be tail agnostic if vmv.v.s tail subsumes it. E.g. if we have

vsetvli zero, a0, e64, m1, tu, ma
vadd.vv v8, v9, v10
vsetvli zero, zero, e64, m1, ta, ma
vmv.v.v v8, v8

We should actually be able to fold it into:

vsetvli zero, a0, e64, m1, ta, ma
vadd.vv v8, v9, v10

Hence why I moved it back into foldVMV_V_V, so it could take advantage of #105788.

But I completely agree with you here that this is could be much cleaner. I'm going to try again to do this as a separate peephole, but explicitly relaxing Src's policy if possible.

%b = call <vscale x 1 x i64> @llvm.riscv.vmv.v.v.nxv1i64(<vscale x 1 x i64> undef, <vscale x 1 x i64> %a, iXLen %avl)
Expand Down
3 changes: 1 addition & 2 deletions llvm/test/CodeGen/RISCV/rvv/vmv.v.v-peephole.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ body: |
; CHECK: liveins: $v8
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: %passthru:vr = COPY $v8
; CHECK-NEXT: %x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
; CHECK-NEXT: %y:vr = PseudoVMV_V_V_M1 $noreg, %x, 4, 5 /* e32 */, 1 /* ta, mu */
; CHECK-NEXT: %x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 1 /* ta, mu */
%passthru:vr = COPY $v8
%x:vr = PseudoVADD_VV_M1 %passthru, $noreg, $noreg, 4, 5 /* e32 */, 0 /* tu, mu */
%y:vr = PseudoVMV_V_V_M1 $noreg, %x, 4, 5 /* e32 */, 1 /* ta, mu */
Loading