Skip to content

Enhance RISCV machine outlining to support a tailcall strategy. #117526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Enhance RISCV machine outlining to support a tailcall strategy. #117526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 81 additions & 26 deletions llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2925,9 +2925,42 @@ bool RISCVInstrInfo::isMBBSafeToOutlineFrom(MachineBasicBlock &MBB,
return TargetInstrInfo::isMBBSafeToOutlineFrom(MBB, Flags);
}

// Enum values indicating how an outlined call should be constructed.
/// Constants defining how certain sequences should be outlined.
/// This encompasses how an outlined function should be called, and what kind of
/// frame should be emitted for that outlined function.
///
/// \p MachineOutlinerCallViaX5 implies that the function should be called with
/// using X5 as an alternative link register.
///
/// That is,
///
/// I1 Materialize addr in X5 OUTLINED_FUNCTION:
/// I2 --> JAL X5 I1
/// I3 I2
/// I3
/// RET X5
///
/// * Call construction overhead: 2 insns
/// * Frame construction overhead: 1 (ret)
/// * Requires stack fixups? No
///
/// \p MachineOutlinerTailCall implies that the function is being created from
/// a sequence of instructions ending in a return.
///
/// That is,
///
/// I1 OUTLINED_FUNCTION:
/// I2 --> B OUTLINED_FUNCTION I1
/// RET I2
/// RET
///
/// * Call construction overhead: 2 insns
/// * Frame construction overhead: 0 (Return included in sequence)
/// * Requires stack fixups? No
///
enum MachineOutlinerConstructionID {
MachineOutlinerDefault
MachineOutlinerCallViaX5,
MachineOutlinerTailCall
};

bool RISCVInstrInfo::shouldOutlineFromFunctionByDefault(
Expand All @@ -2941,14 +2974,33 @@ RISCVInstrInfo::getOutliningCandidateInfo(
std::vector<outliner::Candidate> &RepeatedSequenceLocs,
unsigned MinRepeats) const {

// First we need to filter out candidates where the X5 register (IE t0) can't
// be used to setup the function call.
auto CannotInsertCall = [](outliner::Candidate &C) {
const TargetRegisterInfo *TRI = C.getMF()->getSubtarget().getRegisterInfo();
return !C.isAvailableAcrossAndOutOfSeq(RISCV::X5, *TRI);
};
// If the last instruction in any candidate is a terminator, then we should
// tail call all of the candidates.
bool IsTailCall = RepeatedSequenceLocs[0].back().isTerminator();

if (!IsTailCall) {
// Filter out candidates where the X5 register (IE t0) can't
// be used to setup the function call.
auto CannotInsertCall = [](outliner::Candidate &C) {
const TargetRegisterInfo *TRI =
C.getMF()->getSubtarget().getRegisterInfo();
if (!C.isAvailableAcrossAndOutOfSeq(RISCV::X5, *TRI))
return true;

llvm::erase_if(RepeatedSequenceLocs, CannotInsertCall);
// Don't allow modifying the X5 register which we use for return addresses
// for these outlined functions.
for (const auto &MI : C) {
// FIXME: Why is this case not handled by isAvailableAcrossAndOutOfSeq
// above?
if (MI.modifiesRegister(RISCV::X5, TRI))
return true;
}

return false;
};

llvm::erase_if(RepeatedSequenceLocs, CannotInsertCall);
}

// If the sequence doesn't have enough candidates left, then we're done.
if (RepeatedSequenceLocs.size() < MinRepeats)
Expand All @@ -2961,8 +3013,12 @@ RISCVInstrInfo::getOutliningCandidateInfo(

// call t0, function = 8 bytes.
unsigned CallOverhead = 8;

MachineOutlinerConstructionID OutlinerType =
IsTailCall ? MachineOutlinerTailCall : MachineOutlinerCallViaX5;

for (auto &C : RepeatedSequenceLocs)
C.setCallInfo(MachineOutlinerDefault, CallOverhead);
C.setCallInfo(OutlinerType, CallOverhead);

// jr t0 = 4 bytes, 2 bytes if compressed instructions are enabled.
unsigned FrameOverhead = 4;
Expand All @@ -2972,19 +3028,19 @@ RISCVInstrInfo::getOutliningCandidateInfo(
.hasStdExtCOrZca())
FrameOverhead = 2;

// There is no overhead in the frame when doing a tail call.
if (IsTailCall)
FrameOverhead = 0;

return std::make_unique<outliner::OutlinedFunction>(
RepeatedSequenceLocs, SequenceSize, FrameOverhead,
MachineOutlinerDefault);
RepeatedSequenceLocs, SequenceSize, FrameOverhead, OutlinerType);
}

outliner::InstrType
RISCVInstrInfo::getOutliningTypeImpl(const MachineModuleInfo &MMI,
MachineBasicBlock::iterator &MBBI,
unsigned Flags) const {
MachineInstr &MI = *MBBI;
MachineBasicBlock *MBB = MI.getParent();
const TargetRegisterInfo *TRI =
MBB->getParent()->getSubtarget().getRegisterInfo();
const auto &F = MI.getMF()->getFunction();

// We can manually strip out CFI instructions later.
Expand All @@ -2995,17 +3051,6 @@ RISCVInstrInfo::getOutliningTypeImpl(const MachineModuleInfo &MMI,
return F.needsUnwindTableEntry() ? outliner::InstrType::Illegal
: outliner::InstrType::Invisible;

// We need support for tail calls to outlined functions before return
// statements can be allowed.
if (MI.isReturn())
return outliner::InstrType::Illegal;

// Don't allow modifying the X5 register which we use for return addresses for
// these outlined functions.
if (MI.modifiesRegister(RISCV::X5, TRI) ||
MI.getDesc().hasImplicitDefOfPhysReg(RISCV::X5))
return outliner::InstrType::Illegal;

// Make sure the operands don't reference something unsafe.
for (const auto &MO : MI.operands()) {

Expand Down Expand Up @@ -3041,6 +3086,9 @@ void RISCVInstrInfo::buildOutlinedFrame(

MBB.addLiveIn(RISCV::X5);

if (OF.FrameConstructionID == MachineOutlinerTailCall)
return;

// Add in a return instruction to the end of the outlined frame.
MBB.insert(MBB.end(), BuildMI(MF, DebugLoc(), get(RISCV::JALR))
.addReg(RISCV::X0, RegState::Define)
Expand All @@ -3052,6 +3100,13 @@ MachineBasicBlock::iterator RISCVInstrInfo::insertOutlinedCall(
Module &M, MachineBasicBlock &MBB, MachineBasicBlock::iterator &It,
MachineFunction &MF, outliner::Candidate &C) const {

if (C.CallConstructionID == MachineOutlinerTailCall) {
It = MBB.insert(It, BuildMI(MF, DebugLoc(), get(RISCV::PseudoTAIL))
.addGlobalAddress(M.getNamedValue(MF.getName()), 0,
RISCVII::MO_CALL));
return It;
}

// Add in a call instruction to the outlined function at the given location.
It = MBB.insert(It,
BuildMI(MF, DebugLoc(), get(RISCV::PseudoCALLReg), RISCV::X5)
Expand Down
20 changes: 7 additions & 13 deletions llvm/test/CodeGen/RISCV/machine-outliner-cfi.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,11 @@ body: |
; RV32I-MO-LABEL: name: func1
; RV32I-MO: liveins: $x10, $x11
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func1
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
CFI_INSTRUCTION offset $x1, 0
$x11 = ORI $x11, 1023
Expand All @@ -49,13 +47,11 @@ body: |
; RV32I-MO-LABEL: name: func2
; RV32I-MO: liveins: $x10, $x11
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func2
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
CFI_INSTRUCTION offset $x1, 0
$x11 = ORI $x11, 1023
Expand All @@ -76,13 +72,11 @@ body: |
; RV32I-MO-LABEL: name: func3
; RV32I-MO: liveins: $x10, $x11
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func3
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
CFI_INSTRUCTION offset $x1, -12
$x11 = ORI $x11, 1023
Expand All @@ -103,4 +97,4 @@ body: |
# OUTLINED-NEXT: $x12 = ADDI $x10, 17
# OUTLINED-NEXT: $x11 = AND $x12, $x11
# OUTLINED-NEXT: $x10 = SUB $x10, $x11
# OUTLINED-NEXT: $x0 = JALR $x5, 0
# OUTLINED-NEXT: PseudoRET
13 changes: 8 additions & 5 deletions llvm/test/CodeGen/RISCV/machine-outliner-leaf-descendants.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,25 +94,28 @@ define i32 @_Z2f6v() minsize {
; CHECK-BASELINE-NEXT: li a3, 0x4
; CHECK-BASELINE-NEXT: li a4, 0x5
; CHECK-BASELINE-NEXT: li a5, 0x6
; CHECK-BASELINE-NEXT: jr t0
; CHECK-BASELINE-NEXT: auipc t1, 0x0
; CHECK-BASELINE-NEXT: jr t1 <OUTLINED_FUNCTION_0+0x18>

; CHECK-BASELINE: <OUTLINED_FUNCTION_1>:
; CHECK-BASELINE-NEXT: li a0, 0x1
; CHECK-BASELINE-NEXT: li a1, 0x2
; CHECK-BASELINE-NEXT: li a2, 0x3
; CHECK-BASELINE-NEXT: li a3, 0x4
; CHECK-BASELINE-NEXT: li a4, 0x5
; CHECK-BASELINE-NEXT: li a5, 0x7
; CHECK-BASELINE-NEXT: jr t0
; CHECK-BASELINE-NEXT: li a5, 0x8
; CHECK-BASELINE-NEXT: auipc t1, 0x0
; CHECK-BASELINE-NEXT: jr t1 <OUTLINED_FUNCTION_1+0x18>

; CHECK-BASELINE: <OUTLINED_FUNCTION_2>:
; CHECK-BASELINE-NEXT: li a0, 0x1
; CHECK-BASELINE-NEXT: li a1, 0x2
; CHECK-BASELINE-NEXT: li a2, 0x3
; CHECK-BASELINE-NEXT: li a3, 0x4
; CHECK-BASELINE-NEXT: li a4, 0x5
; CHECK-BASELINE-NEXT: li a5, 0x8
; CHECK-BASELINE-NEXT: jr t0
; CHECK-BASELINE-NEXT: li a5, 0x7
; CHECK-BASELINE-NEXT: auipc t1, 0x0
; CHECK-BASELINE-NEXT: jr t1 <OUTLINED_FUNCTION_2+0x18>

; CHECK-LEAF-DESCENDANTS: <OUTLINED_FUNCTION_0>:
; CHECK-LEAF-DESCENDANTS-NEXT: li a0, 0x1
Expand Down
16 changes: 8 additions & 8 deletions llvm/test/CodeGen/RISCV/machine-outliner-patchable.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ define void @fentry0(i1 %a) nounwind "fentry-call"="true" {
; CHECK-LABEL: fentry0:
; CHECK-NEXT: # %bb.0:
; CHECK-NEXT: # FEntry call
; CHECK: # %bb.1:
; CHECK-NEXT: call t0, OUTLINED_FUNCTION_1
; CHECK: .LBB0_2:
; CHECK-NEXT: tail OUTLINED_FUNCTION_0
entry:
br i1 %a, label %if.then, label %if.end
if.then:
Expand All @@ -26,8 +26,8 @@ define void @fentry1(i1 %a) nounwind "fentry-call"="true" {
; CHECK-LABEL: fentry1:
; CHECK-NEXT: # %bb.0:
; CHECK-NEXT: # FEntry call
; CHECK: # %bb.1:
; CHECK-NEXT: call t0, OUTLINED_FUNCTION_1
; CHECK: .LBB1_2:
; CHECK-NEXT: tail OUTLINED_FUNCTION_0
entry:
br i1 %a, label %if.then, label %if.end
if.then:
Expand All @@ -46,8 +46,8 @@ define void @patchable0(i1 %a) nounwind "patchable-function-entry"="2" {
; CHECK-NEXT: # %bb.0:
; CHECK-NEXT: nop
; CHECK-NEXT: nop
; CHECK: # %bb.1:
; CHECK-NEXT: call t0, OUTLINED_FUNCTION_1
; CHECK: .LBB2_2:
; CHECK-NEXT: tail OUTLINED_FUNCTION_0
entry:
br i1 %a, label %if.then, label %if.end
if.then:
Expand All @@ -64,8 +64,8 @@ define void @patchable1(i1 %a) nounwind "patchable-function-entry"="2" {
; CHECK-NEXT: # %bb.0:
; CHECK-NEXT: nop
; CHECK-NEXT: nop
; CHECK: # %bb.1:
; CHECK-NEXT: call t0, OUTLINED_FUNCTION_1
; CHECK: .LBB3_2:
; CHECK-NEXT: tail OUTLINED_FUNCTION_0
entry:
br i1 %a, label %if.then, label %if.end
if.then:
Expand Down
18 changes: 6 additions & 12 deletions llvm/test/CodeGen/RISCV/machine-outliner-position.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,13 @@ body: |
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x10 = ORI $x10, 1023
; RV32I-MO-NEXT: EH_LABEL <mcsymbol .Ltmp0>
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func1
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x10 = ORI $x10, 1023
; RV64I-MO-NEXT: EH_LABEL <mcsymbol .Ltmp0>
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
EH_LABEL <mcsymbol .Ltmp0>
$x11 = ORI $x11, 1023
Expand All @@ -53,15 +51,13 @@ body: |
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x10 = ORI $x10, 1023
; RV32I-MO-NEXT: GC_LABEL <mcsymbol .Ltmp1>
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func2
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x10 = ORI $x10, 1023
; RV64I-MO-NEXT: GC_LABEL <mcsymbol .Ltmp1>
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
GC_LABEL <mcsymbol .Ltmp1>
$x11 = ORI $x11, 1023
Expand All @@ -81,15 +77,13 @@ body: |
; RV32I-MO-NEXT: {{ $}}
; RV32I-MO-NEXT: $x10 = ORI $x10, 1023
; RV32I-MO-NEXT: ANNOTATION_LABEL <mcsymbol .Ltmp2>
; RV32I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV32I-MO-NEXT: PseudoRET
; RV32I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
; RV64I-MO-LABEL: name: func3
; RV64I-MO: liveins: $x10, $x11
; RV64I-MO-NEXT: {{ $}}
; RV64I-MO-NEXT: $x10 = ORI $x10, 1023
; RV64I-MO-NEXT: ANNOTATION_LABEL <mcsymbol .Ltmp2>
; RV64I-MO-NEXT: $x5 = PseudoCALLReg target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit-def $x5, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x10, implicit $x11
; RV64I-MO-NEXT: PseudoRET
; RV64I-MO-NEXT: PseudoTAIL target-flags(riscv-call) @OUTLINED_FUNCTION_0, implicit $x2, implicit-def $x10, implicit-def $x11, implicit-def $x12, implicit $x2, implicit $x10, implicit $x11
$x10 = ORI $x10, 1023
ANNOTATION_LABEL <mcsymbol .Ltmp2>
$x11 = ORI $x11, 1023
Expand Down
Loading
Loading