Skip to content

Commit c965294

Browse files
committed
[AArch64][SME] Fix restoring callee-saves from FP with hazard padding
Currently, when hazard-padding is enabled a (fixed-size) hazard slot is placed in the CS area, just after the frame record. The size of this slot is part of the "CalleeSaveBaseToFrameRecordOffset". The SVE epilogue emission code assumed this offset was always zero, and incorrectly setting the stack pointer, resulting in all SVE registers being reloaded from incorrect offsets. ``` | prev_lr | | prev_fp | | (a.k.a. "frame record") | |-----------------------------------| <- fp(=x29) | <hazard padding> | |-----------------------------------| <- callee-saved base | | | callee-saved fp/simd/SVE regs | | | |-----------------------------------| <- SVE callee-save base ``` i.e. in the above diagram, the code assumed `fp == callee-saved base`.
1 parent b62488f commit c965294

File tree

2 files changed

+431
-4
lines changed

2 files changed

+431
-4
lines changed

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2539,13 +2539,15 @@ void AArch64FrameLowering::emitEpilogue(MachineFunction &MF,
25392539
// restore the stack pointer from the frame pointer prior to SVE CSR
25402540
// restoration.
25412541
if (AFI->isStackRealigned() || MFI.hasVarSizedObjects()) {
2542-
if (int64_t CalleeSavedSize = AFI->getSVECalleeSavedStackSize()) {
2542+
if (int64_t SVECalleeSavedSize = AFI->getSVECalleeSavedStackSize()) {
25432543
// Set SP to start of SVE callee-save area from which they can
25442544
// be reloaded. The code below will deallocate the stack space
25452545
// space by moving FP -> SP.
2546-
emitFrameOffset(MBB, RestoreBegin, DL, AArch64::SP, AArch64::FP,
2547-
StackOffset::getScalable(-CalleeSavedSize), TII,
2548-
MachineInstr::FrameDestroy);
2546+
emitFrameOffset(
2547+
MBB, RestoreBegin, DL, AArch64::SP, AArch64::FP,
2548+
StackOffset::get(-AFI->getCalleeSaveBaseToFrameRecordOffset(),
2549+
-SVECalleeSavedSize),
2550+
TII, MachineInstr::FrameDestroy);
25492551
}
25502552
} else {
25512553
if (AFI->getSVECalleeSavedStackSize()) {

0 commit comments

Comments
 (0)