Skip to content

[RISCV][CFI] add function epilogue cfi information #110810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 5, 2024

Conversation

dlav-sc
Copy link
Contributor

@dlav-sc dlav-sc commented Oct 2, 2024

This patch adds CFI instructions in the function epilogue.

Before patch:

addi sp, s0, -32
ld ra, 24(sp) # 8-byte Folded Reload
ld s0, 16(sp) # 8-byte Folded Reload
ld s1, 8(sp) # 8-byte Folded Reload
addi sp, sp, 32
ret

After patch:

addi sp, s0, -32
.cfi_def_cfa sp, 32
ld ra, 24(sp) # 8-byte Folded Reload
ld s0, 16(sp) # 8-byte Folded Reload
ld s1, 8(sp) # 8-byte Folded Reload
.cfi_restore ra
.cfi_restore s0
.cfi_restore s1
addi sp, sp, 32
.cfi_def_cfa_offset 0
ret

This functionality is already present in riscv-gcc, but it’s not in clang and this slightly impairs the lldb debugging experience, e.g. backtrace.

@llvmbot
Copy link
Member

llvmbot commented Oct 2, 2024

@llvm/pr-subscribers-llvm-globalisel

@llvm/pr-subscribers-debuginfo

Author: None (dlav-sc)

Changes

This patch adds CFI instructions in a function epilogue, that allows lldb to obtain a valid backtrace at the end of functions.


Patch is 1004.05 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/110810.diff

296 Files Affected:

  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.cpp (+72)
  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.h (+2)
  • (modified) llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll (+36)
  • (modified) llvm/test/CodeGen/RISCV/addrspacecast.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll (+85)
  • (modified) llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll (+85)
  • (modified) llvm/test/CodeGen/RISCV/branch-relaxation.ll (+144)
  • (modified) llvm/test/CodeGen/RISCV/callee-saved-gprs.ll (+88-8)
  • (modified) llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll (+271)
  • (modified) llvm/test/CodeGen/RISCV/cm_mvas_mvsa.ll (+8-4)
  • (modified) llvm/test/CodeGen/RISCV/double-intrinsics.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/double-round-conv.ll (+50)
  • (modified) llvm/test/CodeGen/RISCV/early-clobber-tied-def-subreg-liveness.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/eh-dwarf-cfa.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/exception-pointer-register.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/float-intrinsics.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/float-round-conv.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/fpclamptosat.ll (+176)
  • (modified) llvm/test/CodeGen/RISCV/frame-info.ll (+32)
  • (modified) llvm/test/CodeGen/RISCV/half-convert-strict.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/half-intrinsics.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/half-round-conv.ll (+80)
  • (modified) llvm/test/CodeGen/RISCV/hwasan-check-memaccess.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/intrinsic-cttz-elts-vscale.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/kcfi-mir.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/large-stack.ll (+15)
  • (modified) llvm/test/CodeGen/RISCV/live-sp.mir (+2-1)
  • (modified) llvm/test/CodeGen/RISCV/llvm.exp10.ll (+120)
  • (modified) llvm/test/CodeGen/RISCV/local-stack-slot-allocation.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/lpad.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/miss-sp-restore-eh.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/nontemporal.ll (+60)
  • (modified) llvm/test/CodeGen/RISCV/overflow-intrinsics.ll (+23)
  • (modified) llvm/test/CodeGen/RISCV/pr58025.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/pr58286.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/pr63365.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/pr69586.ll (+29)
  • (modified) llvm/test/CodeGen/RISCV/pr88365.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/prolog-epilogue.ll (+80)
  • (modified) llvm/test/CodeGen/RISCV/push-pop-opt-crash.ll (+27-25)
  • (modified) llvm/test/CodeGen/RISCV/push-pop-popret.ll (+344-44)
  • (modified) llvm/test/CodeGen/RISCV/regalloc-last-chance-recoloring-failure.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rv64-patchpoint.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rv64-statepoint-call-lowering.ll (+21)
  • (modified) llvm/test/CodeGen/RISCV/rvv-cfi-info.ll (+18)
  • (modified) llvm/test/CodeGen/RISCV/rvv/abs-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/access-fixed-objects-by-rvv.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/addi-scalable-offset.mir (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-scalable-array.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-scalable-struct.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-vector-tuple.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/binop-splats.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bitreverse-sdnode.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bitreverse-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bswap-sdnode.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bswap-vp.ll (+21)
  • (modified) llvm/test/CodeGen/RISCV/rvv/callee-saved-regs.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/calling-conv-fastcc.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/rvv/calling-conv.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/ceil-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/compressstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/ctpop-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/cttz-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-fp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv32.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv64.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vector-i8-index-cornercase.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-binop-splats.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitcast.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitreverse-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitreverse.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bswap-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bswap.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-calling-conv-fastcc.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-calling-conv.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ceil-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctlz-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctpop-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-cttz-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-expandload-int.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-floor-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fmaximum-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fminimum-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec-bf16.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec.ll (+100)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-interleave.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fshr-fshl-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-insert-subvector.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-insert.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-buildvec.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-explodevector.ll (+29)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-interleave.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-splat.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-interleaved-access.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-llrint.ll (+43)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-lrint.ll (+28)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-masked-gather.ll (+65)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-masked-scatter.ll (+59)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-fp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-int-vp.ll (+44)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-int.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-rint-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-round-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-roundeven-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-roundtozero-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-setcc-fp-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-setcc-int-vp.ll (+22)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-shuffle-vslide1down.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-shuffle-vslide1up.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-trunc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vaaddu.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vand-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vcopysign-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vdiv-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vdivu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfma-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmax-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmin-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmuladd-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwadd.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwmaccbf16.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwmul.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwsub.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vitofp-constrained-sdnode.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmacc-vp.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmaxu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vminu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vnmsac-vp.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vp-splat.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vpmerge.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vpscatter.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vrem-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vremu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vrsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsadd.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsaddu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsaddu.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vscale-range.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vselect-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssub.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssubu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssubu.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmul.ll (+7)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmulsu.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmulu.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vxor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/floor-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fmaximum-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fmaximum-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fminimum-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fminimum-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fnearbyint-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll (+432)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fshr-fshl-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/get-vlen-debugloc.mir (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/known-never-zero.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/large-rvv-stack-size.mir (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/localvar.ll (+27)
  • (modified) llvm/test/CodeGen/RISCV/rvv/memory-args.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/mgather-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/mscatter-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/named-vector-shuffle-reverse.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/nearbyint-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/no-reserved-frame.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr104480.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr88576.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr93587.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr95865.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/reg-alloc-reserve-bp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/remat.ll (+18)
  • (modified) llvm/test/CodeGen/RISCV/rvv/rint-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/round-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/roundeven-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/roundtozero-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/rvv-args-by-mem.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-fp-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-int-vp.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-integer.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/splat-vector-split-i64-vl-sdnode.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/stack-folding.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/stepvector.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/strided-vpstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/undef-earlyclobber-chain.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vaaddu-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vand-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vand-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vandn-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vandn-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdiv-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdiv-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdivu-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdivu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-deinterleave-load.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-deinterleave.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-interleave-store.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-interleave.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfadd-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfdiv-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfma-vp.ll (+124)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmadd-constrained-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmadd-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmsub-constrained-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmuladd-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfnmadd-constrained-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfnmsub-constrained-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptosi-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptoui-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptrunc-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfsub-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwmacc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwnmacc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwnmsac-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmacc-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmadd-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmax-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmaxu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmaxu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmin-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vminu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vminu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmul-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vnmsac-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vnmsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vor-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vp-reverse-int.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vp-splat.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpmerge-sdnode.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpscatter-sdnode.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vreductions-int-vp.ll (+30)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrem-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrem-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vremu-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vremu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsaddu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsaddu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vselect-int.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vselect-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsitofp-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsplats-i64.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssubu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssubu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vtrunc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vuitofp-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vwadd-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vxor-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vxor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/zvlsseg-spill.mir (+2)
  • (modified) llvm/test/CodeGen/RISCV/shadowcallstack.ll (+28)
  • (modified) llvm/test/CodeGen/RISCV/shl-cttz.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/shrinkwrap-jump-table.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/stack-inst-compress.mir (+36)
  • (modified) llvm/test/CodeGen/RISCV/stack-offset.ll (+48)
  • (modified) llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/stack-realignment.ll (+212)
  • (modified) llvm/test/CodeGen/RISCV/vararg-ilp32e.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/vararg.ll (+38)
  • (modified) llvm/test/CodeGen/RISCV/vlenb.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/xaluo.ll (+27)
  • (modified) llvm/test/CodeGen/RISCV/xcvbi.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/zbb-cmp-combine.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-additional-stack.ll (+7-1)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-cm-popretz.mir (+42-4)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-cm-push-pop.mir (+82)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-with-float.ll (+18-2)
  • (modified) llvm/test/CodeGen/RISCV/zdinx-large-spill.mir (+1)
  • (modified) llvm/test/DebugInfo/RISCV/relax-debug-frame.ll (+15-7)
diff --git a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
index 7cbd1a35b25839..78d3dafaf9c8e2 100644
--- a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
@@ -761,11 +761,18 @@ void RISCVFrameLowering::deallocateStack(MachineFunction &MF,
                                          const DebugLoc &DL, uint64_t StackSize,
                                          int64_t CFAOffset) const {
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
+  const RISCVInstrInfo *TII = STI.getInstrInfo();
 
   Register SPReg = getSPReg(STI);
 
   RI->adjustReg(MBB, MBBI, DL, SPReg, SPReg, StackOffset::getFixed(StackSize),
                 MachineInstr::FrameDestroy, getStackAlign());
+
+  unsigned CFIIndex =
+      MF.addFrameInst(MCCFIInstruction::cfiDefCfaOffset(nullptr, CFAOffset));
+  BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+      .addCFIIndex(CFIIndex)
+      .setMIFlag(MachineInstr::FrameDestroy);
 }
 
 void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
@@ -773,6 +780,7 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
+  const RISCVInstrInfo *TII = STI.getInstrInfo();
   Register FPReg = getFPReg(STI);
   Register SPReg = getSPReg(STI);
 
@@ -826,7 +834,16 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
     if (!RestoreFP) {
       adjustStackForRVV(MF, MBB, LastFrameDestroy, DL, RVVStackSize,
                         MachineInstr::FrameDestroy);
+
+      unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::cfiDefCfa(
+          nullptr, RI->getDwarfRegNum(SPReg, true), RealStackSize));
+      BuildMI(MBB, LastFrameDestroy, DL,
+              TII->get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
     }
+
+    emitCalleeSavedRVVEpilogCFI(MBB, LastFrameDestroy);
   }
 
   if (FirstSPAdjustAmount) {
@@ -841,6 +858,13 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
       RI->adjustReg(MBB, LastFrameDestroy, DL, SPReg, SPReg,
                     StackOffset::getFixed(SecondSPAdjustAmount),
                     MachineInstr::FrameDestroy, getStackAlign());
+
+      unsigned CFIIndex = MF.addFrameInst(
+          MCCFIInstruction::cfiDefCfaOffset(nullptr, FirstSPAdjustAmount));
+      BuildMI(MBB, LastFrameDestroy, DL,
+              TII->get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
     }
   }
 
@@ -858,6 +882,12 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
     RI->adjustReg(MBB, LastFrameDestroy, DL, SPReg, FPReg,
                   StackOffset::getFixed(-FPOffset), MachineInstr::FrameDestroy,
                   getStackAlign());
+
+    unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::cfiDefCfa(
+        nullptr, RI->getDwarfRegNum(SPReg, true), RealStackSize));
+    BuildMI(MBB, LastFrameDestroy, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
   }
 
   bool ApplyPop = RVFI->isPushable(MF) && MBBI != MBB.end() &&
@@ -875,7 +905,24 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
       deallocateStack(MF, MBB, MBBI, DL, StackSize,
                       /*stack_adj of cm.pop instr*/ RealStackSize - StackSize);
 
+    // Update CFA offset. After CM_POP SP should be equal to CFA, so CFA offset
+    // is zero.
     MBBI = std::next(MBBI);
+    unsigned CFIIndex =
+        MF.addFrameInst(MCCFIInstruction::cfiDefCfaOffset(nullptr, 0));
+    BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
+  }
+
+  // Recover callee-saved registers.
+  for (const auto &Entry : CSI) {
+    Register Reg = Entry.getReg();
+    unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::createRestore(
+        nullptr, RI->getDwarfRegNum(Reg, true)));
+    BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
   }
 
   // Deallocate stack if StackSize isn't a zero and if we didn't already do it
@@ -1615,6 +1662,31 @@ void RISCVFrameLowering::emitCalleeSavedRVVPrologCFI(
   }
 }
 
+void RISCVFrameLowering::emitCalleeSavedRVVEpilogCFI(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MI) const {
+  MachineFunction *MF = MBB.getParent();
+  const MachineFrameInfo &MFI = MF->getFrameInfo();
+  const RISCVRegisterInfo *RI = STI.getRegisterInfo();
+  const TargetInstrInfo &TII = *STI.getInstrInfo();
+  DebugLoc DL = MBB.findDebugLoc(MI);
+
+  const auto &RVVCSI = getRVVCalleeSavedInfo(*MF, MFI.getCalleeSavedInfo());
+  if (RVVCSI.empty())
+    return;
+
+  for (auto &CS : RVVCSI) {
+    int FI = CS.getFrameIdx();
+    if (FI >= 0 && MFI.getStackID(FI) == TargetStackID::ScalableVector) {
+      Register Reg = CS.getReg();
+      unsigned CFIIndex = MF->addFrameInst(MCCFIInstruction::createRestore(
+          nullptr, RI->getDwarfRegNum(Reg, true)));
+      BuildMI(MBB, MI, DL, TII.get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
+    }
+  }
+}
+
 bool RISCVFrameLowering::restoreCalleeSavedRegisters(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MI,
     MutableArrayRef<CalleeSavedInfo> CSI, const TargetRegisterInfo *TRI) const {
diff --git a/llvm/lib/Target/RISCV/RISCVFrameLowering.h b/llvm/lib/Target/RISCV/RISCVFrameLowering.h
index 89f95f2aa04aa6..68402bf9d81478 100644
--- a/llvm/lib/Target/RISCV/RISCVFrameLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVFrameLowering.h
@@ -92,6 +92,8 @@ class RISCVFrameLowering : public TargetFrameLowering {
   void emitCalleeSavedRVVPrologCFI(MachineBasicBlock &MBB,
                                    MachineBasicBlock::iterator MI,
                                    bool HasFP) const;
+  void emitCalleeSavedRVVEpilogCFI(MachineBasicBlock &MBB,
+                                   MachineBasicBlock::iterator MI) const;
 
   void deallocateStack(MachineFunction &MF, MachineBasicBlock &MBB,
                        MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll b/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
index 014283cf38b26b..caa749729ce198 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
@@ -25,10 +25,15 @@ define void @test_scoped_alloca(i64 %n) {
 ; RV32-NEXT:    call use_addr
 ; RV32-NEXT:    mv sp, s1
 ; RV32-NEXT:    addi sp, s0, -16
+; RV32-NEXT:    .cfi_def_cfa sp, 16
 ; RV32-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32-NEXT:    lw s1, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT:    .cfi_restore ra
+; RV32-NEXT:    .cfi_restore s0
+; RV32-NEXT:    .cfi_restore s1
 ; RV32-NEXT:    addi sp, sp, 16
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: test_scoped_alloca:
@@ -51,10 +56,15 @@ define void @test_scoped_alloca(i64 %n) {
 ; RV64-NEXT:    call use_addr
 ; RV64-NEXT:    mv sp, s1
 ; RV64-NEXT:    addi sp, s0, -32
+; RV64-NEXT:    .cfi_def_cfa sp, 32
 ; RV64-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s1, 8(sp) # 8-byte Folded Reload
+; RV64-NEXT:    .cfi_restore ra
+; RV64-NEXT:    .cfi_restore s0
+; RV64-NEXT:    .cfi_restore s1
 ; RV64-NEXT:    addi sp, sp, 32
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
   %sp = call ptr @llvm.stacksave.p0()
   %addr = alloca i8, i64 %n
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll b/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
index 01cab0d0e157bd..5ea87499d2c855 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
@@ -59,6 +59,7 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV32-NEXT:    sw a1, 12(sp)
 ; RV32-NEXT:    lw a0, 0(a0)
 ; RV32-NEXT:    addi sp, sp, 48
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va1:
@@ -84,6 +85,7 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV64-NEXT:    sw a2, 12(sp)
 ; RV64-NEXT:    lw a0, 0(a0)
 ; RV64-NEXT:    addi sp, sp, 80
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va1:
@@ -111,7 +113,10 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 48
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va1:
@@ -144,7 +149,10 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 96
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %va = alloca ptr
   call void @llvm.va_start(ptr %va)
@@ -1588,6 +1596,7 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV32-NEXT:    lui a1, 24414
 ; RV32-NEXT:    addi a1, a1, 304
 ; RV32-NEXT:    add sp, sp, a1
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_large_stack:
@@ -1633,6 +1642,7 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV64-NEXT:    lui a1, 24414
 ; RV64-NEXT:    addiw a1, a1, 336
 ; RV64-NEXT:    add sp, sp, a1
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_large_stack:
@@ -1667,9 +1677,13 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    lui a1, 24414
 ; RV32-WITHFP-NEXT:    addi a1, a1, -1728
 ; RV32-WITHFP-NEXT:    add sp, sp, a1
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 2032
 ; RV32-WITHFP-NEXT:    lw ra, 1996(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 1992(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 2032
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_large_stack:
@@ -1709,9 +1723,13 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    lui a1, 24414
 ; RV64-WITHFP-NEXT:    addiw a1, a1, -1680
 ; RV64-WITHFP-NEXT:    add sp, sp, a1
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 2032
 ; RV64-WITHFP-NEXT:    ld ra, 1960(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 1952(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 2032
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %large = alloca [ 100000000 x i8 ]
   %va = alloca ptr
@@ -1739,6 +1757,7 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV32-NEXT:    sw a1, 8(sp)
 ; RV32-NEXT:    lw a0, 0(a0)
 ; RV32-NEXT:    addi sp, sp, 16
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_vprintf:
@@ -1755,6 +1774,7 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV64-NEXT:    sd a1, 0(sp)
 ; RV64-NEXT:    ld a0, 0(a0)
 ; RV64-NEXT:    addi sp, sp, 16
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_vprintf:
@@ -1778,7 +1798,10 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV32-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 16
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_vprintf:
@@ -1802,7 +1825,10 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV64-WITHFP-NEXT:    ld a0, 0(a0)
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 32
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %args = alloca ptr
   %args_cp = alloca ptr
@@ -1832,7 +1858,9 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV32-NEXT:    sw a7, 44(sp)
 ; RV32-NEXT:    call va_vprintf
 ; RV32-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
+; RV32-NEXT:    .cfi_restore ra
 ; RV32-NEXT:    addi sp, sp, 48
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_printf:
@@ -1853,7 +1881,9 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV64-NEXT:    sd a7, 72(sp)
 ; RV64-NEXT:    call va_vprintf
 ; RV64-NEXT:    ld ra, 8(sp) # 8-byte Folded Reload
+; RV64-NEXT:    .cfi_restore ra
 ; RV64-NEXT:    addi sp, sp, 80
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_printf:
@@ -1879,7 +1909,10 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    call va_vprintf
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 48
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_printf:
@@ -1905,7 +1938,10 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    call va_vprintf
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 96
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %args = alloca ptr
   call void @llvm.va_start(ptr %args)
diff --git a/llvm/test/CodeGen/RISCV/addrspacecast.ll b/llvm/test/CodeGen/RISCV/addrspacecast.ll
index e55a57a5167822..80a0efb043ebdf 100644
--- a/llvm/test/CodeGen/RISCV/addrspacecast.ll
+++ b/llvm/test/CodeGen/RISCV/addrspacecast.ll
@@ -28,7 +28,9 @@ define void @cast1(ptr %ptr) {
 ; RV32I-NEXT:    .cfi_offset ra, -4
 ; RV32I-NEXT:    call foo
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
 ; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV64I-LABEL: cast1:
@@ -39,7 +41,9 @@ define void @cast1(ptr %ptr) {
 ; RV64I-NEXT:    .cfi_offset ra, -8
 ; RV64I-NEXT:    call foo
 ; RV64I-NEXT:    ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
 ; RV64I-NEXT:    addi sp, sp, 16
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
   %castptr = addrspacecast ptr %ptr to ptr addrspace(10)
   call void @foo(ptr addrspace(10) %castptr)
diff --git a/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll b/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
index 0d6ae3a51e2469..f22115130117a8 100644
--- a/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
+++ b/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
@@ -51,7 +51,12 @@ define i8 @atomicrmw_usub_cond_i8(ptr %ptr, i8 %val) {
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i8:
@@ -129,7 +134,12 @@ define i8 @atomicrmw_usub_cond_i8(ptr %ptr, i8 %val) {
 ; RV64I-NEXT:    ld s0, 32(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i8:
@@ -216,7 +226,13 @@ define i16 @atomicrmw_usub_cond_i16(ptr %ptr, i16 %val) {
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
+; RV32I-NEXT:    .cfi_restore s3
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i16:
@@ -300,7 +316,13 @@ define i16 @atomicrmw_usub_cond_i16(ptr %ptr, i16 %val) {
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s3, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
+; RV64I-NEXT:    .cfi_restore s3
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i16:
@@ -378,7 +400,11 @@ define i32 @atomicrmw_usub_cond_i32(ptr %ptr, i32 %val) {
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 4(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
 ; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i32:
@@ -442,7 +468,12 @@ define i32 @atomicrmw_usub_cond_i32(ptr %ptr, i32 %val) {
 ; RV64I-NEXT:    ld s0, 32(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i32:
@@ -529,7 +560,12 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i64:
@@ -586,7 +622,12 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV32IA-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32IA-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32IA-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32IA-NEXT:    .cfi_restore ra
+; RV32IA-NEXT:    .cfi_restore s0
+; RV32IA-NEXT:    .cfi_restore s1
+; RV32IA-NEXT:    .cfi_restore s2
 ; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    .cfi_def_cfa_offset 0
 ; RV32IA-NEXT:    ret
 ;
 ; RV64I-LABEL: atomicrmw_usub_cond_i64:
@@ -621,7 +662,11 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV64I-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
 ; RV64I-NEXT:    addi sp, sp, 32
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i64:
@@ -686,7 +731,11 @@ define i8 @atomicrmw_usub_sat_i8(ptr %ptr, i8 %val) {
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    ...
[truncated]

@llvmbot
Copy link
Member

llvmbot commented Oct 2, 2024

@llvm/pr-subscribers-backend-risc-v

Author: None (dlav-sc)

Changes

This patch adds CFI instructions in a function epilogue, that allows lldb to obtain a valid backtrace at the end of functions.


Patch is 1004.05 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/110810.diff

296 Files Affected:

  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.cpp (+72)
  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.h (+2)
  • (modified) llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll (+36)
  • (modified) llvm/test/CodeGen/RISCV/addrspacecast.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll (+85)
  • (modified) llvm/test/CodeGen/RISCV/atomicrmw-uinc-udec-wrap.ll (+85)
  • (modified) llvm/test/CodeGen/RISCV/branch-relaxation.ll (+144)
  • (modified) llvm/test/CodeGen/RISCV/callee-saved-gprs.ll (+88-8)
  • (modified) llvm/test/CodeGen/RISCV/calling-conv-ilp32e.ll (+271)
  • (modified) llvm/test/CodeGen/RISCV/cm_mvas_mvsa.ll (+8-4)
  • (modified) llvm/test/CodeGen/RISCV/double-intrinsics.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/double-round-conv.ll (+50)
  • (modified) llvm/test/CodeGen/RISCV/early-clobber-tied-def-subreg-liveness.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/eh-dwarf-cfa.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/exception-pointer-register.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/float-intrinsics.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/float-round-conv.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/fpclamptosat.ll (+176)
  • (modified) llvm/test/CodeGen/RISCV/frame-info.ll (+32)
  • (modified) llvm/test/CodeGen/RISCV/half-convert-strict.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/half-intrinsics.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/half-round-conv.ll (+80)
  • (modified) llvm/test/CodeGen/RISCV/hwasan-check-memaccess.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/intrinsic-cttz-elts-vscale.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/kcfi-mir.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/large-stack.ll (+15)
  • (modified) llvm/test/CodeGen/RISCV/live-sp.mir (+2-1)
  • (modified) llvm/test/CodeGen/RISCV/llvm.exp10.ll (+120)
  • (modified) llvm/test/CodeGen/RISCV/local-stack-slot-allocation.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/lpad.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/miss-sp-restore-eh.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/nontemporal.ll (+60)
  • (modified) llvm/test/CodeGen/RISCV/overflow-intrinsics.ll (+23)
  • (modified) llvm/test/CodeGen/RISCV/pr58025.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/pr58286.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/pr63365.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/pr69586.ll (+29)
  • (modified) llvm/test/CodeGen/RISCV/pr88365.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/prolog-epilogue.ll (+80)
  • (modified) llvm/test/CodeGen/RISCV/push-pop-opt-crash.ll (+27-25)
  • (modified) llvm/test/CodeGen/RISCV/push-pop-popret.ll (+344-44)
  • (modified) llvm/test/CodeGen/RISCV/regalloc-last-chance-recoloring-failure.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rv64-patchpoint.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rv64-statepoint-call-lowering.ll (+21)
  • (modified) llvm/test/CodeGen/RISCV/rvv-cfi-info.ll (+18)
  • (modified) llvm/test/CodeGen/RISCV/rvv/abs-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/access-fixed-objects-by-rvv.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/addi-scalable-offset.mir (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-scalable-array.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-scalable-struct.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/alloca-load-store-vector-tuple.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/binop-splats.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bitreverse-sdnode.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bitreverse-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bswap-sdnode.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/bswap-vp.ll (+21)
  • (modified) llvm/test/CodeGen/RISCV/rvv/callee-saved-regs.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/calling-conv-fastcc.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/rvv/calling-conv.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/ceil-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/compressstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/ctpop-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/cttz-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-fp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv32.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv64.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vector-i8-index-cornercase.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-binop-splats.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitcast.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitreverse-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitreverse.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bswap-vp.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bswap.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-calling-conv-fastcc.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-calling-conv.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ceil-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctlz-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctpop-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-cttz-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-expandload-int.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-floor-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fmaximum-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fminimum-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec-bf16.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec.ll (+100)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-interleave.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fshr-fshl-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-insert-subvector.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-insert.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-buildvec.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-explodevector.ll (+29)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-interleave.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-splat.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-interleaved-access.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-llrint.ll (+43)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-lrint.ll (+28)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-masked-gather.ll (+65)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-masked-scatter.ll (+59)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-fp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-int-vp.ll (+44)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-int.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-rint-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-round-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-roundeven-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-roundtozero-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-setcc-fp-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-setcc-int-vp.ll (+22)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-shuffle-vslide1down.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-shuffle-vslide1up.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-trunc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vaaddu.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vand-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vcopysign-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vdiv-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vdivu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfma-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmax-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmin-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfmuladd-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwadd.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwmaccbf16.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwmul.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vfwsub.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vitofp-constrained-sdnode.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmacc-vp.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmaxu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vminu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vnmsac-vp.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vp-splat.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vpmerge.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vpscatter.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vrem-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vremu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vrsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsadd.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsaddu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsaddu.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vscale-range.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vselect-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssub.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssubu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vssubu.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmul.ll (+7)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmulsu.ll (+9)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vwmulu.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vxor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/floor-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fmaximum-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fmaximum-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fminimum-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fminimum-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fnearbyint-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fpclamptosat_vec.ll (+432)
  • (modified) llvm/test/CodeGen/RISCV/rvv/fshr-fshl-vp.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/get-vlen-debugloc.mir (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/known-never-zero.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/large-rvv-stack-size.mir (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/localvar.ll (+27)
  • (modified) llvm/test/CodeGen/RISCV/rvv/memory-args.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/mgather-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/mscatter-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/named-vector-shuffle-reverse.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/nearbyint-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/no-reserved-frame.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr104480.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr88576.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr93587.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/pr95865.ll (+14)
  • (modified) llvm/test/CodeGen/RISCV/rvv/reg-alloc-reserve-bp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/remat.ll (+18)
  • (modified) llvm/test/CodeGen/RISCV/rvv/rint-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/round-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/roundeven-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/roundtozero-vp.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/rvv/rvv-args-by-mem.ll (+5)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-fp-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-int-vp.ll (+40)
  • (modified) llvm/test/CodeGen/RISCV/rvv/setcc-integer.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/rvv/splat-vector-split-i64-vl-sdnode.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/stack-folding.ll (+24)
  • (modified) llvm/test/CodeGen/RISCV/rvv/stepvector.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/strided-vpstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/undef-earlyclobber-chain.ll (+1)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vaaddu-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vand-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vand-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vandn-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vandn-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdiv-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdiv-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdivu-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vdivu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-deinterleave-load.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-deinterleave.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-interleave-store.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vector-interleave.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfadd-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfdiv-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfma-vp.ll (+124)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmadd-constrained-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmadd-sdnode.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmsub-constrained-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfmuladd-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfnmadd-constrained-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfnmsub-constrained-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptosi-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptoui-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfptrunc-vp.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfsub-vp.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwmacc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwnmacc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vfwnmsac-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmacc-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmadd-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmax-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmax-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmaxu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmaxu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmin-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmin-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vminu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vminu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmul-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vmul-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vnmsac-vp.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vnmsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vor-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vp-reverse-int.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vp-splat.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpmerge-sdnode.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpscatter-sdnode.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vpstore.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vreductions-int-vp.ll (+30)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrem-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrem-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vremu-sdnode.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vremu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vrsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsadd-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsadd-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsaddu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsaddu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vselect-int.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vselect-vp.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsitofp-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsplats-i64.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssubu-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vssubu-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsub-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vsub-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vtrunc-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vuitofp-vp.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vwadd-sdnode.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vxor-sdnode.ll (+4)
  • (modified) llvm/test/CodeGen/RISCV/rvv/vxor-vp.ll (+8)
  • (modified) llvm/test/CodeGen/RISCV/rvv/zvlsseg-spill.mir (+2)
  • (modified) llvm/test/CodeGen/RISCV/shadowcallstack.ll (+28)
  • (modified) llvm/test/CodeGen/RISCV/shl-cttz.ll (+16)
  • (modified) llvm/test/CodeGen/RISCV/shrinkwrap-jump-table.ll (+2)
  • (modified) llvm/test/CodeGen/RISCV/stack-inst-compress.mir (+36)
  • (modified) llvm/test/CodeGen/RISCV/stack-offset.ll (+48)
  • (modified) llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll (+20)
  • (modified) llvm/test/CodeGen/RISCV/stack-realignment.ll (+212)
  • (modified) llvm/test/CodeGen/RISCV/vararg-ilp32e.ll (+12)
  • (modified) llvm/test/CodeGen/RISCV/vararg.ll (+38)
  • (modified) llvm/test/CodeGen/RISCV/vlenb.ll (+3)
  • (modified) llvm/test/CodeGen/RISCV/xaluo.ll (+27)
  • (modified) llvm/test/CodeGen/RISCV/xcvbi.ll (+6)
  • (modified) llvm/test/CodeGen/RISCV/zbb-cmp-combine.ll (+10)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-additional-stack.ll (+7-1)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-cm-popretz.mir (+42-4)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-cm-push-pop.mir (+82)
  • (modified) llvm/test/CodeGen/RISCV/zcmp-with-float.ll (+18-2)
  • (modified) llvm/test/CodeGen/RISCV/zdinx-large-spill.mir (+1)
  • (modified) llvm/test/DebugInfo/RISCV/relax-debug-frame.ll (+15-7)
diff --git a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
index 7cbd1a35b25839..78d3dafaf9c8e2 100644
--- a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
@@ -761,11 +761,18 @@ void RISCVFrameLowering::deallocateStack(MachineFunction &MF,
                                          const DebugLoc &DL, uint64_t StackSize,
                                          int64_t CFAOffset) const {
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
+  const RISCVInstrInfo *TII = STI.getInstrInfo();
 
   Register SPReg = getSPReg(STI);
 
   RI->adjustReg(MBB, MBBI, DL, SPReg, SPReg, StackOffset::getFixed(StackSize),
                 MachineInstr::FrameDestroy, getStackAlign());
+
+  unsigned CFIIndex =
+      MF.addFrameInst(MCCFIInstruction::cfiDefCfaOffset(nullptr, CFAOffset));
+  BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+      .addCFIIndex(CFIIndex)
+      .setMIFlag(MachineInstr::FrameDestroy);
 }
 
 void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
@@ -773,6 +780,7 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
+  const RISCVInstrInfo *TII = STI.getInstrInfo();
   Register FPReg = getFPReg(STI);
   Register SPReg = getSPReg(STI);
 
@@ -826,7 +834,16 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
     if (!RestoreFP) {
       adjustStackForRVV(MF, MBB, LastFrameDestroy, DL, RVVStackSize,
                         MachineInstr::FrameDestroy);
+
+      unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::cfiDefCfa(
+          nullptr, RI->getDwarfRegNum(SPReg, true), RealStackSize));
+      BuildMI(MBB, LastFrameDestroy, DL,
+              TII->get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
     }
+
+    emitCalleeSavedRVVEpilogCFI(MBB, LastFrameDestroy);
   }
 
   if (FirstSPAdjustAmount) {
@@ -841,6 +858,13 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
       RI->adjustReg(MBB, LastFrameDestroy, DL, SPReg, SPReg,
                     StackOffset::getFixed(SecondSPAdjustAmount),
                     MachineInstr::FrameDestroy, getStackAlign());
+
+      unsigned CFIIndex = MF.addFrameInst(
+          MCCFIInstruction::cfiDefCfaOffset(nullptr, FirstSPAdjustAmount));
+      BuildMI(MBB, LastFrameDestroy, DL,
+              TII->get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
     }
   }
 
@@ -858,6 +882,12 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
     RI->adjustReg(MBB, LastFrameDestroy, DL, SPReg, FPReg,
                   StackOffset::getFixed(-FPOffset), MachineInstr::FrameDestroy,
                   getStackAlign());
+
+    unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::cfiDefCfa(
+        nullptr, RI->getDwarfRegNum(SPReg, true), RealStackSize));
+    BuildMI(MBB, LastFrameDestroy, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
   }
 
   bool ApplyPop = RVFI->isPushable(MF) && MBBI != MBB.end() &&
@@ -875,7 +905,24 @@ void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
       deallocateStack(MF, MBB, MBBI, DL, StackSize,
                       /*stack_adj of cm.pop instr*/ RealStackSize - StackSize);
 
+    // Update CFA offset. After CM_POP SP should be equal to CFA, so CFA offset
+    // is zero.
     MBBI = std::next(MBBI);
+    unsigned CFIIndex =
+        MF.addFrameInst(MCCFIInstruction::cfiDefCfaOffset(nullptr, 0));
+    BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
+  }
+
+  // Recover callee-saved registers.
+  for (const auto &Entry : CSI) {
+    Register Reg = Entry.getReg();
+    unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::createRestore(
+        nullptr, RI->getDwarfRegNum(Reg, true)));
+    BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
+        .addCFIIndex(CFIIndex)
+        .setMIFlag(MachineInstr::FrameDestroy);
   }
 
   // Deallocate stack if StackSize isn't a zero and if we didn't already do it
@@ -1615,6 +1662,31 @@ void RISCVFrameLowering::emitCalleeSavedRVVPrologCFI(
   }
 }
 
+void RISCVFrameLowering::emitCalleeSavedRVVEpilogCFI(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MI) const {
+  MachineFunction *MF = MBB.getParent();
+  const MachineFrameInfo &MFI = MF->getFrameInfo();
+  const RISCVRegisterInfo *RI = STI.getRegisterInfo();
+  const TargetInstrInfo &TII = *STI.getInstrInfo();
+  DebugLoc DL = MBB.findDebugLoc(MI);
+
+  const auto &RVVCSI = getRVVCalleeSavedInfo(*MF, MFI.getCalleeSavedInfo());
+  if (RVVCSI.empty())
+    return;
+
+  for (auto &CS : RVVCSI) {
+    int FI = CS.getFrameIdx();
+    if (FI >= 0 && MFI.getStackID(FI) == TargetStackID::ScalableVector) {
+      Register Reg = CS.getReg();
+      unsigned CFIIndex = MF->addFrameInst(MCCFIInstruction::createRestore(
+          nullptr, RI->getDwarfRegNum(Reg, true)));
+      BuildMI(MBB, MI, DL, TII.get(TargetOpcode::CFI_INSTRUCTION))
+          .addCFIIndex(CFIIndex)
+          .setMIFlag(MachineInstr::FrameDestroy);
+    }
+  }
+}
+
 bool RISCVFrameLowering::restoreCalleeSavedRegisters(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MI,
     MutableArrayRef<CalleeSavedInfo> CSI, const TargetRegisterInfo *TRI) const {
diff --git a/llvm/lib/Target/RISCV/RISCVFrameLowering.h b/llvm/lib/Target/RISCV/RISCVFrameLowering.h
index 89f95f2aa04aa6..68402bf9d81478 100644
--- a/llvm/lib/Target/RISCV/RISCVFrameLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVFrameLowering.h
@@ -92,6 +92,8 @@ class RISCVFrameLowering : public TargetFrameLowering {
   void emitCalleeSavedRVVPrologCFI(MachineBasicBlock &MBB,
                                    MachineBasicBlock::iterator MI,
                                    bool HasFP) const;
+  void emitCalleeSavedRVVEpilogCFI(MachineBasicBlock &MBB,
+                                   MachineBasicBlock::iterator MI) const;
 
   void deallocateStack(MachineFunction &MF, MachineBasicBlock &MBB,
                        MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll b/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
index 014283cf38b26b..caa749729ce198 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/stacksave-stackrestore.ll
@@ -25,10 +25,15 @@ define void @test_scoped_alloca(i64 %n) {
 ; RV32-NEXT:    call use_addr
 ; RV32-NEXT:    mv sp, s1
 ; RV32-NEXT:    addi sp, s0, -16
+; RV32-NEXT:    .cfi_def_cfa sp, 16
 ; RV32-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32-NEXT:    lw s1, 4(sp) # 4-byte Folded Reload
+; RV32-NEXT:    .cfi_restore ra
+; RV32-NEXT:    .cfi_restore s0
+; RV32-NEXT:    .cfi_restore s1
 ; RV32-NEXT:    addi sp, sp, 16
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: test_scoped_alloca:
@@ -51,10 +56,15 @@ define void @test_scoped_alloca(i64 %n) {
 ; RV64-NEXT:    call use_addr
 ; RV64-NEXT:    mv sp, s1
 ; RV64-NEXT:    addi sp, s0, -32
+; RV64-NEXT:    .cfi_def_cfa sp, 32
 ; RV64-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s1, 8(sp) # 8-byte Folded Reload
+; RV64-NEXT:    .cfi_restore ra
+; RV64-NEXT:    .cfi_restore s0
+; RV64-NEXT:    .cfi_restore s1
 ; RV64-NEXT:    addi sp, sp, 32
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
   %sp = call ptr @llvm.stacksave.p0()
   %addr = alloca i8, i64 %n
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll b/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
index 01cab0d0e157bd..5ea87499d2c855 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/vararg.ll
@@ -59,6 +59,7 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV32-NEXT:    sw a1, 12(sp)
 ; RV32-NEXT:    lw a0, 0(a0)
 ; RV32-NEXT:    addi sp, sp, 48
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va1:
@@ -84,6 +85,7 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV64-NEXT:    sw a2, 12(sp)
 ; RV64-NEXT:    lw a0, 0(a0)
 ; RV64-NEXT:    addi sp, sp, 80
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va1:
@@ -111,7 +113,10 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 48
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va1:
@@ -144,7 +149,10 @@ define i32 @va1(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 96
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %va = alloca ptr
   call void @llvm.va_start(ptr %va)
@@ -1588,6 +1596,7 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV32-NEXT:    lui a1, 24414
 ; RV32-NEXT:    addi a1, a1, 304
 ; RV32-NEXT:    add sp, sp, a1
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_large_stack:
@@ -1633,6 +1642,7 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV64-NEXT:    lui a1, 24414
 ; RV64-NEXT:    addiw a1, a1, 336
 ; RV64-NEXT:    add sp, sp, a1
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_large_stack:
@@ -1667,9 +1677,13 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    lui a1, 24414
 ; RV32-WITHFP-NEXT:    addi a1, a1, -1728
 ; RV32-WITHFP-NEXT:    add sp, sp, a1
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 2032
 ; RV32-WITHFP-NEXT:    lw ra, 1996(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 1992(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 2032
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_large_stack:
@@ -1709,9 +1723,13 @@ define i32 @va_large_stack(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    lui a1, 24414
 ; RV64-WITHFP-NEXT:    addiw a1, a1, -1680
 ; RV64-WITHFP-NEXT:    add sp, sp, a1
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 2032
 ; RV64-WITHFP-NEXT:    ld ra, 1960(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 1952(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 2032
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %large = alloca [ 100000000 x i8 ]
   %va = alloca ptr
@@ -1739,6 +1757,7 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV32-NEXT:    sw a1, 8(sp)
 ; RV32-NEXT:    lw a0, 0(a0)
 ; RV32-NEXT:    addi sp, sp, 16
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_vprintf:
@@ -1755,6 +1774,7 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV64-NEXT:    sd a1, 0(sp)
 ; RV64-NEXT:    ld a0, 0(a0)
 ; RV64-NEXT:    addi sp, sp, 16
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_vprintf:
@@ -1778,7 +1798,10 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV32-WITHFP-NEXT:    lw a0, 0(a0)
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 16
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_vprintf:
@@ -1802,7 +1825,10 @@ define iXLen @va_vprintf(ptr %fmt, ptr %arg_start) {
 ; RV64-WITHFP-NEXT:    ld a0, 0(a0)
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 32
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %args = alloca ptr
   %args_cp = alloca ptr
@@ -1832,7 +1858,9 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV32-NEXT:    sw a7, 44(sp)
 ; RV32-NEXT:    call va_vprintf
 ; RV32-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
+; RV32-NEXT:    .cfi_restore ra
 ; RV32-NEXT:    addi sp, sp, 48
+; RV32-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: va_printf:
@@ -1853,7 +1881,9 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV64-NEXT:    sd a7, 72(sp)
 ; RV64-NEXT:    call va_vprintf
 ; RV64-NEXT:    ld ra, 8(sp) # 8-byte Folded Reload
+; RV64-NEXT:    .cfi_restore ra
 ; RV64-NEXT:    addi sp, sp, 80
+; RV64-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-NEXT:    ret
 ;
 ; RV32-WITHFP-LABEL: va_printf:
@@ -1879,7 +1909,10 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV32-WITHFP-NEXT:    call va_vprintf
 ; RV32-WITHFP-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32-WITHFP-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
+; RV32-WITHFP-NEXT:    .cfi_restore ra
+; RV32-WITHFP-NEXT:    .cfi_restore s0
 ; RV32-WITHFP-NEXT:    addi sp, sp, 48
+; RV32-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV32-WITHFP-NEXT:    ret
 ;
 ; RV64-WITHFP-LABEL: va_printf:
@@ -1905,7 +1938,10 @@ define i32 @va_printf(ptr %fmt, ...) {
 ; RV64-WITHFP-NEXT:    call va_vprintf
 ; RV64-WITHFP-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64-WITHFP-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64-WITHFP-NEXT:    .cfi_restore ra
+; RV64-WITHFP-NEXT:    .cfi_restore s0
 ; RV64-WITHFP-NEXT:    addi sp, sp, 96
+; RV64-WITHFP-NEXT:    .cfi_def_cfa_offset 0
 ; RV64-WITHFP-NEXT:    ret
   %args = alloca ptr
   call void @llvm.va_start(ptr %args)
diff --git a/llvm/test/CodeGen/RISCV/addrspacecast.ll b/llvm/test/CodeGen/RISCV/addrspacecast.ll
index e55a57a5167822..80a0efb043ebdf 100644
--- a/llvm/test/CodeGen/RISCV/addrspacecast.ll
+++ b/llvm/test/CodeGen/RISCV/addrspacecast.ll
@@ -28,7 +28,9 @@ define void @cast1(ptr %ptr) {
 ; RV32I-NEXT:    .cfi_offset ra, -4
 ; RV32I-NEXT:    call foo
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
 ; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV64I-LABEL: cast1:
@@ -39,7 +41,9 @@ define void @cast1(ptr %ptr) {
 ; RV64I-NEXT:    .cfi_offset ra, -8
 ; RV64I-NEXT:    call foo
 ; RV64I-NEXT:    ld ra, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
 ; RV64I-NEXT:    addi sp, sp, 16
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
   %castptr = addrspacecast ptr %ptr to ptr addrspace(10)
   call void @foo(ptr addrspace(10) %castptr)
diff --git a/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll b/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
index 0d6ae3a51e2469..f22115130117a8 100644
--- a/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
+++ b/llvm/test/CodeGen/RISCV/atomicrmw-cond-sub-clamp.ll
@@ -51,7 +51,12 @@ define i8 @atomicrmw_usub_cond_i8(ptr %ptr, i8 %val) {
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i8:
@@ -129,7 +134,12 @@ define i8 @atomicrmw_usub_cond_i8(ptr %ptr, i8 %val) {
 ; RV64I-NEXT:    ld s0, 32(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i8:
@@ -216,7 +226,13 @@ define i16 @atomicrmw_usub_cond_i16(ptr %ptr, i16 %val) {
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
+; RV32I-NEXT:    .cfi_restore s3
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i16:
@@ -300,7 +316,13 @@ define i16 @atomicrmw_usub_cond_i16(ptr %ptr, i16 %val) {
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s3, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
+; RV64I-NEXT:    .cfi_restore s3
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i16:
@@ -378,7 +400,11 @@ define i32 @atomicrmw_usub_cond_i32(ptr %ptr, i32 %val) {
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 4(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
 ; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i32:
@@ -442,7 +468,12 @@ define i32 @atomicrmw_usub_cond_i32(ptr %ptr, i32 %val) {
 ; RV64I-NEXT:    ld s0, 32(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s2, 16(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
+; RV64I-NEXT:    .cfi_restore s2
 ; RV64I-NEXT:    addi sp, sp, 48
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i32:
@@ -529,7 +560,12 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32I-NEXT:    .cfi_restore ra
+; RV32I-NEXT:    .cfi_restore s0
+; RV32I-NEXT:    .cfi_restore s1
+; RV32I-NEXT:    .cfi_restore s2
 ; RV32I-NEXT:    addi sp, sp, 32
+; RV32I-NEXT:    .cfi_def_cfa_offset 0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IA-LABEL: atomicrmw_usub_cond_i64:
@@ -586,7 +622,12 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV32IA-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32IA-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32IA-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload
+; RV32IA-NEXT:    .cfi_restore ra
+; RV32IA-NEXT:    .cfi_restore s0
+; RV32IA-NEXT:    .cfi_restore s1
+; RV32IA-NEXT:    .cfi_restore s2
 ; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    .cfi_def_cfa_offset 0
 ; RV32IA-NEXT:    ret
 ;
 ; RV64I-LABEL: atomicrmw_usub_cond_i64:
@@ -621,7 +662,11 @@ define i64 @atomicrmw_usub_cond_i64(ptr %ptr, i64 %val) {
 ; RV64I-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s1, 8(sp) # 8-byte Folded Reload
+; RV64I-NEXT:    .cfi_restore ra
+; RV64I-NEXT:    .cfi_restore s0
+; RV64I-NEXT:    .cfi_restore s1
 ; RV64I-NEXT:    addi sp, sp, 32
+; RV64I-NEXT:    .cfi_def_cfa_offset 0
 ; RV64I-NEXT:    ret
 ;
 ; RV64IA-LABEL: atomicrmw_usub_cond_i64:
@@ -686,7 +731,11 @@ define i8 @atomicrmw_usub_sat_i8(ptr %ptr, i8 %val) {
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    ...
[truncated]

@dlav-sc
Copy link
Contributor Author

dlav-sc commented Oct 2, 2024

@topperc @kito-cheng FYI

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch from 0573a78 to 46e215f Compare October 2, 2024 10:21
@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-sp-recovery branch 5 times, most recently from a58cf3f to ac123f9 Compare October 2, 2024 11:40
@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch 3 times, most recently from e7e68ef to 19d29c9 Compare October 2, 2024 12:37
Copy link

github-actions bot commented Oct 2, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch 3 times, most recently from db0616b to 0c83e7d Compare October 3, 2024 00:14
@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-sp-recovery branch from ac123f9 to 536cbea Compare October 3, 2024 00:25
@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch from 0c83e7d to 8c86fc2 Compare October 3, 2024 00:28
Base automatically changed from users/dlav-sc/riscv-sp-recovery to main October 4, 2024 09:22
@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch from 8c86fc2 to 2d7be29 Compare October 4, 2024 09:25
@dlav-sc
Copy link
Contributor Author

dlav-sc commented Oct 15, 2024

@kito-cheng @topperc, could you take another look, please?

@lenary
Copy link
Member

lenary commented Oct 15, 2024

Sorry, those three comments were supposed to come together, like a review. Hopefully they are helpful though. Please test looking at the frame variable information (or equivalent in LLDB) at all instruction boundaries in the epilog.

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch 2 times, most recently from 9305f19 to 30c2634 Compare October 31, 2024 20:34
@lenary
Copy link
Member

lenary commented Nov 1, 2024

What changed since I reviewed it? every time you force push it's quite difficult to work out what changed, did you just rebase?

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch from 30c2634 to d7f70f1 Compare November 1, 2024 14:26
@dlav-sc
Copy link
Contributor Author

dlav-sc commented Nov 1, 2024

What changed since I reviewed it? every time you force push it's quite difficult to work out what changed, did you just rebase?

Yep, I've just made a rebase and resolved conflicts due to 18f0f70

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch 2 times, most recently from 7ce99db to 9631627 Compare November 5, 2024 15:49
Copy link
Member

@kito-cheng kito-cheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well :)

@dlav-sc dlav-sc force-pushed the users/dlav-sc/riscv-cfi-epilog branch from 9631627 to a2d62de Compare November 5, 2024 16:11
@lenary
Copy link
Member

lenary commented Nov 5, 2024

I'm not concerned by the windows failures - they look unrelated.

@dlav-sc
Copy link
Contributor Author

dlav-sc commented Nov 5, 2024

I'm not concerned by the windows failures - they look unrelated.

If linux testing will pass, I'm going to merge this tomorrow. Sorry for the inconvenience (the testing is quite unstable these days)

@dlav-sc dlav-sc merged commit 97982a8 into main Nov 5, 2024
6 of 8 checks passed
@dlav-sc dlav-sc deleted the users/dlav-sc/riscv-cfi-epilog branch November 5, 2024 21:20
@dlav-sc
Copy link
Contributor Author

dlav-sc commented Nov 5, 2024

@lenary, @kito-cheng, thank you for the review!

@lenary
Copy link
Member

lenary commented Nov 5, 2024

No worries, sorry it took so long to get over the line. Thanks for the contribution, and for taking a look at all the weird cases I found.

dlav-sc added a commit that referenced this pull request Nov 18, 2024
This patch refactor PR #110810
to remove code duplication.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants