Skip to content

[BOLT] Register Linux kernel dynamic branch offsets #90677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 2, 2024
Merged

[BOLT] Register Linux kernel dynamic branch offsets #90677

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions bolt/lib/Rewrite/LinuxKernelRewriter.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1695,6 +1695,9 @@ Error LinuxKernelRewriter::readStaticKeysJumpTable() {
if (!BC.MIB->getSize(*Inst))
BC.MIB->setSize(*Inst, Size);

if (!BC.MIB->getOffset(*Inst))
BC.MIB->setOffset(*Inst, JumpAddress - BF->getAddress());

if (opts::LongJumpLabels)
BC.MIB->setSize(*Inst, 5);
}
Expand Down
29 changes: 23 additions & 6 deletions bolt/test/X86/linux-static-keys.s
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
## Check that BOLT correctly updates the Linux kernel static keys jump table.

# RUN: llvm-mc -filetype=obj -triple x86_64-unknown-unknown %s -o %t.o
# RUN: link_fdata %s %t.o %t.fdata
# RUN: llvm-strip --strip-unneeded %t.o
# RUN: %clang %cflags -nostdlib %t.o -o %t.exe \
# RUN: -Wl,--image-base=0xffffffff80000000,--no-dynamic-linker,--no-eh-frame-hdr

Expand All @@ -11,6 +13,12 @@
# RUN: llvm-bolt %t.exe --print-normalized -o %t.out --keep-nops=0 \
# RUN: --bolt-info=0 |& FileCheck %s

## Verify that profile is matched correctly.

# RUN: llvm-bolt %t.exe --print-normalized -o %t.out --keep-nops=0 \
# RUN: --bolt-info=0 --data %t.fdata |& \
# RUN: FileCheck --check-prefix=CHECK-FDATA %s

## Verify the bindings again on the rewritten binary with nops removed.

# RUN: llvm-bolt %t.out -o %t.out.1 --print-normalized |& FileCheck %s
Expand All @@ -25,15 +33,24 @@ _start:
# CHECK: Binary Function "_start"
nop
.L0:
jmp .L1
jmp L1
# CHECK: jit
# CHECK-SAME: # ID: 1 {{.*}} # Likely: 0 # InitValue: 1
nop
.L1:
L1:
.nops 5
jmp .L0
# CHECK: jit
# CHECK-SAME: # ID: 2 {{.*}} # Likely: 1 # InitValue: 1
.L2:

## Check that a branch profile associated with a NOP is handled properly when
## dynamic branch is created.

# FDATA: 1 _start #L1# 1 _start #L2# 3 42
# CHECK-FDATA: jit {{.*}} # ID: 2
# CHECK-FDATA-NEXT: jmp
# CHECK-FDATA-NEXT: Successors: {{.*}} (mispreds: 3, count: 42)
L2:
nop
.size _start, .-_start

Expand All @@ -51,11 +68,11 @@ foo:
__start___jump_table:

.long .L0 - . # Jump address
.long .L1 - . # Target address
.long L1 - . # Target address
.quad 1 # Key address

.long .L1 - . # Jump address
.long .L2 - . # Target address
.long L1 - . # Jump address
.long L2 - . # Target address
.quad 0 # Key address

.globl __stop___jump_table
Expand Down