Skip to content

[NFC]In codegen pipeline, turn static-data-splitter pass on/off with its own option #134752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 18 additions & 13 deletions llvm/lib/CodeGen/TargetPassConfig.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1235,13 +1235,9 @@ void TargetPassConfig::addMachinePasses() {
addPass(createMIRAddFSDiscriminatorsPass(
sampleprof::FSDiscriminatorPass::PassLast));

// Machine function splitter uses the basic block sections feature.
// When used along with `-basic-block-sections=`, the basic-block-sections
// feature takes precedence. This means functions eligible for
// basic-block-sections optimizations (`=all`, or `=list=` with function
// included in the list profile) will get that optimization instead.
if (TM->Options.EnableMachineFunctionSplitter ||
EnableMachineFunctionSplitter) {
EnableMachineFunctionSplitter || SplitStaticData ||
TM->Options.EnableStaticDataPartitioning) {
const std::string ProfileFile = getFSProfileFile(TM);
if (!ProfileFile.empty()) {
if (EnableFSDiscriminator) {
Expand All @@ -1256,14 +1252,23 @@ void TargetPassConfig::addMachinePasses() {
"performance.\n";
}
}
}

// Machine function splitter uses the basic block sections feature.
// When used along with `-basic-block-sections=`, the basic-block-sections
// feature takes precedence. This means functions eligible for
// basic-block-sections optimizations (`=all`, or `=list=` with function
// included in the list profile) will get that optimization instead.
if (TM->Options.EnableMachineFunctionSplitter ||
EnableMachineFunctionSplitter)
addPass(createMachineFunctionSplitterPass());
if (SplitStaticData || TM->Options.EnableStaticDataPartitioning) {
// The static data splitter pass is a machine function pass. and
// static data annotator pass is a module-wide pass. See the file comment
// in StaticDataAnnotator.cpp for the motivation.
addPass(createStaticDataSplitterPass());
addPass(createStaticDataAnnotatorPass());
}

if (SplitStaticData || TM->Options.EnableStaticDataPartitioning) {
// The static data splitter pass is a machine function pass. and
// static data annotator pass is a module-wide pass. See the file comment
// in StaticDataAnnotator.cpp for the motivation.
addPass(createStaticDataSplitterPass());
addPass(createStaticDataAnnotatorPass());
}
// We run the BasicBlockSections pass if either we need BB sections or BB
// address map (or both).
Expand Down
10 changes: 4 additions & 6 deletions llvm/test/CodeGen/AArch64/constant-pool-partition.ll
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
; RUN: llc -mtriple=aarch64 -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true \
; RUN: -unique-section-names=false \
; RUN: llc -mtriple=aarch64 -partition-static-data-sections \
; RUN: -function-sections -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always

; Repeat the RUN command above for big-endian systems.
; RUN: llc -mtriple=aarch64_be -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true \
; RUN: -unique-section-names=false \
; RUN: llc -mtriple=aarch64_be -partition-static-data-sections \
; RUN: -function-sections -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always

; Tests that constant pool hotness is aggregated across the module. The
Expand Down
14 changes: 6 additions & 8 deletions llvm/test/CodeGen/X86/constant-pool-partition.ll
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,16 @@ target triple = "x86_64-grtev4-linux-gnu"
; 2. Similarly if a constant is accessed by both cold function and un-profiled
; function, constant pools for this constant should not have .unlikely suffix.

; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true -data-sections=true \
; RUN: -unique-section-names=false \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections -data-sections -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always

; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true -data-sections=true \
; RUN: -unique-section-names=true \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections -data-sections -unique-section-names \
; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always

; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=false -data-sections=false \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections=false -data-sections=false \
; RUN: -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --dump-input=always

Expand Down
36 changes: 18 additions & 18 deletions llvm/test/CodeGen/X86/global-variable-partition.ll
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,22 @@ target triple = "x86_64-unknown-linux-gnu"

; This RUN command sets `-data-sections=true -unique-section-names=true` so data
; sections are uniqufied by numbers.
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -data-sections=true \
; RUN: -unique-section-names=true -relocation-model=pic \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -relocation-model=pic \
; RUN: -partition-static-data-sections=true \
; RUN: -data-sections=true -unique-section-names=true \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=SYM,COMMON --dump-input=always

; This RUN command sets `-data-sections=true -unique-section-names=false` so
; data sections are uniqufied by variable names.
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -data-sections=true \
; RUN: -unique-section-names=false -relocation-model=pic \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -relocation-model=pic \
; RUN: -partition-static-data-sections=true \
; RUN: -data-sections=true -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=UNIQ,COMMON --dump-input=always

; This RUN command sets `-data-sections=false -unique-section-names=false`.
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -data-sections=false \
; RUN: -unique-section-names=false -relocation-model=pic \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -relocation-model=pic \
; RUN: -partition-static-data-sections=true \
; RUN: -data-sections=false -unique-section-names=false \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=AGG,COMMON --dump-input=always

; For @.str and @.str.1
Expand All @@ -42,19 +42,19 @@ target triple = "x86_64-unknown-linux-gnu"
; For @hot_relro_array
; COMMON: .type hot_relro_array,@object
; SYM-NEXT: .section .data.rel.ro.hot.hot_relro_array
; UNIQ-NEXT: .section .data.rel.ro.hot.,"aw",@progbits,unique,3
; UNIQ-NEXT: .section .data.rel.ro.hot.,"aw",@progbits,unique,1
; AGG-NEXT: .section .data.rel.ro.hot.,"aw",@progbits

; For @hot_data, which is accessed by {cold_func, unprofiled_func, hot_func}.
; COMMON: .type hot_data,@object
; SYM-NEXT: .section .data.hot.hot_data,"aw",@progbits
; UNIQ-NEXT: .section .data.hot.,"aw",@progbits,unique,4
; UNIQ-NEXT: .section .data.hot.,"aw",@progbits,unique,2
; AGG-NEXT: .section .data.hot.,"aw",@progbits

; For @hot_bss, which is accessed by {unprofiled_func, hot_func}.
; COMMON: .type hot_bss,@object
; SYM-NEXT: .section .bss.hot.hot_bss,"aw",@nobits
; UNIQ-NEXT: .section .bss.hot.,"aw",@nobits,unique,5
; UNIQ-NEXT: .section .bss.hot.,"aw",@nobits,unique,3
; AGG-NEXT: .section .bss.hot.,"aw",@nobits

; For @.str.2
Expand All @@ -68,13 +68,13 @@ target triple = "x86_64-unknown-linux-gnu"
; For @cold_bss
; COMMON: .type cold_bss,@object
; SYM-NEXT: .section .bss.unlikely.cold_bss,"aw",@nobits
; UNIQ-NEXT: .section .bss.unlikely.,"aw",@nobits,unique,6
; UNIQ-NEXT: .section .bss.unlikely.,"aw",@nobits,unique,4
; AGG-NEXT: .section .bss.unlikely.,"aw",@nobits

; For @cold_data
; COMMON: .type cold_data,@object
; SYM-NEXT: .section .data.unlikely.cold_data,"aw",@progbits
; UNIQ-NEXT: .section .data.unlikely.,"aw",@progbits,unique,7
; UNIQ-NEXT: .section .data.unlikely.,"aw",@progbits,unique,5
; AGG-NEXT: .section .data.unlikely.,"aw",@progbits

; For @cold_data_custom_foo_section
Expand All @@ -87,7 +87,7 @@ target triple = "x86_64-unknown-linux-gnu"
; For @cold_relro_array
; COMMON: .type cold_relro_array,@object
; SYM-NEXT: .section .data.rel.ro.unlikely.cold_relro_array,"aw",@progbits
; UNIQ-NEXT: .section .data.rel.ro.unlikely.,"aw",@progbits,unique,8
; UNIQ-NEXT: .section .data.rel.ro.unlikely.,"aw",@progbits,unique,6
; AGG-NEXT: .section .data.rel.ro.unlikely.,"aw",@progbits

; Currently static-data-splitter only analyzes access from code.
Expand All @@ -97,19 +97,19 @@ target triple = "x86_64-unknown-linux-gnu"
; For @bss2
; COMMON: .type bss2,@object
; SYM-NEXT: .section .bss.unlikely.bss2,"aw",@nobits
; UNIQ-NEXT: .section .bss.unlikely.,"aw",@nobits,unique,9
; UNIQ-NEXT: .section .bss.unlikely.,"aw",@nobits,unique,7
; AGG-NEXT: .section .bss.unlikely.,"aw",@nobits

; For @data3
; COMMON: .type data3,@object
; SYM-NEXT: .section .data.unlikely.data3,"aw",@progbits
; UNIQ-NEXT: .section .data.unlikely.,"aw",@progbits,unique,10
; UNIQ-NEXT: .section .data.unlikely.,"aw",@progbits,unique,8
; AGG-NEXT: .section .data.unlikely.,"aw",@progbits

; For @data_with_unknown_hotness
; SYM: .type .Ldata_with_unknown_hotness,@object # @data_with_unknown_hotness
; SYM: .section .data..Ldata_with_unknown_hotness,"aw",@progbits
; UNIQ: .section .data,"aw",@progbits,unique,11
; UNIQ: .section .data,"aw",@progbits,unique,9
; The `.section` directive is omitted for .data with -unique-section-names=false.
; See MCSectionELF::shouldOmitSectionDirective for the implementation details.
; AGG: .data
Expand Down
18 changes: 9 additions & 9 deletions llvm/test/CodeGen/X86/jump-table-partition.ll
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,19 @@
; STAT: 2 static-data-splitter - Number of hot jump tables seen
; STAT: 2 static-data-splitter - Number of jump tables with unknown hotness

; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true \
; RUN: -min-jump-table-entries=2 -unique-section-names=false \
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections -unique-section-names=false \
; RUN: -min-jump-table-entries=2 \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=NUM,JT

; Section names will optionally have `.<func>` if -function-sections is enabled.
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=true \
; RUN: -min-jump-table-entries=2 %s -o - 2>&1 | FileCheck %s --check-prefixes=FUNC,JT
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections -min-jump-table-entries=2 \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=FUNC,JT

; RUN: llc -mtriple=x86_64-unknown-linux-gnu -enable-split-machine-functions \
; RUN: -partition-static-data-sections=true -function-sections=false \
; RUN: -min-jump-table-entries=2 %s -o - 2>&1 | FileCheck %s --check-prefixes=FUNCLESS,JT
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -partition-static-data-sections \
; RUN: -function-sections=false -min-jump-table-entries=2 \
; RUN: %s -o - 2>&1 | FileCheck %s --check-prefixes=FUNCLESS,JT

; In function @foo, the 2 switch instructions to jt0.* and jt1.* are placed in
; hot-prefixed sections, and the 2 switch instructions to jt2.* and jt3.* are
Expand Down