Skip to content

Commit a5ebe90

Browse files
red1bluelostred1bluelost
committed
Implements asm printer with tests.
Implements remaining relevant tests for asm-printer. Adds comma separation to the option. Decouples tests from block frequency value. Adds a check if the option is set without labels. Addresses initial feedback. Updates AsmPrinter code after rework to PGOAnalysisMap. Adds minor optimizations to avoid unnecessary analysis passes. Allows direct usage of immutable MBPI analysis. Updates after moving PGO analysis map after each function. Updates asm printer with encoded feature type.
1 parent 0d02ecc commit a5ebe90

8 files changed

+360
-12
lines changed

‎llvm/docs/Extensions.rst

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,6 +451,90 @@ Example:
451451
.uleb128 .LBB_END0_1-.LBB0_1 # BB_1 size
452452
.byte y # BB_1 metadata
453453
454+
PGO Analysis Map Extra Data
455+
"""""""""""""""""""""""""""
456+
457+
PGO related analysis data can be emitted after each function within the
458+
BBAddrMap through the optional ``pgo-analysis-map`` flag. Supported analyses
459+
currently are Function Entry Count, Basic Block Frequencies, and Branch
460+
Probabilities.
461+
462+
Each analysis is enabled or disabled via a bit in the feature byte. Currently
463+
those bits are:
464+
465+
#. Function Entry Count - Number of times the function was called as taken
466+
from a PGO profile. This will always be zero if PGO was not used or the
467+
function was not encountered in the profile.
468+
469+
#. Basic Block Frequencies - Encoded as raw block frequency value taken from
470+
MBFI analysis. This value is an integer that encodes the relative frequency
471+
compared to the entry block. More information can be found in
472+
'llvm/Support/BlockFrequency.h'.
473+
474+
#. Branch Probabilities - Encoded as raw numerator for branch probability
475+
taken from MBPI analysis. This value is the numerator for a fixed point ratio
476+
defined in 'llvm/Support/BranchProbability.h'. It indicates the probability
477+
that the block is followed by a given successor block during execution.
478+
479+
This extra data requires version 2 or above. This is necessary since successors
480+
of basic blocks won't know their index but will know their BB ID.
481+
482+
Example of BBAddrMap with PGO data:
483+
484+
.. code-block:: gas
485+
486+
.section ".llvm_bb_addr_map","",@llvm_bb_addr_map
487+
.byte 2 # version number
488+
.byte 7 # feature byte - PGO analyses enabled mask
489+
.quad .Lfunc_begin0 # address of the function
490+
.uleb128 4 # number of basic blocks
491+
# BB record for BB_0
492+
.uleb128 0 # BB_0 BB ID
493+
.uleb128 .Lfunc_begin0-.Lfunc_begin0 # BB_0 offset relative to function entry (always zero)
494+
.uleb128 .LBB_END0_0-.Lfunc_begin0 # BB_0 size
495+
.byte 0x18 # BB_0 metadata (multiple successors)
496+
# BB record for BB_1
497+
.uleb128 1 # BB_1 BB ID
498+
.uleb128 .LBB0_1-.LBB_END0_0 # BB_1 offset relative to the end of last block (BB_0).
499+
.uleb128 .LBB_END0_1-.LBB0_1 # BB_1 size
500+
.byte 0x0 # BB_1 metadata (two successors)
501+
# BB record for BB_2
502+
.uleb128 2 # BB_2 BB ID
503+
.uleb128 .LBB0_2-.LBB_END1_0 # BB_2 offset relative to the end of last block (BB_1).
504+
.uleb128 .LBB_END0_2-.LBB0_2 # BB_2 size
505+
.byte 0x0 # BB_2 metadata (one successor)
506+
# BB record for BB_3
507+
.uleb128 3 # BB_3 BB ID
508+
.uleb128 .LBB0_3-.LBB_END0_2 # BB_3 offset relative to the end of last block (BB_2).
509+
.uleb128 .LBB_END0_3-.LBB0_3 # BB_3 size
510+
.byte 0x0 # BB_3 metadata (zero successors)
511+
# PGO Analysis Map
512+
.uleb128 1000 # function entry count (only when enabled)
513+
# PGO data record for BB_0
514+
.uleb128 1000 # BB_0 basic block frequency (only when enabled)
515+
.uleb128 3 # BB_0 successors count (only enabled with branch probabilities)
516+
.uleb128 1 # BB_0 successor 1 BB ID (only enabled with branch probabilities)
517+
.uleb128 0x22222222 # BB_0 successor 1 branch probability (only enabled with branch probabilities)
518+
.uleb128 2 # BB_0 successor 2 BB ID (only enabled with branch probabilities)
519+
.uleb128 0x33333333 # BB_0 successor 2 branch probability (only enabled with branch probabilities)
520+
.uleb128 3 # BB_0 successor 3 BB ID (only enabled with branch probabilities)
521+
.uleb128 0xaaaaaaaa # BB_0 successor 3 branch probability (only enabled with branch probabilities)
522+
# PGO data record for BB_1
523+
.uleb128 133 # BB_1 basic block frequency (only when enabled)
524+
.uleb128 2 # BB_1 successors count (only enabled with branch probabilities)
525+
.uleb128 2 # BB_1 successor 1 BB ID (only enabled with branch probabilities)
526+
.uleb128 0x11111111 # BB_1 successor 1 branch probability (only enabled with branch probabilities)
527+
.uleb128 3 # BB_1 successor 2 BB ID (only enabled with branch probabilities)
528+
.uleb128 0x11111111 # BB_1 successor 2 branch probability (only enabled with branch probabilities)
529+
# PGO data record for BB_2
530+
.uleb128 18 # BB_2 basic block frequency (only when enabled)
531+
.uleb128 1 # BB_2 successors count (only enabled with branch probabilities)
532+
.uleb128 3 # BB_2 successor 1 BB ID (only enabled with branch probabilities)
533+
.uleb128 0xffffffff # BB_2 successor 1 branch probability (only enabled with branch probabilities)
534+
# PGO data record for BB_3
535+
.uleb128 1000 # BB_3 basic block frequency (only when enabled)
536+
.uleb128 0 # BB_3 successors count (only enabled with branch probabilities)
537+
454538
``SHT_LLVM_OFFLOADING`` Section (offloading data)
455539
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
456540
This section stores the binary data used to perform offloading device linking

‎llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

Lines changed: 77 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
#include "llvm/CodeGen/GCMetadataPrinter.h"
4141
#include "llvm/CodeGen/LazyMachineBlockFrequencyInfo.h"
4242
#include "llvm/CodeGen/MachineBasicBlock.h"
43+
#include "llvm/CodeGen/MachineBranchProbabilityInfo.h"
4344
#include "llvm/CodeGen/MachineConstantPool.h"
4445
#include "llvm/CodeGen/MachineDominators.h"
4546
#include "llvm/CodeGen/MachineFrameInfo.h"
@@ -139,6 +140,26 @@ static cl::opt<std::string> BasicBlockProfileDump(
139140
"performed with -basic-block-sections=labels. Enabling this "
140141
"flag during in-process ThinLTO is not supported."));
141142

143+
// This is a replication of fields of object::PGOAnalysisMap::Features. It
144+
// should match the order of the fields so that
145+
// `object::PGOAnalysisMap::Features::decode(PgoAnalysisMapFeatures.getBits())`
146+
// succeeds.
147+
enum class PGOMapFeaturesEnum {
148+
FuncEntryCount,
149+
BBFreq,
150+
BrProb,
151+
};
152+
static cl::bits<PGOMapFeaturesEnum> PgoAnalysisMapFeatures(
153+
"pgo-analysis-map", cl::Hidden, cl::CommaSeparated,
154+
cl::values(clEnumValN(PGOMapFeaturesEnum::FuncEntryCount, "func-entry-count",
155+
"Function Entry Count"),
156+
clEnumValN(PGOMapFeaturesEnum::BBFreq, "bb-freq",
157+
"Basic Block Frequency"),
158+
clEnumValN(PGOMapFeaturesEnum::BrProb, "br-prob",
159+
"Branch Probability")),
160+
cl::desc("Enable extended information within the BBAddrMap that is "
161+
"extracted from PGO related analysis."));
162+
142163
const char DWARFGroupName[] = "dwarf";
143164
const char DWARFGroupDescription[] = "DWARF Emission";
144165
const char DbgTimerName[] = "emit";
@@ -427,6 +448,7 @@ void AsmPrinter::getAnalysisUsage(AnalysisUsage &AU) const {
427448
AU.addRequired<MachineOptimizationRemarkEmitterPass>();
428449
AU.addRequired<GCModuleInfo>();
429450
AU.addRequired<LazyMachineBlockFrequencyInfoPass>();
451+
AU.addRequired<MachineBranchProbabilityInfo>();
430452
}
431453

432454
bool AsmPrinter::doInitialization(Module &M) {
@@ -1377,7 +1399,8 @@ void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
13771399
uint8_t BBAddrMapVersion = OutStreamer->getContext().getBBAddrMapVersion();
13781400
OutStreamer->emitInt8(BBAddrMapVersion);
13791401
OutStreamer->AddComment("feature");
1380-
OutStreamer->emitInt8(0);
1402+
auto FeaturesBits = static_cast<uint8_t>(PgoAnalysisMapFeatures.getBits());
1403+
OutStreamer->emitInt8(FeaturesBits);
13811404
OutStreamer->AddComment("function address");
13821405
OutStreamer->emitSymbolValue(FunctionSymbol, getPointerSize());
13831406
OutStreamer->AddComment("number of basic blocks");
@@ -1407,6 +1430,51 @@ void AsmPrinter::emitBBAddrMapSection(const MachineFunction &MF) {
14071430
OutStreamer->emitULEB128IntValue(getBBAddrMapMetadata(MBB));
14081431
PrevMBBEndSymbol = MBB.getEndSymbol();
14091432
}
1433+
1434+
if (FeaturesBits != 0) {
1435+
assert(BBAddrMapVersion >= 2 &&
1436+
"PGOAnalysisMap only supports version 2 or later");
1437+
1438+
auto FeatEnable =
1439+
cantFail(object::PGOAnalysisMap::Features::decode(FeaturesBits));
1440+
1441+
if (FeatEnable.FuncEntryCount) {
1442+
OutStreamer->AddComment("function entry count");
1443+
auto MaybeEntryCount = MF.getFunction().getEntryCount();
1444+
OutStreamer->emitULEB128IntValue(
1445+
MaybeEntryCount ? MaybeEntryCount->getCount() : 0);
1446+
}
1447+
const MachineBlockFrequencyInfo *MBFI =
1448+
FeatEnable.BBFreq
1449+
? &getAnalysis<LazyMachineBlockFrequencyInfoPass>().getBFI()
1450+
: nullptr;
1451+
const MachineBranchProbabilityInfo *MBPI =
1452+
FeatEnable.BrProb ? &getAnalysis<MachineBranchProbabilityInfo>()
1453+
: nullptr;
1454+
1455+
if (FeatEnable.BBFreq || FeatEnable.BrProb) {
1456+
for (const MachineBasicBlock &MBB : MF) {
1457+
if (FeatEnable.BBFreq) {
1458+
OutStreamer->AddComment("basic block frequency");
1459+
OutStreamer->emitULEB128IntValue(
1460+
MBFI->getBlockFreq(&MBB).getFrequency());
1461+
}
1462+
if (FeatEnable.BrProb) {
1463+
unsigned SuccCount = MBB.succ_size();
1464+
OutStreamer->AddComment("basic block successor count");
1465+
OutStreamer->emitULEB128IntValue(SuccCount);
1466+
for (const MachineBasicBlock *SuccMBB : MBB.successors()) {
1467+
OutStreamer->AddComment("successor BB ID");
1468+
OutStreamer->emitULEB128IntValue(SuccMBB->getBBID()->BaseID);
1469+
OutStreamer->AddComment("successor branch probability");
1470+
OutStreamer->emitULEB128IntValue(
1471+
MBPI->getEdgeProbability(&MBB, SuccMBB).getNumerator());
1472+
}
1473+
}
1474+
}
1475+
}
1476+
}
1477+
14101478
OutStreamer->popSection();
14111479
}
14121480

@@ -1932,8 +2000,15 @@ void AsmPrinter::emitFunctionBody() {
19322000

19332001
// Emit section containing BB address offsets and their metadata, when
19342002
// BB labels are requested for this function. Skip empty functions.
1935-
if (MF->hasBBLabels() && HasAnyRealCode)
2003+
bool HasLabels = MF->hasBBLabels();
2004+
if (HasLabels && HasAnyRealCode)
19362005
emitBBAddrMapSection(*MF);
2006+
else if (!HasLabels && HasAnyRealCode &&
2007+
PgoAnalysisMapFeatures.getBits() != 0)
2008+
MF->getContext().reportWarning(
2009+
SMLoc(), "pgo-analysis-map is enabled but the following machine "
2010+
"function was does not have labels: " +
2011+
MF->getName());
19372012

19382013
// Emit sections containing instruction and function PCs.
19392014
emitPCSections(*MF);

‎llvm/test/CodeGen/X86/basic-block-labels-mir-parse.mir

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
# Start after bbsections0-prepare and check that the BB address map is generated.
22
# RUN: llc -mtriple x86_64-unknown-linux-gnu -start-after=bbsections-prepare %s -o - | FileCheck %s -check-prefix=CHECK
33

4+
# Also verify that this holds for PGO extension
5+
# RUN: llc -mtriple x86_64-unknown-linux-gnu -start-after=bbsections-prepare -pgo-analysis-map=func-entry-count,bb-freq,br-prob %s -o - | FileCheck %s -check-prefix=CHECK
6+
# RUN: llc -mtriple x86_64-unknown-linux-gnu -start-after=bbsections-prepare -pgo-analysis-map=func-entry-count %s -o - | FileCheck %s -check-prefix=CHECK
7+
# RUN: llc -mtriple x86_64-unknown-linux-gnu -start-after=bbsections-prepare -pgo-analysis-map=bb-freq %s -o - | FileCheck %s -check-prefix=CHECK
8+
# RUN: llc -mtriple x86_64-unknown-linux-gnu -start-after=bbsections-prepare -pgo-analysis-map=br-prob %s -o - | FileCheck %s -check-prefix=CHECK
9+
410
# How to generate the input:
511
# foo.cc
612
# int foo(bool k) {

‎llvm/test/CodeGen/X86/basic-block-sections-labels-empty-block.ll

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
;; This test verifies that with -gc-empty-basic-blocks SHT_LLVM_BB_ADDR_MAP will not include entries for empty blocks.
22
; RUN: llc < %s -mtriple=x86_64 -O0 -basic-block-sections=labels -gc-empty-basic-blocks | FileCheck --check-prefix=CHECK %s
3+
;; Additionally test that this holds for pgo extension
4+
; RUN: llc < %s -mtriple=x86_64 -O0 -basic-block-sections=labels -pgo-analysis-map=bb-freq -gc-empty-basic-blocks | FileCheck --check-prefixes=CHECK %s
35

46
define void @foo(i1 zeroext %0) nounwind {
57
br i1 %0, label %2, label %empty_block

‎llvm/test/CodeGen/X86/basic-block-sections-labels-empty-function.ll

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
;; Verify that the BB address map is not emitted for empty functions.
2-
; RUN: llc < %s -mtriple=x86_64 -basic-block-sections=labels | FileCheck %s
2+
; RUN: llc < %s -mtriple=x86_64 -basic-block-sections=labels | FileCheck %s --check-prefixes=CHECK,BASIC
3+
; RUN: llc < %s -mtriple=x86_64 -basic-block-sections=labels -pgo-analysis-map=bb-freq | FileCheck %s --check-prefixes=CHECK,PGO
34

45
define void @empty_func() {
56
entry:
@@ -19,5 +20,6 @@ entry:
1920
; CHECK: .Lfunc_begin1:
2021
; CHECK: .section .llvm_bb_addr_map,"o",@llvm_bb_addr_map,.text{{$}}
2122
; CHECK-NEXT: .byte 2 # version
22-
; CHECK-NEXT: .byte 0 # feature
23+
; BASIC-NEXT: .byte 0 # feature
24+
; PGO-NEXT: .byte 2 # feature
2325
; CHECK-NEXT: .quad .Lfunc_begin1 # function address

‎llvm/test/CodeGen/X86/basic-block-sections-labels-functions-sections.ll

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
; RUN: llc < %s -mtriple=x86_64 -function-sections -basic-block-sections=labels | FileCheck %s
1+
; RUN: llc < %s -mtriple=x86_64 -function-sections -basic-block-sections=labels | FileCheck %s --check-prefixes=CHECK,BASIC
2+
; RUN: llc < %s -mtriple=x86_64 -function-sections -basic-block-sections=labels -pgo-analysis-map=func-entry-count,bb-freq | FileCheck %s --check-prefixes=CHECK,PGO
23

34
$_Z4fooTIiET_v = comdat any
45

@@ -11,8 +12,15 @@ define dso_local i32 @_Z3barv() {
1112
; CHECK-NEXT: [[BAR_BEGIN:.Lfunc_begin[0-9]+]]:
1213
; CHECK: .section .llvm_bb_addr_map,"o",@llvm_bb_addr_map,.text._Z3barv{{$}}
1314
; CHECK-NEXT: .byte 2 # version
14-
; CHECK-NEXT: .byte 0 # feature
15+
; BASIC-NEXT: .byte 0 # feature
16+
; PGO-NEXT: .byte 3 # feature
1517
; CHECK-NEXT: .quad [[BAR_BEGIN]] # function address
18+
; CHECK-NEXT: .byte 1 # number of basic blocks
19+
; CHECK-NEXT: .byte 0 # BB id
20+
; CHECK-NEXT: .uleb128 .Lfunc_begin0-.Lfunc_begin0
21+
; CHECK-NEXT: .uleb128 .LBB_END0_0-.Lfunc_begin0
22+
; CHECK-NEXT: .byte 1
23+
; PGO-NEXT: .byte 0 # function entry count
1624

1725

1826
define dso_local i32 @_Z3foov() {
@@ -24,8 +32,15 @@ define dso_local i32 @_Z3foov() {
2432
; CHECK-NEXT: [[FOO_BEGIN:.Lfunc_begin[0-9]+]]:
2533
; CHECK: .section .llvm_bb_addr_map,"o",@llvm_bb_addr_map,.text._Z3foov{{$}}
2634
; CHECK-NEXT: .byte 2 # version
27-
; CHECK-NEXT: .byte 0 # feature
35+
; BASIC-NEXT: .byte 0 # feature
36+
; PGO-NEXT: .byte 3 # feature
2837
; CHECK-NEXT: .quad [[FOO_BEGIN]] # function address
38+
; CHECK-NEXT: .byte 1 # number of basic blocks
39+
; CHECK-NEXT: .byte 0 # BB id
40+
; CHECK-NEXT: .uleb128 .Lfunc_begin1-.Lfunc_begin1
41+
; CHECK-NEXT: .uleb128 .LBB_END1_0-.Lfunc_begin1
42+
; CHECK-NEXT: .byte 1
43+
; PGO-NEXT: .byte 0 # function entry count
2944

3045

3146
define linkonce_odr dso_local i32 @_Z4fooTIiET_v() comdat {
@@ -37,5 +52,12 @@ define linkonce_odr dso_local i32 @_Z4fooTIiET_v() comdat {
3752
; CHECK-NEXT: [[FOOCOMDAT_BEGIN:.Lfunc_begin[0-9]+]]:
3853
; CHECK: .section .llvm_bb_addr_map,"Go",@llvm_bb_addr_map,_Z4fooTIiET_v,comdat,.text._Z4fooTIiET_v{{$}}
3954
; CHECK-NEXT: .byte 2 # version
40-
; CHECK-NEXT: .byte 0 # feature
55+
; BASIC-NEXT: .byte 0 # feature
56+
; PGO-NEXT: .byte 3 # feature
4157
; CHECK-NEXT: .quad [[FOOCOMDAT_BEGIN]] # function address
58+
; CHECK-NEXT: .byte 1 # number of basic blocks
59+
; CHECK-NEXT: .byte 0 # BB id
60+
; CHECK-NEXT: .uleb128 .Lfunc_begin2-.Lfunc_begin2
61+
; CHECK-NEXT: .uleb128 .LBB_END2_0-.Lfunc_begin2
62+
; CHECK-NEXT: .byte 1
63+
; PGO-NEXT: .byte 0 # function entry count

0 commit comments

Comments
 (0)