Skip to content

Commit 31c2cf1

Browse files
zjaffalzjaffal
authored
[Remarks] Introduce count subcommand for llvm-remarkutil. (#66214)
This tool is a generic remark counter reporting count based on specified properties. The counter can be used to count remarks individually and filter them based on name, type and pass or count using remark arguments.
1 parent ab6a66d commit 31c2cf1

24 files changed

+914
-0
lines changed

‎llvm/docs/CommandGuide/llvm-remarkutil.rst

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,80 @@ if `--use-debug-loc` is passed then the CSV will include the source path, line n
110110
Source,Function,Count
111111
path:line:column,foo,3
112112

113+
.. _count_subcommand:
114+
115+
count
116+
~~~~~
117+
118+
..program:: llvm-remarkutil count
119+
120+
USAGE: :program:`llvm-remarkutil` count [*options*] <input file>
121+
122+
Summary
123+
^^^^^^^
124+
125+
:program:`llvm-remarkutil count` counts `remarks <https://llvm.org/docs/Remarks.html>` based on specified properties.
126+
By default the tool counts remarks based on how many occour in a source file or function or total for the generated remark file.
127+
The tool also supports collecting count based on specific remark arguments. The specified arguments should have an integer value to be able to report a count.
128+
129+
The tool contains utilities to filter the remark count based on remark name, pass name, argument value and remark type.
130+
OPTIONS
131+
-------
132+
133+
.. option:: --parser=<yaml|bitstream>
134+
135+
Select the type of input remark parser. Required.
136+
* ``yaml``: The tool will parse YAML remarks.
137+
* ``bitstream``: The tool will parse bitstream remarks.
138+
139+
.. option:: --count-by<value>
140+
Select option to collect remarks by.
141+
* ``remark-name``: count how many individual remarks exist.
142+
* ``arg``: count remarks based on specified arguments passed by --(r)args. The argument value must be a number.
143+
144+
.. option:: --group-by=<value>
145+
group count of remarks by property.
146+
* ``source``: Count will be collected per source path. Remarks with no debug location will not be counted.
147+
* ``function``: Count is collected per function.
148+
* ``function-with-loc``: Count is collected per function per source. Remarks with no debug location will not be counted.
149+
* ``Total``: Report a count for the provided remark file.
150+
151+
.. option:: --args[=arguments]
152+
If `count-by` is set to `arg` this flag can be used to collect from specified remark arguments represented as a comma seperated string.
153+
The arguments must have a numeral value to be able to count remarks by
154+
155+
.. option:: --rargs[=arguments]
156+
If `count-by` is set to `arg` this flag can be used to collect from specified remark arguments using regular expression.
157+
The arguments must have a numeral value to be able to count remarks by
158+
159+
.. option:: --pass-name[=<string>]
160+
Filter count by pass name.
161+
162+
.. option:: --rpass-name[=<string>]
163+
Filter count by pass name using regular expressions.
164+
165+
.. option:: --remark-name[=<string>]
166+
Filter count by remark name.
167+
168+
.. option:: --rremark-name[=<string>]
169+
Filter count by remark name using regular expressions.
170+
171+
.. option:: --filter-arg-by[=<string>]
172+
Filter count by argument value.
173+
174+
.. option:: --rfilter-arg-by[=<string>]
175+
Filter count by argument value using regular expressions.
176+
177+
.. option:: --remark-type=<value>
178+
Filter remarks by type with the following options.
179+
* ``unknown``
180+
* ``passed``
181+
* ``missed``
182+
* ``analysis``
183+
* ``analysis-fp-commute``
184+
* ``analysis-aliasing``
185+
* ``failure``
186+
113187
.. _size-diff_subcommand:
114188

115189
size-diff

‎llvm/include/llvm/Remarks/Remark.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,10 @@ struct Argument {
5252

5353
/// Implement operator<< on Argument.
5454
void print(raw_ostream &OS) const;
55+
/// Return the value of argument as int.
56+
std::optional<int> getValAsInt() const;
57+
/// Check if the argument value can be parsed as int.
58+
bool isValInt() const;
5559
};
5660

5761
// Create wrappers for C Binding types (see CBindingWrapping.h).

‎llvm/lib/Remarks/Remark.cpp

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
//===----------------------------------------------------------------------===//
1212

1313
#include "llvm/Remarks/Remark.h"
14+
#include "llvm/ADT/APInt.h"
1415
#include "llvm/ADT/ArrayRef.h"
1516
#include <optional>
1617

@@ -25,6 +26,16 @@ std::string Remark::getArgsAsMsg() const {
2526
return OS.str();
2627
}
2728

29+
/// Returns the value of a specified key parsed from StringRef.
30+
std::optional<int> Argument::getValAsInt() const {
31+
APInt KeyVal;
32+
if (Val.getAsInteger(10, KeyVal))
33+
return std::nullopt;
34+
return KeyVal.getSExtValue();
35+
}
36+
37+
bool Argument::isValInt() const { return getValAsInt().has_value(); }
38+
2839
void RemarkLocation::print(raw_ostream &OS) const {
2940
OS << "{ "
3041
<< "File: " << SourceFilePath << ", Line: " << SourceLine

‎llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,14 @@
11
RUN: llvm-remarkutil annotation-count --use-debug-loc --parser=yaml --annotation-type=remark %p/Inputs/annotation-count-with-dbg-loc.yaml | FileCheck %s
22
RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count-with-dbg-loc.yaml | llvm-remarkutil annotation-count --use-debug-loc --parser=bitstream --annotation-type=remark | FileCheck %s
3+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function-with-loc --remark-name="AnnotationSummary" %p/Inputs/annotation-count-with-dbg-loc.yaml | FileCheck %s --check-prefix=COUNT-CHECK
4+
RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count-with-dbg-loc.yaml | llvm-remarkutil count --parser=bitstream --count-by=arg --group-by=function-with-loc --remark-name="AnnotationSummary" | FileCheck %s --check-prefix=COUNT-CHECK
35

46
; CHECK-LABEL: Source,Function,Count
57
; CHECK: path/to/anno.c:1:2,func1,1
68
; CHECK: path/to/anno2.c:1:2,func2,2
79
; CHECK: path/to/anno3.c:1:2,func3,3
10+
11+
; COUNT-CHECK-LABEL: FuctionWithDebugLoc,count
12+
; COUNT-CHECK: path/to/anno.c:func1,1
13+
; COUNT-CHECK: path/to/anno2.c:func2,2
14+
; COUNT-CHECK: path/to/anno3.c:func3,3

‎llvm/test/tools/llvm-remarkutil/annotation-count.test

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,14 @@
11
RUN: llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/annotation-count.yaml | FileCheck %s
22
RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count.yaml | llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark | FileCheck %s
3+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function --remark-name="AnnotationSummary" %p/Inputs/annotation-count.yaml | FileCheck %s --check-prefix=COUNT-CHECK
4+
RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count.yaml | llvm-remarkutil count --parser=bitstream --count-by=arg --group-by=function --remark-name="AnnotationSummary" | FileCheck %s --check-prefix=COUNT-CHECK
35

46
; CHECK-LABEL: Function,Count
57
; CHECK: func1,1
68
; CHECK: func2,2
79
; CHECK: func3,3
10+
11+
; COUNT-CHECK-LABEL: Function,count
12+
; COUNT-CHECK: func1,1
13+
; COUNT-CHECK: func2,2
14+
; COUNT-CHECK: func3,3

‎llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
RUN: not llvm-remarkutil bitstream2yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
22
RUN: not llvm-remarkutil instruction-count --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
33
RUN: not llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
4+
RUN: not llvm-remarkutil count --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
5+
46
CHECK: error: Unknown magic number: expecting RMRK, got --- .

‎llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
RUN: not llvm-remarkutil yaml2bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
22
RUN: not llvm-remarkutil instruction-count --parser=yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
33
RUN: not llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
4+
RUN: not llvm-remarkutil count --parser=yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
5+
46
CHECK: error: Type, Pass, Name or Function missing

‎llvm/test/tools/llvm-remarkutil/count/Inputs/remark-count-by.yaml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
--- !Analysis
2+
Pass: generic-remarks-pass
3+
Name: Remark
4+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
5+
Function: func1
6+
Args:
7+
- count1: '1'
8+
- count2: '2'
9+
- count3: '3'
10+
- count4: '4'
11+
- String: ' instructions with '
12+
- type: remark
13+
--- !Analysis
14+
Pass: generic-remarks-pass
15+
Name: Remark2
16+
DebugLoc: { File: path/to/anno2.c, Line: 1, Column: 2 }
17+
Function: func1
18+
Args:
19+
- count1: '1'
20+
- count2: '2'
21+
- count3: '3'
22+
- String: ' instructions with '
23+
- type: remark
24+
--- !Analysis
25+
Pass: generic-remarks-pass
26+
Name: Remark3
27+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
28+
Function: func1
29+
Args:
30+
- count1: '1'
31+
- String: ' instructions with '
32+
- type: remark
33+
--- !Analysis
34+
Pass: generic-remarks-pass
35+
Name: Remark
36+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
37+
Function: func2
38+
Args:
39+
- count1: '1'
40+
- count2: '2'
41+
- count3: '3'
42+
- String: ' instructions with '
43+
- type: remark

‎llvm/test/tools/llvm-remarkutil/count/Inputs/remark-filter-by.yaml

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
--- !Analysis
2+
Pass: generic-remarks-pass
3+
Name: Remark
4+
DebugLoc: { File: path/to/anno2.c, Line: 1, Column: 2 }
5+
Function: func1
6+
Args:
7+
- count1: '1'
8+
- String: ' instructions with '
9+
- type: remark
10+
--- !Analysis
11+
Pass: generic-remarks-pass2
12+
Name: Remark2
13+
DebugLoc: { File: path/to/anno2.c, Line: 1, Column: 2 }
14+
Function: func1
15+
Args:
16+
- count1: '1'
17+
- count2: '2'
18+
- count3: '3'
19+
- String: ' instructions with '
20+
- type: remark
21+
--- !Missed
22+
Pass: generic-remarks-pass
23+
Name: Remark3
24+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
25+
Function: func1
26+
Args:
27+
- count1: '1'
28+
- String: ' instructions with '
29+
- type: remark
30+
--- !Passed
31+
Pass: generic-remarks-pass
32+
Name: Remark
33+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
34+
Function: func1
35+
Args:
36+
- count1: '1'
37+
- count2: '2'
38+
- count3: '3'
39+
- String: ' instructions with '
40+
- type: remark

‎llvm/test/tools/llvm-remarkutil/count/Inputs/remark-group-by.yaml

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
--- !Analysis
2+
Pass: generic-remarks-pass
3+
Name: Remark
4+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
5+
Function: func1
6+
Args:
7+
- count: '1'
8+
- String: ' instructions with '
9+
- type: remark
10+
--- !Missed
11+
Pass: generic-remarks-pass
12+
Name: Remark
13+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
14+
Function: func1
15+
Args:
16+
- count: '3'
17+
- String: ' instructions with '
18+
- type: remark
19+
--- !Passed
20+
Pass: generic-remarks-pass
21+
Name: Remark
22+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
23+
Function: func2
24+
Args:
25+
- count: '3'
26+
- String: ' instructions with '
27+
- type: remark
28+
--- !Analysis
29+
Pass: generic-remarks-pass2
30+
Name: Remark
31+
DebugLoc: { File: path/to/anno3.c, Line: 1, Column: 2 }
32+
Function: func1
33+
Args:
34+
- count: '3'
35+
- String: ' instructions with '
36+
- type: remark
37+
--- !Analysis
38+
Pass: generic-remarks-pass3
39+
Name: Remark
40+
DebugLoc: { File: path/to/anno.c, Line: 1, Column: 2 }
41+
Function: func2
42+
Args:
43+
- count: '2'
44+
- String: ' instructions with '
45+
- type: remark
46+
--- !Analysis
47+
Pass: generic-remarks-pass4
48+
Name: Remark
49+
DebugLoc: { File: path/to/anno2.c, Line: 1, Column: 2 }
50+
Function: func3
51+
Args:
52+
- count: '2'
53+
- String: ' instructions with '
54+
- type: remark

‎llvm/test/tools/llvm-remarkutil/count/count-by-keys.test

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=source %p/Inputs/remark-count-by.yaml | FileCheck %s
2+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNC
3+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function-with-loc %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNCLOC
4+
RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=total %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKTOTAL
5+
6+
; CHECK-LABEL: Source,count1,count2,count3,count4
7+
; CHECK: path/to/anno.c,3,4,6,4
8+
; CHECK: path/to/anno2.c,1,2,3,0
9+
10+
; CHECKFUNC-LABEL: Function,count1,count2,count3,count4
11+
; CHECKFUNC: func1,3,4,6,4
12+
; CHECKFUNC: func2,1,2,3,0
13+
14+
; CHECKFUNCLOC-LABEL: FuctionWithDebugLoc,count1,count2,count3,count4
15+
; CHECKFUNCLOC: path/to/anno.c:func1,2,2,3,4
16+
; CHECKFUNCLOC: path/to/anno.c:func2,1,2,3,0
17+
; CHECKFUNCLOC: path/to/anno2.c:func1,1,2,3,0
18+
19+
; CHECKTOTAL-LABEL: Total,count1,count2,count3,count4
20+
; CHECKTOTAL: Total,4,6,9,4

‎llvm/test/tools/llvm-remarkutil/count/count-by-remark.test

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=source %p/Inputs/remark-count-by.yaml | FileCheck %s
2+
RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=function %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNC
3+
RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=function-with-loc %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNCLOC
4+
RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=total %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKTOTAL
5+
6+
; CHECK-LABEL: Source,Count
7+
; CHECK: path/to/anno.c,3
8+
; CHECK: path/to/anno2.c,1
9+
10+
; CHECKFUNC-LABEL: Function,Count
11+
; CHECKFUNC: func1,3
12+
; CHECKFUNC: func2,1
13+
14+
; CHECKFUNCLOC-LABEL: FuctionWithDebugLoc,Count
15+
; CHECKFUNCLOC: path/to/anno.c:func1,2
16+
; CHECKFUNCLOC: path/to/anno.c:func2,1
17+
; CHECKFUNCLOC: path/to/anno2.c:func1,1
18+
19+
; CHECKTOTAL-LABEL: Total,Count
20+
; CHECKTOTAL: Total,4

‎llvm/test/tools/llvm-remarkutil/count/filter-by-pass-name.test

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
RUN: llvm-remarkutil count --parser=yaml --pass-name=generic-remarks-pass %p/Inputs/remark-filter-by.yaml | FileCheck %s
2+
RUN: llvm-remarkutil count --parser=yaml --rpass-name=.* %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=CHECKALL
3+
4+
; CHECK-LABEL: Source,Count
5+
; CHECK: path/to/anno.c,2
6+
; CHECK: path/to/anno2.c,1
7+
8+
; CHECKALL-LABEL: Source,Count
9+
; CHECKALL: path/to/anno.c,2
10+
; CHECKALL: path/to/anno2.c,2

‎llvm/test/tools/llvm-remarkutil/count/filter-by-remark-name.test

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
RUN: llvm-remarkutil count --parser=yaml --remark-name=Remark %p/Inputs/remark-filter-by.yaml | FileCheck %s
2+
RUN: llvm-remarkutil count --parser=yaml --rremark-name=R.* %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=CHECKALL
3+
4+
; CHECK-LABEL: Source,Count
5+
; CHECK: path/to/anno.c,1
6+
; CHECK: path/to/anno2.c,1
7+
8+
; CHECKALL-LABEL: Source,Count
9+
; CHECKALL: path/to/anno.c,2
10+
; CHECKALL: path/to/anno2.c,2

‎llvm/test/tools/llvm-remarkutil/count/filter-by-type.test

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
RUN: llvm-remarkutil count --parser=yaml --remark-type=missed %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=MISSED
2+
RUN: llvm-remarkutil count --parser=yaml --remark-type=passed %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=PASSED
3+
RUN: llvm-remarkutil count --parser=yaml --remark-type=analysis %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=ANALYSIS
4+
RUN: llvm-remarkutil count --parser=yaml --remark-type=unknown %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=UNKNOWN
5+
6+
; MISSED-LABEL: Source,Count
7+
; MISSED: path/to/anno.c,1
8+
9+
; PASSED-LABEL: Source,Count
10+
; PASSED: path/to/anno.c,1
11+
12+
; ANALYSIS-LABEL: Source,Count
13+
; ANALYSIS: path/to/anno2.c,2
14+
15+
; UNKNOWN: Source,Count
16+
; UNKNOWN-EMPTY:

‎llvm/test/tools/llvm-remarkutil/count/group-by-function-with-loc.test

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
RUN: llvm-remarkutil count --parser=yaml --group-by=function-with-loc %p/Inputs/remark-group-by.yaml | FileCheck %s
2+
3+
; CHECK-LABEL: FuctionWithDebugLoc,Count
4+
; CHECK: path/to/anno.c:func1,2
5+
; CHECK: path/to/anno.c:func2,2
6+
; CHECK: path/to/anno2.c:func3,1
7+
; CHECK: path/to/anno3.c:func1,1

‎llvm/test/tools/llvm-remarkutil/count/group-by-function.test

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
2+
RUN: llvm-remarkutil count --parser=yaml --group-by=function %p/Inputs/remark-group-by.yaml | FileCheck %s
3+
4+
; CHECK-LABEL: Function,Count
5+
; CHECK: func1,3
6+
; CHECK: func2,2
7+
; CHECK: func3,1

0 commit comments

Comments
 (0)