Skip to content

[Remarks] Introduce tool to count remarks. #66214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 12, 2023

Conversation

zjaffal
Copy link
Contributor

@zjaffal zjaffal commented Sep 13, 2023

This tool is a generic remark counter reporting count based on specified properties. The counter can be used to count remarks individually and filter them based on name, type and pass or count using remark arguments.

This tool is a generic remark counter reporting count based on specified
properties. The counter can be used to count remarks individually and
filter them based on name, type and pass or count using remark
arguments.
@zjaffal zjaffal added llvm-tools All llvm tools that do not have corresponding tag optimization-remarks labels Sep 13, 2023
@zjaffal zjaffal self-assigned this Sep 13, 2023
Copy link
Contributor

@fhahn fhahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks great, thanks for the detailed documentation!

A bunch of suggestions inline

Zain Jaffal added 3 commits September 28, 2023 13:25
1. Change the description for command line options to be more consistent
2. Fix doumentation issues
3. Change argumetns to use ArrayRef
4. Use unsigned instead of signed when reporting count.
5. Simplify some of the code by using `any_of`
@llvmbot
Copy link
Member

llvmbot commented Sep 28, 2023

@llvm/pr-subscribers-llvm-binary-utilities

Changes

This tool is a generic remark counter reporting count based on specified properties. The counter can be used to count remarks individually and filter them based on name, type and pass or count using remark arguments.


Patch is 45.72 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/66214.diff

24 Files Affected:

  • (modified) llvm/docs/CommandGuide/llvm-remarkutil.rst (+74)
  • (modified) llvm/include/llvm/Remarks/Remark.h (+4)
  • (modified) llvm/lib/Remarks/Remark.cpp (+11)
  • (modified) llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test (+7)
  • (modified) llvm/test/tools/llvm-remarkutil/annotation-count.test (+7)
  • (modified) llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test (+2)
  • (modified) llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test (+2)
  • (added) llvm/test/tools/llvm-remarkutil/count/Inputs/remark-count-by.yaml (+43)
  • (added) llvm/test/tools/llvm-remarkutil/count/Inputs/remark-filter-by.yaml (+40)
  • (added) llvm/test/tools/llvm-remarkutil/count/Inputs/remark-group-by.yaml (+54)
  • (added) llvm/test/tools/llvm-remarkutil/count/count-by-keys.test (+20)
  • (added) llvm/test/tools/llvm-remarkutil/count/count-by-remark.test (+20)
  • (added) llvm/test/tools/llvm-remarkutil/count/filter-by-pass-name.test (+10)
  • (added) llvm/test/tools/llvm-remarkutil/count/filter-by-remark-name.test (+10)
  • (added) llvm/test/tools/llvm-remarkutil/count/filter-by-type.test (+16)
  • (added) llvm/test/tools/llvm-remarkutil/count/group-by-function-with-loc.test (+7)
  • (added) llvm/test/tools/llvm-remarkutil/count/group-by-function.test (+7)
  • (added) llvm/test/tools/llvm-remarkutil/count/group-by-source.test (+6)
  • (modified) llvm/test/tools/llvm-remarkutil/empty-file.test (+5)
  • (modified) llvm/test/tools/llvm-remarkutil/instruction-count-with-dbg-loc.test (+7)
  • (modified) llvm/test/tools/llvm-remarkutil/instruction-count.test (+7)
  • (modified) llvm/tools/llvm-remarkutil/CMakeLists.txt (+1)
  • (added) llvm/tools/llvm-remarkutil/RemarkCounter.cpp (+336)
  • (added) llvm/tools/llvm-remarkutil/RemarkCounter.h (+203)
diff --git a/llvm/docs/CommandGuide/llvm-remarkutil.rst b/llvm/docs/CommandGuide/llvm-remarkutil.rst
index d13a51707a93c58..c92cb1d7e768868 100644
--- a/llvm/docs/CommandGuide/llvm-remarkutil.rst
+++ b/llvm/docs/CommandGuide/llvm-remarkutil.rst
@@ -110,6 +110,80 @@ if `--use-debug-loc` is passed then the CSV will include the source path, line n
   Source,Function,Count
   path:line:column,foo,3
 
+.. _count_subcommand:
+
+count
+~~~~~
+
+..program:: llvm-remarkutil count
+
+USAGE: :program:`llvm-remarkutil` count [*options*] <input file>
+
+Summary
+^^^^^^^
+
+:program:`llvm-remarkutil count` counts `remakrs <https://llvm.org/docs/Remarks.html>` based on specified properties. 
+By default the tool counts remarks based on how many occour in a source file or function or total for the generated remark file.
+The tool also supports collecting count based on specific remark arguments. The specified arguments should have an integer value to be able to report a count.
+
+The tool contains utilities to filter the remark count based on remark name, pass name, argument value and remark type.
+OPTIONS
+-------
+
+.. option:: --parser=<yaml|bitstream>
+
+  Select the type of input remark parser. Required.
+  * ``yaml``: The tool will parse YAML remarks.
+  * ``bitstream``: The tool will parse bitstream remarks.
+
+.. option:: --count-by<value>
+  Select option to collect remarks by.
+  * ``remark-name``: count how many individual remarks exist.
+  * ``arg``: count remarks based on specified arguments passed by --(r)args. The argument value must be a number.
+
+.. option:: --group-by=<value>
+  group count of remarks by property.
+  * ``source``: Count will be collected per source path. Requires remarks to have debug loc info
+  * ``function``: Count is collected per function.
+  * ``function-with-loc``: Count is collected per function per source. Requires remarks to have debug loc info 
+  * ``Total``: Report a count for the provided remark file.
+
+.. option:: --args[=arguments]
+  If `count-by` is set to `arg` this flag can be used to collect from specified remark arguments represented as a comma seperated string.
+  The arguments must have a numeral value to be able to count remarks by
+  
+.. option:: --rargs[=arguments]
+  If `count-by` is set to `arg` this flag can be used to collect from specified remark arguments using regular expression.
+  The arguments must have a numeral value to be able to count remarks by
+
+.. option:: --pass-name[=<string>]
+  Filter count by pass name.
+
+.. option:: --rpass-name[=<string>]
+  Filter count by pass name using regular expressions.
+
+.. option:: --remark-name[=<string>]
+  Filter count by remark name.
+
+.. option:: --rremark-name[=<string>]
+  Filter count by remark name using regular expressions.
+
+.. option:: --filter-arg-by[=<string>]
+  Filter count by argument value.
+
+.. option:: --rfilter-arg-by[=<string>]
+  Filter count by argument value using regular expressions.
+
+.. option:: --remark-type=<value>
+  Filter remarks by type with the following options.
+  * ``unknown``
+  * ``passed``               
+  * ``missed``               
+  * ``analysis``             
+  * ``analysis-fp-commute``
+  * ``analysis-aliasing``    
+  * ``failure``              
+
 .. _size-diff_subcommand:
 
 size-diff
diff --git a/llvm/include/llvm/Remarks/Remark.h b/llvm/include/llvm/Remarks/Remark.h
index a66f7ed73f2f507..de81c5a992805b4 100644
--- a/llvm/include/llvm/Remarks/Remark.h
+++ b/llvm/include/llvm/Remarks/Remark.h
@@ -52,6 +52,10 @@ struct Argument {
 
   /// Implement operator<< on Argument.
   void print(raw_ostream &OS) const;
+  /// Return the value of argument as int.
+  std::optional<int> getValAsInt() const;
+  /// Check if the argument value can be parsed as int.
+  bool isValInt() const;
 };
 
 // Create wrappers for C Binding types (see CBindingWrapping.h).
diff --git a/llvm/lib/Remarks/Remark.cpp b/llvm/lib/Remarks/Remark.cpp
index 1b248db41747ed0..ef42271a3c8da6f 100644
--- a/llvm/lib/Remarks/Remark.cpp
+++ b/llvm/lib/Remarks/Remark.cpp
@@ -11,6 +11,7 @@
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Remarks/Remark.h"
+#include "llvm/ADT/APInt.h"
 #include "llvm/ADT/ArrayRef.h"
 #include <optional>
 
@@ -25,6 +26,16 @@ std::string Remark::getArgsAsMsg() const {
   return OS.str();
 }
 
+/// Returns the value of a specified key parsed from StringRef.
+std::optional<int> Argument::getValAsInt() const {
+  APInt KeyVal;
+  if (Val.getAsInteger(10, KeyVal))
+    return std::nullopt;
+  return KeyVal.getSExtValue();
+}
+
+bool Argument::isValInt() const { return getValAsInt().has_value(); }
+
 void RemarkLocation::print(raw_ostream &OS) const {
   OS << "{ "
      << "File: " << SourceFilePath << ", Line: " << SourceLine
diff --git a/llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test b/llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test
index cdc731c003f6a04..673a38001f760b4 100644
--- a/llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test
+++ b/llvm/test/tools/llvm-remarkutil/annotation-count-with-dbg-loc.test
@@ -1,7 +1,14 @@
 RUN: llvm-remarkutil annotation-count --use-debug-loc --parser=yaml --annotation-type=remark %p/Inputs/annotation-count-with-dbg-loc.yaml | FileCheck %s
 RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count-with-dbg-loc.yaml | llvm-remarkutil annotation-count --use-debug-loc --parser=bitstream --annotation-type=remark | FileCheck %s
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function-with-loc --remark-name="AnnotationSummary" %p/Inputs/annotation-count-with-dbg-loc.yaml | FileCheck %s --check-prefix=COUNT-CHECK
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count-with-dbg-loc.yaml | llvm-remarkutil count --parser=bitstream --count-by=arg --group-by=function-with-loc --remark-name="AnnotationSummary" | FileCheck %s --check-prefix=COUNT-CHECK
 
 ; CHECK-LABEL: Source,Function,Count
 ; CHECK: path/to/anno.c:1:2,func1,1
 ; CHECK: path/to/anno2.c:1:2,func2,2
 ; CHECK: path/to/anno3.c:1:2,func3,3
+
+; COUNT-CHECK-LABEL: FuctionWithDebugLoc,count
+; COUNT-CHECK: path/to/anno.c:func1,1
+; COUNT-CHECK: path/to/anno2.c:func2,2
+; COUNT-CHECK: path/to/anno3.c:func3,3
diff --git a/llvm/test/tools/llvm-remarkutil/annotation-count.test b/llvm/test/tools/llvm-remarkutil/annotation-count.test
index 73582402201dc98..e006220c64f38fc 100644
--- a/llvm/test/tools/llvm-remarkutil/annotation-count.test
+++ b/llvm/test/tools/llvm-remarkutil/annotation-count.test
@@ -1,7 +1,14 @@
 RUN: llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/annotation-count.yaml | FileCheck %s
 RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count.yaml | llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark | FileCheck %s
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function --remark-name="AnnotationSummary" %p/Inputs/annotation-count.yaml | FileCheck %s --check-prefix=COUNT-CHECK
+RUN: llvm-remarkutil yaml2bitstream %p/Inputs/annotation-count.yaml | llvm-remarkutil count --parser=bitstream --count-by=arg --group-by=function --remark-name="AnnotationSummary" | FileCheck %s --check-prefix=COUNT-CHECK
 
 ; CHECK-LABEL: Function,Count
 ; CHECK: func1,1
 ; CHECK: func2,2
 ; CHECK: func3,3
+
+; COUNT-CHECK-LABEL: Function,count
+; COUNT-CHECK: func1,1
+; COUNT-CHECK: func2,2
+; COUNT-CHECK: func3,3
diff --git a/llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test b/llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test
index 515c096f540eb47..78011aece08f720 100644
--- a/llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test
+++ b/llvm/test/tools/llvm-remarkutil/broken-bitstream-remark.test
@@ -1,4 +1,6 @@
 RUN: not llvm-remarkutil bitstream2yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil instruction-count --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil annotation-count --parser=bitstream --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+RUN: not llvm-remarkutil count --parser=bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+
 CHECK: error: Unknown magic number: expecting RMRK, got --- .
diff --git a/llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test b/llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test
index df87e3db80442ef..0f0650660336379 100644
--- a/llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test
+++ b/llvm/test/tools/llvm-remarkutil/broken-yaml-remark.test
@@ -1,4 +1,6 @@
 RUN: not llvm-remarkutil yaml2bitstream %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil instruction-count --parser=yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
 RUN: not llvm-remarkutil annotation-count --parser=yaml --annotation-type=remark %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+RUN: not llvm-remarkutil count --parser=yaml %p/Inputs/broken-remark -o - 2>&1 | FileCheck %s
+
 CHECK: error: Type, Pass, Name or Function missing
diff --git a/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-count-by.yaml b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-count-by.yaml
new file mode 100644
index 000000000000000..3bd0783b7a0a12b
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-count-by.yaml
@@ -0,0 +1,43 @@
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '3'
+  - count4:           '4'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark2
+DebugLoc:        { File: path/to/anno2.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '3'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark3 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func2
+Args:
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '3'
+  - String:          ' instructions with '
+  - type:            remark
diff --git a/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-filter-by.yaml b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-filter-by.yaml
new file mode 100644
index 000000000000000..e9267bd9404848e
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-filter-by.yaml
@@ -0,0 +1,40 @@
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno2.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass2
+Name:            Remark2
+DebugLoc:        { File: path/to/anno2.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '3'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Missed
+Pass:            generic-remarks-pass
+Name:            Remark3 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Passed
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count1:           '1'
+  - count2:           '2'
+  - count3:           '3'
+  - String:          ' instructions with '
+  - type:            remark
diff --git a/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-group-by.yaml b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-group-by.yaml
new file mode 100644
index 000000000000000..5f9222214f2c7bf
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/Inputs/remark-group-by.yaml
@@ -0,0 +1,54 @@
+--- !Analysis
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count:           '1'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Missed
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count:           '3'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Passed
+Pass:            generic-remarks-pass
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func2
+Args:
+  - count:           '3'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass2
+Name:            Remark 
+DebugLoc:        { File: path/to/anno3.c, Line: 1, Column: 2 }
+Function:        func1
+Args:
+  - count:           '3'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass3
+Name:            Remark 
+DebugLoc:        { File: path/to/anno.c, Line: 1, Column: 2 }
+Function:        func2
+Args:
+  - count:           '2'
+  - String:          ' instructions with '
+  - type:            remark
+--- !Analysis
+Pass:            generic-remarks-pass4
+Name:            Remark 
+DebugLoc:        { File: path/to/anno2.c, Line: 1, Column: 2 }
+Function:        func3
+Args:
+  - count:           '2'
+  - String:          ' instructions with '
+  - type:            remark
diff --git a/llvm/test/tools/llvm-remarkutil/count/count-by-keys.test b/llvm/test/tools/llvm-remarkutil/count/count-by-keys.test
new file mode 100644
index 000000000000000..dc414620c3aa550
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/count-by-keys.test
@@ -0,0 +1,20 @@
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=source %p/Inputs/remark-count-by.yaml | FileCheck %s
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNC
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=function-with-loc %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNCLOC
+RUN: llvm-remarkutil count --parser=yaml --count-by=arg --group-by=total %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKTOTAL 
+
+; CHECK-LABEL: Source,count1,count2,count3,count4
+; CHECK: path/to/anno.c,3,4,6,4
+; CHECK: path/to/anno2.c,1,2,3,0
+
+; CHECKFUNC-LABEL: Function,count1,count2,count3,count4
+; CHECKFUNC: func1,3,4,6,4
+; CHECKFUNC: func2,1,2,3,0
+
+; CHECKFUNCLOC-LABEL: FuctionWithDebugLoc,count1,count2,count3,count4
+; CHECKFUNCLOC: path/to/anno.c:func1,2,2,3,4
+; CHECKFUNCLOC: path/to/anno.c:func2,1,2,3,0
+; CHECKFUNCLOC: path/to/anno2.c:func1,1,2,3,0
+
+; CHECKTOTAL-LABEL: Total,count1,count2,count3,count4
+; CHECKTOTAL: Total,4,6,9,4
diff --git a/llvm/test/tools/llvm-remarkutil/count/count-by-remark.test b/llvm/test/tools/llvm-remarkutil/count/count-by-remark.test
new file mode 100644
index 000000000000000..b0248b9b6ec7111
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/count-by-remark.test
@@ -0,0 +1,20 @@
+RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=source %p/Inputs/remark-count-by.yaml | FileCheck %s
+RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=function %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNC
+RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=function-with-loc %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKFUNCLOC
+RUN: llvm-remarkutil count --parser=yaml --count-by=remark-name --group-by=total %p/Inputs/remark-count-by.yaml | FileCheck %s --check-prefix=CHECKTOTAL 
+
+; CHECK-LABEL: Source,Count
+; CHECK: path/to/anno.c,3
+; CHECK: path/to/anno2.c,1
+
+; CHECKFUNC-LABEL: Function,Count
+; CHECKFUNC: func1,3
+; CHECKFUNC: func2,1
+
+; CHECKFUNCLOC-LABEL: FuctionWithDebugLoc,Count
+; CHECKFUNCLOC: path/to/anno.c:func1,2
+; CHECKFUNCLOC: path/to/anno.c:func2,1
+; CHECKFUNCLOC: path/to/anno2.c:func1,1
+
+; CHECKTOTAL-LABEL: Total,Count
+; CHECKTOTAL: Total,4
diff --git a/llvm/test/tools/llvm-remarkutil/count/filter-by-pass-name.test b/llvm/test/tools/llvm-remarkutil/count/filter-by-pass-name.test
new file mode 100644
index 000000000000000..481d6fd2f5820b8
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/filter-by-pass-name.test
@@ -0,0 +1,10 @@
+RUN: llvm-remarkutil count --parser=yaml --pass-name=generic-remarks-pass %p/Inputs/remark-filter-by.yaml | FileCheck %s 
+RUN: llvm-remarkutil count --parser=yaml --rpass-name=.* %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=CHECKALL 
+
+; CHECK-LABEL: Source,Count
+; CHECK: path/to/anno.c,2
+; CHECK: path/to/anno2.c,1
+
+; CHECKALL-LABEL: Source,Count
+; CHECKALL: path/to/anno.c,2
+; CHECKALL: path/to/anno2.c,2
diff --git a/llvm/test/tools/llvm-remarkutil/count/filter-by-remark-name.test b/llvm/test/tools/llvm-remarkutil/count/filter-by-remark-name.test
new file mode 100644
index 000000000000000..20684d57f648cd6
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/filter-by-remark-name.test
@@ -0,0 +1,10 @@
+RUN: llvm-remarkutil count --parser=yaml --remark-name=Remark %p/Inputs/remark-filter-by.yaml | FileCheck %s 
+RUN: llvm-remarkutil count --parser=yaml --rremark-name=R.* %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=CHECKALL 
+
+; CHECK-LABEL: Source,Count
+; CHECK: path/to/anno.c,1
+; CHECK: path/to/anno2.c,1
+
+; CHECKALL-LABEL: Source,Count
+; CHECKALL: path/to/anno.c,2
+; CHECKALL: path/to/anno2.c,2
diff --git a/llvm/test/tools/llvm-remarkutil/count/filter-by-type.test b/llvm/test/tools/llvm-remarkutil/count/filter-by-type.test
new file mode 100644
index 000000000000000..c392fe43aa199e5
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/filter-by-type.test
@@ -0,0 +1,16 @@
+RUN: llvm-remarkutil count --parser=yaml --remark-type=missed %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=MISSED
+RUN: llvm-remarkutil count --parser=yaml --remark-type=passed %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=PASSED
+RUN: llvm-remarkutil count --parser=yaml --remark-type=analysis %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=ANALYSIS
+RUN: llvm-remarkutil count --parser=yaml --remark-type=unknown %p/Inputs/remark-filter-by.yaml | FileCheck %s --check-prefix=UNKNOWN
+
+; MISSED-LABEL: Source,Count
+; MISSED: path/to/anno.c,1
+
+; PASSED-LABEL: Source,Count
+; PASSED: path/to/anno.c,1
+
+; ANALYSIS-LABEL: Source,Count
+; ANALYSIS: path/to/anno2.c,2
+
+; UNKNOWN: Source,Count
+; UNKNOWN-EMPTY: 
diff --git a/llvm/test/tools/llvm-remarkutil/count/group-by-function-with-loc.test b/llvm/test/tools/llvm-remarkutil/count/group-by-function-with-loc.test
new file mode 100644
index 000000000000000..2ceb45b38f29f6e
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/group-by-function-with-loc.test
@@ -0,0 +1,7 @@
+RUN: llvm-remarkutil count --parser=yaml --group-by=function-with-loc %p/Inputs/remark-group-by.yaml | FileCheck %s
+
+; CHECK-LABEL: FuctionWithDebugLoc,Count
+; CHECK: path/to/anno.c:func1,2
+; CHECK: path/to/anno.c:func2,2
+; CHECK: path/to/anno2.c:func3,1
+; CHECK: path/to/anno3.c:func1,1
diff --git a/llvm/test/tools/llvm-remarkutil/count/group-by-function.test b/llvm/test/tools/llvm-remarkutil/count/group-by-function.test
new file mode 100644
index 000000000000000..f3d04bb00c269a9
--- /dev/null
+++ b/llvm/test/tools/llvm-remarkutil/count/group-by-function.test
@@ -0,0 +1,7 @@
+
+RUN: llvm-remarkutil count --parser=yaml --group-by=function %p/Inputs/remark-group-by.yaml | FileCheck %s
+
+; CHECK-LABEL: Function,Count
+; CHECK: func1,3
+; CHECK: func2,2
+; CHECK: func3,...
[truncated]

Filter count by remark name.

.. option:: --rremark-name[=<string>]
Filter count by remark name using regular expressions.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have both name and regex for these options? Isn't the regex enough?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is just following the same idea that other tools have. For example llvm-extract has --func and --rfunc, I think if you pass something like remark-name to the llvm regex parser it will also match with remark-name1, remark-name2 so it is better to seperate regex option

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be fine to have them separately initially. Discussed this with Zain offline and having a separate remark-name makes it easier to match names exactly; with one option that takes a regex only the user would have to write a more complicated regex

@zjaffal zjaffal requested a review from nilanjana87 October 2, 2023 15:12
Filter count by remark name.

.. option:: --rremark-name[=<string>]
Filter count by remark name using regular expressions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be fine to have them separately initially. Discussed this with Zain offline and having a separate remark-name makes it easier to match names exactly; with one option that takes a regex only the user would have to write a more complicated regex

Zain Jaffal added 2 commits October 10, 2023 12:12
The class and the subsiquent variables are used to collect count
information from remark arguments. It is easier for any future
contributers and users of the count tool to get that when the naming
align with the specified function
Copy link
Contributor

@fhahn fhahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Contributor

@fhahn fhahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed there are some build failures: https://buildkite.com/llvm-project/github-pull-requests/builds/6840#018b196d-9977-4b10-abf6-002c37b825ae

std::variant is only available in c++17, but LLVM defaults to c++14, so it cannot be used.

@dlumma
Copy link

dlumma commented Oct 10, 2023

std::variant is available in c++17 but LLVM defaults to c++14.
@fhahn fhahn changed the title Remark Util introduce remark count [Remarks] Introduce tool to count remarks. Oct 12, 2023
Copy link
Contributor

@fhahn fhahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@zjaffal zjaffal merged commit 31c2cf1 into llvm:main Oct 12, 2023
@zjaffal zjaffal deleted the llvm/tools/llvm-remarkutil/remarkcounter branch October 12, 2023 15:03
@jplehr
Copy link
Contributor

jplehr commented Oct 12, 2023

This appears to have broken the AMDGPU OpenMP buildbot https://lab.llvm.org/buildbot/#/builders/193/builds/39996 and the Clang HIP buildbot https://lab.llvm.org/buildbot/#/builders/165/builds/44145

@artagnon
Copy link
Contributor

My LLVM build is broken with this commit:

In file included from /user/rarm/src/riscv-sdk-llvm/llvm/tools/llvm-remarkutil/RemarkCounter.cpp:13:
/user/rarm/src/riscv-sdk-llvm/llvm/tools/llvm-remarkutil/RemarkCounter.h:113:11: error: declaration of ‘llvm::remarks::GroupBy llvm::remarks::Counter::GroupBy’ changes meaning of ‘GroupBy’ [-fpermissive]
  113 |   GroupBy GroupBy;
      |           ^~~~~~~

artagnon added a commit to artagnon/llvm-project that referenced this pull request Oct 12, 2023
…vm#66214)"

This reverts commit 31c2cf1, as it
results in a broken LLVM build.
@AaronBallman
Copy link
Collaborator

There are still broken bots from this change: https://lab.llvm.org/buildbot/#/builders/124/builds/8595 @zjaffal, please revert the changes until you have a fix ready.

nstester pushed a commit to nstester/llvm-project that referenced this pull request Oct 14, 2023
Don't use reserved identifier (_GroupBy)
Fix GCC 7.4/7.5 builds (return Filter => return std::move(Filter))
Remove trailing spaces
@MaskRay
Copy link
Member

MaskRay commented Oct 14, 2023

There are still broken bots from this change: lab.llvm.org/buildbot/#/builders/124/builds/8595 @zjaffal, please revert the changes until you have a fix ready.

The GCC 7.5 build issue was due to return Filter;. I fixed it and the reserved identifier _GroupBy (introduced by a follow-up c136e72 to this patch) in 0603737.

It would be cleaner the revert the patch, but there are so many follow-up commits (and would cause conflicts), so I added another follow-up instead of reverting the whole patch.

@AaronBallman
Copy link
Collaborator

There are still broken bots from this change: lab.llvm.org/buildbot/#/builders/124/builds/8595 @zjaffal, please revert the changes until you have a fix ready.

The GCC 7.5 build issue was due to return Filter;. I fixed it and the reserved identifier _GroupBy (introduced by a follow-up c136e72 to this patch) in 0603737.

It would be cleaner the revert the patch, but there are so many follow-up commits (and would cause conflicts), so I added another follow-up instead of reverting the whole patch.

Thank you! A fix forward is also wonderful. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
llvm:binary-utilities llvm-tools All llvm tools that do not have corresponding tag optimization-remarks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants