Skip to content

[llvm-cov] Export decision coverage to output json #145325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 24, 2025

Conversation

uthmanna
Copy link
Contributor

Original PR is here . + fixed tests. I don't really understand the project or its structure, especially with regard to the tests. I have now built with clang19 / x86_64 and executed the target "check-llvm". I hope that's OK?
fyi: @evodius96

@llvmbot llvmbot added the PGO Profile Guided Optimizations label Jun 23, 2025
@llvmbot
Copy link
Member

llvmbot commented Jun 23, 2025

@llvm/pr-subscribers-pgo

Author: None (uthmanna)

Changes

Original PR is here . + fixed tests. I don't really understand the project or its structure, especially with regard to the tests. I have now built with clang19 / x86_64 and executed the target "check-llvm". I hope that's OK?
fyi: @evodius96


Full diff: https://github.com/llvm/llvm-project/pull/145325.diff

4 Files Affected:

  • (modified) llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h (+12)
  • (modified) llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json (+1-1)
  • (modified) llvm/test/tools/llvm-cov/mcdc-export-json.test (+4-4)
  • (modified) llvm/tools/llvm-cov/CoverageExporterJson.cpp (+4-2)
diff --git a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
index d1230b0ba7c58..8e6180be25b51 100644
--- a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
+++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
@@ -31,6 +31,7 @@
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/raw_ostream.h"
+#include <algorithm>
 #include <cassert>
 #include <cstdint>
 #include <iterator>
@@ -494,6 +495,17 @@ struct MCDCRecord {
     return TV[TestVectorIndex].first[PosToID[Condition]];
   }
 
+  /// Return the number of True and False decisions for all executed test
+  /// vectors.
+  std::pair<unsigned, unsigned> getDecisions() const {
+    const unsigned TrueDecisions =
+        std::count_if(TV.begin(), TV.end(), [](const auto &TestVec) {
+          return TestVec.second == CondState::MCDC_True;
+        });
+
+    return {TrueDecisions, TV.size() - TrueDecisions};
+  }
+
   /// Return the Result evaluation for an executed test vector.
   /// See MCDCRecordProcessor::RecordTestVector().
   CondState getTVResult(unsigned TestVectorIndex) {
diff --git a/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json b/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
index 33c517da91b5e..3887b43c6eda4 100644
--- a/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
+++ b/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
@@ -33,4 +33,4 @@ CHECK-SAME:     "mcdc":{"count":0,"covered":0,"notcovered":0,"percent":0},
 CHECK-SAME:     "regions":{"count":1,"covered":1,"notcovered":0,"percent":100}}}
 CHECK-SAME: ],
 CHECK-SAME: "type":"llvm.coverage.json.export"
-CHECK-SAME: "version":"2.0.1"
+CHECK-SAME: "version":"3.0.0"
diff --git a/llvm/test/tools/llvm-cov/mcdc-export-json.test b/llvm/test/tools/llvm-cov/mcdc-export-json.test
index c2cebf52db8cc..3f9201d338cf1 100644
--- a/llvm/test/tools/llvm-cov/mcdc-export-json.test
+++ b/llvm/test/tools/llvm-cov/mcdc-export-json.test
@@ -1,10 +1,10 @@
 // RUN: llvm-profdata merge %S/Inputs/mcdc-general.proftext -o %t.profdata
 // RUN: llvm-cov export --format=text %S/Inputs/mcdc-general.o -instr-profile %t.profdata | FileCheck %s
 
-// CHECK: 12,7,12,27,0,5,[true,true,true,true]
-// CHECK: 15,7,15,13,0,5,[true,true]
-// CHECK: 15,19,15,25,0,5,[true,false]
-// CHECK: 18,7,19,15,0,5,[true,true,false,true]
+// CHECK: 12,7,12,27,2,4,0,5,[true,true,true,true]
+// CHECK: 15,7,15,13,1,2,0,5,[true,true]
+// CHECK: 15,19,15,25,1,1,0,5,[true,false]
+// CHECK: 18,7,19,15,1,3,0,5,[true,true,false,true]
 // CHECK: "mcdc":{"count":12,"covered":10,"notcovered":2,"percent":83.333333333333343}
 
 Instructions for regenerating the test:
diff --git a/llvm/tools/llvm-cov/CoverageExporterJson.cpp b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
index 4088c1b053aa8..024693a24cc23 100644
--- a/llvm/tools/llvm-cov/CoverageExporterJson.cpp
+++ b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
@@ -62,7 +62,7 @@
 #include <utility>
 
 /// The semantic version combined as a string.
-#define LLVM_COVERAGE_EXPORT_JSON_STR "2.0.1"
+#define LLVM_COVERAGE_EXPORT_JSON_STR "3.0.0"
 
 /// Unique type identifier for JSON coverage export.
 #define LLVM_COVERAGE_EXPORT_JSON_TYPE_STR "llvm.coverage.json.export"
@@ -110,8 +110,10 @@ json::Array gatherConditions(const coverage::MCDCRecord &Record) {
 
 json::Array renderMCDCRecord(const coverage::MCDCRecord &Record) {
   const llvm::coverage::CounterMappingRegion &CMR = Record.getDecisionRegion();
+  const auto [TrueDecisions, FalseDecisions] = Record.getDecisions();
   return json::Array({CMR.LineStart, CMR.ColumnStart, CMR.LineEnd,
-                      CMR.ColumnEnd, CMR.ExpandedFileID, int64_t(CMR.Kind),
+                      CMR.ColumnEnd, TrueDecisions, FalseDecisions,
+                      CMR.ExpandedFileID, int64_t(CMR.Kind),
                       gatherConditions(Record)});
 }
 

Copy link
Contributor

@evodius96 evodius96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. I checked out your PR and ran the tests myself just to be sure.

@evodius96
Copy link
Contributor

I'll attempt another merge on Tuesday

@evodius96 evodius96 merged commit da5c442 into llvm:main Jun 24, 2025
9 checks passed
DrSergei pushed a commit to DrSergei/llvm-project that referenced this pull request Jun 24, 2025
…)" (llvm#145325)

Tests fixed

This commit adds decision coverage counts derived from MC/DC test vector execution to the JSON output of llvm-cov, as discussed here: [Missing Decision Coverage (DC) in output json](https://discourse.llvm.org/t/missing-decision-coverage-dc-in-output-json/86783) with @evodius96
@uthmanna uthmanna deleted the export_decision_coverage_to_json branch June 25, 2025 06:41
anthonyhatran pushed a commit to anthonyhatran/llvm-project that referenced this pull request Jun 26, 2025
…)" (llvm#145325)

Tests fixed

This commit adds decision coverage counts derived from MC/DC test vector execution to the JSON output of llvm-cov, as discussed here: [Missing Decision Coverage (DC) in output json](https://discourse.llvm.org/t/missing-decision-coverage-dc-in-output-json/86783) with @evodius96
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PGO Profile Guided Optimizations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants