-
Notifications
You must be signed in to change notification settings - Fork 14.3k
[llvm-cov] Export decision coverage to output json #145325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[llvm-cov] Export decision coverage to output json #145325
Conversation
@llvm/pr-subscribers-pgo Author: None (uthmanna) ChangesOriginal PR is here . + fixed tests. I don't really understand the project or its structure, especially with regard to the tests. I have now built with clang19 / x86_64 and executed the target "check-llvm". I hope that's OK? Full diff: https://github.com/llvm/llvm-project/pull/145325.diff 4 Files Affected:
diff --git a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
index d1230b0ba7c58..8e6180be25b51 100644
--- a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
+++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
@@ -31,6 +31,7 @@
#include "llvm/Support/Endian.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/raw_ostream.h"
+#include <algorithm>
#include <cassert>
#include <cstdint>
#include <iterator>
@@ -494,6 +495,17 @@ struct MCDCRecord {
return TV[TestVectorIndex].first[PosToID[Condition]];
}
+ /// Return the number of True and False decisions for all executed test
+ /// vectors.
+ std::pair<unsigned, unsigned> getDecisions() const {
+ const unsigned TrueDecisions =
+ std::count_if(TV.begin(), TV.end(), [](const auto &TestVec) {
+ return TestVec.second == CondState::MCDC_True;
+ });
+
+ return {TrueDecisions, TV.size() - TrueDecisions};
+ }
+
/// Return the Result evaluation for an executed test vector.
/// See MCDCRecordProcessor::RecordTestVector().
CondState getTVResult(unsigned TestVectorIndex) {
diff --git a/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json b/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
index 33c517da91b5e..3887b43c6eda4 100644
--- a/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
+++ b/llvm/test/tools/llvm-cov/Inputs/binary-formats.canonical.json
@@ -33,4 +33,4 @@ CHECK-SAME: "mcdc":{"count":0,"covered":0,"notcovered":0,"percent":0},
CHECK-SAME: "regions":{"count":1,"covered":1,"notcovered":0,"percent":100}}}
CHECK-SAME: ],
CHECK-SAME: "type":"llvm.coverage.json.export"
-CHECK-SAME: "version":"2.0.1"
+CHECK-SAME: "version":"3.0.0"
diff --git a/llvm/test/tools/llvm-cov/mcdc-export-json.test b/llvm/test/tools/llvm-cov/mcdc-export-json.test
index c2cebf52db8cc..3f9201d338cf1 100644
--- a/llvm/test/tools/llvm-cov/mcdc-export-json.test
+++ b/llvm/test/tools/llvm-cov/mcdc-export-json.test
@@ -1,10 +1,10 @@
// RUN: llvm-profdata merge %S/Inputs/mcdc-general.proftext -o %t.profdata
// RUN: llvm-cov export --format=text %S/Inputs/mcdc-general.o -instr-profile %t.profdata | FileCheck %s
-// CHECK: 12,7,12,27,0,5,[true,true,true,true]
-// CHECK: 15,7,15,13,0,5,[true,true]
-// CHECK: 15,19,15,25,0,5,[true,false]
-// CHECK: 18,7,19,15,0,5,[true,true,false,true]
+// CHECK: 12,7,12,27,2,4,0,5,[true,true,true,true]
+// CHECK: 15,7,15,13,1,2,0,5,[true,true]
+// CHECK: 15,19,15,25,1,1,0,5,[true,false]
+// CHECK: 18,7,19,15,1,3,0,5,[true,true,false,true]
// CHECK: "mcdc":{"count":12,"covered":10,"notcovered":2,"percent":83.333333333333343}
Instructions for regenerating the test:
diff --git a/llvm/tools/llvm-cov/CoverageExporterJson.cpp b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
index 4088c1b053aa8..024693a24cc23 100644
--- a/llvm/tools/llvm-cov/CoverageExporterJson.cpp
+++ b/llvm/tools/llvm-cov/CoverageExporterJson.cpp
@@ -62,7 +62,7 @@
#include <utility>
/// The semantic version combined as a string.
-#define LLVM_COVERAGE_EXPORT_JSON_STR "2.0.1"
+#define LLVM_COVERAGE_EXPORT_JSON_STR "3.0.0"
/// Unique type identifier for JSON coverage export.
#define LLVM_COVERAGE_EXPORT_JSON_TYPE_STR "llvm.coverage.json.export"
@@ -110,8 +110,10 @@ json::Array gatherConditions(const coverage::MCDCRecord &Record) {
json::Array renderMCDCRecord(const coverage::MCDCRecord &Record) {
const llvm::coverage::CounterMappingRegion &CMR = Record.getDecisionRegion();
+ const auto [TrueDecisions, FalseDecisions] = Record.getDecisions();
return json::Array({CMR.LineStart, CMR.ColumnStart, CMR.LineEnd,
- CMR.ColumnEnd, CMR.ExpandedFileID, int64_t(CMR.Kind),
+ CMR.ColumnEnd, TrueDecisions, FalseDecisions,
+ CMR.ExpandedFileID, int64_t(CMR.Kind),
gatherConditions(Record)});
}
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great. I checked out your PR and ran the tests myself just to be sure.
I'll attempt another merge on Tuesday |
…)" (llvm#145325) Tests fixed This commit adds decision coverage counts derived from MC/DC test vector execution to the JSON output of llvm-cov, as discussed here: [Missing Decision Coverage (DC) in output json](https://discourse.llvm.org/t/missing-decision-coverage-dc-in-output-json/86783) with @evodius96
…)" (llvm#145325) Tests fixed This commit adds decision coverage counts derived from MC/DC test vector execution to the JSON output of llvm-cov, as discussed here: [Missing Decision Coverage (DC) in output json](https://discourse.llvm.org/t/missing-decision-coverage-dc-in-output-json/86783) with @evodius96
Original PR is here . + fixed tests. I don't really understand the project or its structure, especially with regard to the tests. I have now built with clang19 / x86_64 and executed the target "check-llvm". I hope that's OK?
fyi: @evodius96