[lldb] Add count for number of DWO files loaded in statistics #144424

qxy11 · 2025-06-16T19:49:40Z

Summary

A new totalLoadedDwoFileCount and totalDwoFileCount counters to available statisctics when calling "statistics dump".

GetDwoFileCounts is created, and returns a pair of ints representing the number of loaded DWO files and the total number of DWO files, respectively. An override is implemented for SymbolFileDWARF that loops through each compile unit, and adds to a counter if it's a DWO unit, and then uses GetDwoSymbolFile(false) to check whether the DWO file was already loaded/parsed.
In Statistics, use GetSeparateDebugInfo to sum up the total number of loaded/parsed DWO files along with the total number of DWO files. This is done by checking whether the DWO file was already successfully loaded in the collected DWO data, anding adding to the totalLoadedDwoFileCount, and adding to totalDwoFileCount for all CU units.

Expected Behavior

When binaries are compiled with split-dwarf and separate DWO files, totalLoadedDwoFileCount would be the number of loaded DWO files and totalDwoFileCount would be the total count of DWO files.
When using a DWP file instead of separate DWO files, totalLoadedDwoFileCount would be the number of parsed compile units, while totalDwoFileCount would be the total number of CUs in the DWP file. This should be similar to the counts we get from loading separate DWO files rather than only counting whether a single DWP file was loaded.
When not using split-dwarf, we expect both totalDwoFileCount and totalLoadedDwoFileCount to be 0 since no separate debug info is loaded.

Testing

Manual Testing
On an internal script that has many DWO files, statistics dump was called before and after a type lookup command. The totalLoadedDwoFileCount increased as expected after the type lookup.

(lldb) statistics dump
{
  ...
  "totalLoadedDwoFileCount": 29,
}
(lldb) type lookup folly::Optional<unsigned int>::Storage
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
...
(lldb) statistics dump
{
  ...
  "totalLoadedDwoFileCount": 2160,
}

Unit test
Added three unit tests that build with new "third.cpp" and "baz.cpp" files. For tests with w/ flags -gsplit-dwarf -gpubnames, this generates 2 DWO files. Then, the test incrementally adds breakpoints, and does a type lookup, and the count should increase for each of these as new DWO files get loaded to support these.

$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 20 tests in 211.738s

OK (skipped=3)

github-actions · 2025-06-16T19:50:02Z

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

llvmbot · 2025-06-16T19:50:33Z

@llvm/pr-subscribers-lldb

Author: None (qxy11)

Changes

Summary

A new totalLoadedDwoFileCount is added to available statisctics when calling "statistics dump".

The counter in SymbolFileDWARF m_loaded_dwo_file_count is added, and accessed through the GetLoadedDwoFileCount()
m_loaded_dwo_file_count is incremented through a new protected function IncrementLoadedDwoFileCount(), which is called in the constructor of SymbolFileDWARFDwo (as opposed to directly incrementing the counter in SymbolFileDwarf to prevent inconsistent updates to the counter in the future).
The GetLoadedDwoFileCount() API is added to SymbolFile.h for access to the DWO count from the symbol files loaded in Statistics, which returns 0 by default for all symbol file types except for SymbolFileDWARF.

Testing

Manual Testing
On an internal script that has many DWO files, statistics dump was called before and after a type lookup command. The totalLoadedDwoFileCount increased as expected after the type lookup.

(lldb) statistics dump
{
  ...
  "totalLoadedDwoFileCount": 29,
}
(lldb) type lookup folly::Optional&lt;unsigned int&gt;::Storage
typedef std::conditional&lt;true, folly::Optional&lt;unsigned int&gt;::StorageTriviallyDestructible, folly::Optional&lt;unsigned int&gt;::StorageNonTriviallyDestructible&gt;::type
typedef std::conditional&lt;true, folly::Optional&lt;unsigned int&gt;::StorageTriviallyDestructible, folly::Optional&lt;unsigned int&gt;::StorageNonTriviallyDestructible&gt;::type
...
(lldb) statistics dump
{
  ...
  "totalLoadedDwoFileCount": 2160,
}

Unit test
Added a unit test that builds new "third.cpp" and "baz.cpp" files w/ flags -gsplit-dwarf -gpubnames. This generated 2 DWO files. Then, the test incrementally adds breakpoints, and does a type lookup, and the count should increase for each of these as new DWO files get loaded to support these.

$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 20 tests in 211.738s

OK (skipped=3)

Full diff: https://github.com/llvm/llvm-project/pull/144424.diff

10 Files Affected:

(modified) lldb/include/lldb/Symbol/SymbolFile.h (+5)
(modified) lldb/packages/Python/lldbsuite/test/builders/builder.py (+18-7)
(modified) lldb/packages/Python/lldbsuite/test/make/Makefile.rules (+4)
(modified) lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp (+1)
(modified) lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h (+9)
(modified) lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp (+1-1)
(modified) lldb/source/Target/Statistics.cpp (+5)
(modified) lldb/test/API/commands/statistics/basic/TestStats.py (+31)
(added) lldb/test/API/commands/statistics/basic/baz.cpp (+12)
(added) lldb/test/API/commands/statistics/basic/third.cpp (+7)

diff --git a/lldb/include/lldb/Symbol/SymbolFile.h b/lldb/include/lldb/Symbol/SymbolFile.h
index 75c7f230ddf3d..42fbc2508889a 100644
--- a/lldb/include/lldb/Symbol/SymbolFile.h
+++ b/lldb/include/lldb/Symbol/SymbolFile.h
@@ -425,6 +425,11 @@ class SymbolFile : public PluginInterface {
   /// Reset the statistics for the symbol file.
   virtual void ResetStatistics() {}
 
+  /// Get the number of loaded DWO files for this symbol file.
+  /// This is used for statistics gathering and will return 0 for most
+  /// symbol file implementations except DWARF symbol files.
+  virtual uint32_t GetLoadedDwoFileCount() const { return 0; }
+
   /// Get the additional modules that this symbol file uses to parse debug info.
   ///
   /// Some debug info is stored in stand alone object files that are represented
diff --git a/lldb/packages/Python/lldbsuite/test/builders/builder.py b/lldb/packages/Python/lldbsuite/test/builders/builder.py
index de05732469448..572abf590345c 100644
--- a/lldb/packages/Python/lldbsuite/test/builders/builder.py
+++ b/lldb/packages/Python/lldbsuite/test/builders/builder.py
@@ -247,13 +247,24 @@ def getLLDBObjRoot(self):
     def _getDebugInfoArgs(self, debug_info):
         if debug_info is None:
             return []
-        if debug_info == "dwarf":
-            return ["MAKE_DSYM=NO"]
-        if debug_info == "dwo":
-            return ["MAKE_DSYM=NO", "MAKE_DWO=YES"]
-        if debug_info == "gmodules":
-            return ["MAKE_DSYM=NO", "MAKE_GMODULES=YES"]
-        return None
+        
+        debug_options = debug_info if isinstance(debug_info, list) else [debug_info]
+        option_flags = {
+            "dwarf": {"MAKE_DSYM": "NO"},
+            "dwo": {"MAKE_DSYM": "NO", "MAKE_DWO": "YES"},
+            "gmodules": {"MAKE_DSYM": "NO", "MAKE_GMODULES": "YES"},
+            "debug_names": {"MAKE_DEBUG_NAMES": "YES"},
+        }
+        
+        # Collect all flags, with later options overriding earlier ones
+        flags = {}
+        
+        for option in debug_options:
+            if not option or option not in option_flags:
+                return None # Invalid options
+            flags.update(option_flags[option])
+        
+        return [f"{key}={value}" for key, value in flags.items()]
 
     def getBuildCommand(
         self,
diff --git a/lldb/packages/Python/lldbsuite/test/make/Makefile.rules b/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
index 06959f226066a..58833e1b0cc78 100644
--- a/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
+++ b/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
@@ -276,6 +276,10 @@ ifeq "$(MAKE_DWO)" "YES"
 	CFLAGS += -gsplit-dwarf
 endif
 
+ifeq "$(MAKE_DEBUG_NAMES)" "YES"
+	CFLAGS += -gpubnames
+endif
+
 ifeq "$(USE_PRIVATE_MODULE_CACHE)" "YES"
 THE_CLANG_MODULE_CACHE_DIR := $(BUILDDIR)/private-module-cache
 else
diff --git a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
index 71f204c03a42a..acd9d2230c201 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
+++ b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
@@ -4358,6 +4358,7 @@ StatsDuration::Duration SymbolFileDWARF::GetDebugInfoIndexTime() {
 
 void SymbolFileDWARF::ResetStatistics() {
   m_parse_time.reset();
+  m_loaded_dwo_file_count = 0;
   if (m_index)
     return m_index->ResetStatistics();
 }
diff --git a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
index d2d30d7decb16..d695961bbfc1d 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
+++ b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
@@ -315,6 +315,11 @@ class SymbolFileDWARF : public SymbolFileCommon {
 
   void ResetStatistics() override;
 
+  /// Get the number of loaded DWO files for this symbol file
+  uint32_t GetLoadedDwoFileCount() const override {
+    return m_loaded_dwo_file_count;
+  }
+
   virtual lldb::offset_t
   GetVendorDWARFOpcodeSize(const DataExtractor &data,
                            const lldb::offset_t data_offset,
@@ -497,6 +502,8 @@ class SymbolFileDWARF : public SymbolFileCommon {
 
   void InitializeFirstCodeAddress();
 
+  void IncrementLoadedDwoFileCount() { m_loaded_dwo_file_count++; }
+
   void
   GetCompileOptions(std::unordered_map<lldb::CompUnitSP, Args> &args) override;
 
@@ -550,6 +557,8 @@ class SymbolFileDWARF : public SymbolFileCommon {
   /// valid value that can be used in DIERef objects which will contain
   /// an index that identifies the .DWO or .o file.
   std::optional<uint64_t> m_file_index;
+  /// Count of loaded DWO files for this symbol file
+  uint32_t m_loaded_dwo_file_count = 0;
 };
 } // namespace dwarf
 } // namespace lldb_private::plugin
diff --git a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp
index c1829abe72933..1f0e847da2905 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp
+++ b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARFDwo.cpp
@@ -31,7 +31,7 @@ SymbolFileDWARFDwo::SymbolFileDWARFDwo(SymbolFileDWARF &base_symbol_file,
                                    /*update_module_section_list*/ false)),
       m_base_symbol_file(base_symbol_file) {
   SetFileIndex(id);
-
+  m_base_symbol_file.IncrementLoadedDwoFileCount();
   // Parsing of the dwarf unit index is not thread-safe, so we need to prime it
   // to enable subsequent concurrent lookups.
   m_context.GetAsLLVM().getCUIndex();
diff --git a/lldb/source/Target/Statistics.cpp b/lldb/source/Target/Statistics.cpp
index 6ec8f8963baf9..b30d7f755f2fc 100644
--- a/lldb/source/Target/Statistics.cpp
+++ b/lldb/source/Target/Statistics.cpp
@@ -322,6 +322,7 @@ llvm::json::Value DebuggerStats::ReportStatistics(
   uint32_t num_modules_with_incomplete_types = 0;
   uint32_t num_stripped_modules = 0;
   uint32_t symtab_symbol_count = 0;
+  uint32_t total_loaded_dwo_file_count = 0;
   for (size_t image_idx = 0; image_idx < num_modules; ++image_idx) {
     Module *module = target != nullptr
                          ? target->GetImages().GetModuleAtIndex(image_idx).get()
@@ -353,6 +354,9 @@ llvm::json::Value DebuggerStats::ReportStatistics(
         for (const auto &symbol_module : symbol_modules.Modules())
           module_stat.symfile_modules.push_back((intptr_t)symbol_module.get());
       }
+      // Add DWO file count from this symbol file (will be 0 for non-DWARF
+      // symbol files)
+      total_loaded_dwo_file_count += sym_file->GetLoadedDwoFileCount();
       module_stat.debug_info_index_loaded_from_cache =
           sym_file->GetDebugInfoIndexWasLoadedFromCache();
       if (module_stat.debug_info_index_loaded_from_cache)
@@ -427,6 +431,7 @@ llvm::json::Value DebuggerStats::ReportStatistics(
       {"totalDebugInfoEnabled", num_debug_info_enabled_modules},
       {"totalSymbolTableStripped", num_stripped_modules},
       {"totalSymbolTableSymbolCount", symtab_symbol_count},
+      {"totalLoadedDwoFileCount", total_loaded_dwo_file_count},
   };
 
   if (include_targets) {
diff --git a/lldb/test/API/commands/statistics/basic/TestStats.py b/lldb/test/API/commands/statistics/basic/TestStats.py
index 83132b40d85db..de4dafaf0a77d 100644
--- a/lldb/test/API/commands/statistics/basic/TestStats.py
+++ b/lldb/test/API/commands/statistics/basic/TestStats.py
@@ -177,6 +177,7 @@ def test_default_no_run(self):
             "totalDebugInfoIndexLoadedFromCache",
             "totalDebugInfoIndexSavedToCache",
             "totalDebugInfoParseTime",
+            "totalLoadedDwoFileCount",
         ]
         self.verify_keys(debug_stats, '"debug_stats"', debug_stat_keys, None)
         if self.getPlatform() != "windows":
@@ -287,6 +288,7 @@ def test_default_with_run(self):
             "totalDebugInfoIndexLoadedFromCache",
             "totalDebugInfoIndexSavedToCache",
             "totalDebugInfoParseTime",
+            "totalLoadedDwoFileCount",
         ]
         self.verify_keys(debug_stats, '"debug_stats"', debug_stat_keys, None)
         stats = debug_stats["targets"][0]
@@ -325,6 +327,7 @@ def test_memory(self):
             "totalDebugInfoIndexLoadedFromCache",
             "totalDebugInfoIndexSavedToCache",
             "totalDebugInfoByteSize",
+            "totalLoadedDwoFileCount",
         ]
         self.verify_keys(debug_stats, '"debug_stats"', debug_stat_keys, None)
 
@@ -377,6 +380,7 @@ def test_modules(self):
             "totalDebugInfoIndexLoadedFromCache",
             "totalDebugInfoIndexSavedToCache",
             "totalDebugInfoByteSize",
+            "totalLoadedDwoFileCount",
         ]
         self.verify_keys(debug_stats, '"debug_stats"', debug_stat_keys, None)
         stats = debug_stats["targets"][0]
@@ -485,6 +489,7 @@ def test_breakpoints(self):
             "totalDebugInfoIndexLoadedFromCache",
             "totalDebugInfoIndexSavedToCache",
             "totalDebugInfoByteSize",
+            "totalLoadedDwoFileCount",
         ]
         self.verify_keys(debug_stats, '"debug_stats"', debug_stat_keys, None)
         target_stats = debug_stats["targets"][0]
@@ -512,6 +517,32 @@ def test_breakpoints(self):
             self.verify_keys(
                 breakpoint, 'target_stats["breakpoints"]', bp_keys_exist, None
             )
+    
+    def test_loaded_dwo_file_count(self):
+        """
+        Test "statistics dump" and the loaded dwo file count.
+        Builds a binary w/ separate .dwo files and debug_names, and then
+        verifies the loaded dwo file count is the expected count after running
+        various commands
+        """
+        da = {"CXX_SOURCES": "third.cpp baz.cpp", "EXE": self.getBuildArtifact("a.out")}
+        self.build(dictionary=da, debug_info=["dwo", "debug_names"])
+        self.addTearDownCleanup(dictionary=da)
+        exe = self.getBuildArtifact("a.out")
+        target = self.createTestTarget(file_path=exe)
+        debug_stats = self.get_stats()
+
+        self.assertIn("totalLoadedDwoFileCount", debug_stats)
+        self.assertEqual(debug_stats["totalLoadedDwoFileCount"], 0)
+        
+        self.runCmd("b main")
+        debug_stats_after_main = self.get_stats()
+        self.assertEqual(debug_stats_after_main["totalLoadedDwoFileCount"], 1)
+        
+        self.runCmd("type lookup Baz")
+        debug_stats_after_type_lookup = self.get_stats()
+        self.assertEqual(debug_stats_after_type_lookup["totalLoadedDwoFileCount"], 2)
+        
 
     @skipUnlessDarwin
     @no_debug_info_test
diff --git a/lldb/test/API/commands/statistics/basic/baz.cpp b/lldb/test/API/commands/statistics/basic/baz.cpp
new file mode 100644
index 0000000000000..536758b17d839
--- /dev/null
+++ b/lldb/test/API/commands/statistics/basic/baz.cpp
@@ -0,0 +1,12 @@
+// Helper that the lldb command `statistics dump` works in split-dwarf mode.
+
+struct Baz {
+  int x;
+  bool y;
+};
+
+void baz() {
+  Baz b;
+  b.x = 1;
+  b.y = true;
+}
diff --git a/lldb/test/API/commands/statistics/basic/third.cpp b/lldb/test/API/commands/statistics/basic/third.cpp
new file mode 100644
index 0000000000000..3943b9c2faafe
--- /dev/null
+++ b/lldb/test/API/commands/statistics/basic/third.cpp
@@ -0,0 +1,7 @@
+// Test that the lldb command `statistics dump` works.
+
+void baz();
+int main(void) {
+  baz();
+  return 0;
+}

qxy11 · 2025-06-16T19:56:31Z

cc @dmpots @jeffreytan81 @clayborg

dmpots · 2025-06-16T20:29:25Z

lldb/include/lldb/Symbol/SymbolFile.h

+  /// Get the number of loaded DWO files for this symbol file.
+  /// This is used for statistics gathering and will return 0 for most
+  /// symbol file implementations except DWARF symbol files.
+  virtual uint32_t GetLoadedDwoFileCount() const { return 0; }


There is a GetSeparateDebugInfo member function that returns info about separate debug info (which dwo is one type). Looking here it looks like we could use it to find all the dwos that were loaded.

I wonder if it would be better to use this existing function when generating the stats instead of adding a new overload here.

Thanks for the reference! In GetSeparateDebugInfo here, the call to GetDwoSymbolFile ends up loading all DWO files (even if they weren't already loaded). I think it'd work if we set the load_all_debug_info to false in the call and do GetDwoSymbolFile(false), in which case we'd have to add an extra load_all_debug_info argument to GetSeparateDebugInfo as well so that other existing use cases for GetSeparateDebugInfo don't get effected. I can try that instead of adding the new overload.

lldb/test/API/commands/statistics/basic/TestStats.py

jeffreytan81 · 2025-06-16T23:49:33Z

Besides totalLoadedDwoFileCount, we need another totalDwoFileCount. The ratio between them will show us how much X% of dwo files have been loaded for the debug session.

Also, we should test this against dwp file situation, and non-split-dwarf situation, what are the numbers we should report?

Address comments to use `GetSeparateDebugInfo` instead of adding a new Symbol File function. Also add a totalLoadedDwoFileCount to statistics dump and add some unit tests for expected behavior in the DWP case and non-split-dwarf cases.