Skip to content

[lldb] Index static const members of classes, structs and unions as global variables in DWARF 4 and earlier #111859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 7, 2024

Conversation

kuilpd
Copy link
Contributor

@kuilpd kuilpd commented Oct 10, 2024

In DWARF 4 and earlier static const members of structs, classes and unions have an entry tag DW_TAG_member, and are also tagged as DW_AT_declaration, but otherwise follow the same rules as DW_TAG_variable.

@llvmbot
Copy link
Member

llvmbot commented Oct 10, 2024

@llvm/pr-subscribers-lldb

Author: Ilia Kuklin (kuilpd)

Changes

Static const members initialized inside a class definition might not have a corresponding DW_TAG_variable (DWARF 4 and earlier), so they're not indexed by ManualDWARFIndex.

Add an additional lookup in FindGlobalVariables. Try looking up the enclosing type (e.g. foo::bar for foo::bar::A) and then searching for a static const member (A) within this type.


Full diff: https://github.com/llvm/llvm-project/pull/111859.diff

5 Files Affected:

  • (modified) lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp (+129)
  • (modified) lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h (+4)
  • (added) lldb/test/API/python_api/frame/globals/Makefile (+4)
  • (added) lldb/test/API/python_api/frame/globals/TestTargetGlobals.py (+42)
  • (added) lldb/test/API/python_api/frame/globals/main.cpp (+12)
diff --git a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
index 9287d4baf19e9c..f5a68a9c46c509 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
+++ b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp
@@ -2439,6 +2439,48 @@ void SymbolFileDWARF::FindGlobalVariables(
     return variables.GetSize() - original_size < max_matches;
   });
 
+  // If we don't have enough matches and the variable context is not empty, try
+  // to resolve the context as a type and look for static const members.
+  if (variables.GetSize() - original_size < max_matches && !context.empty()) {
+    llvm::StringRef type_name;
+    if (std::optional<Type::ParsedName> parsed_name =
+        Type::GetTypeScopeAndBasename(context))
+      type_name = parsed_name->basename;
+    else
+      type_name = context;
+
+    m_index->GetTypes(ConstString(type_name), [&](DWARFDIE parent) {
+      llvm::StringRef parent_type_name = parent.GetDWARFDeclContext()
+                                             .GetQualifiedNameAsConstString()
+                                             .GetStringRef();
+
+      // This type is from another scope, skip it.
+      if (!parent_type_name.ends_with(context))
+        return true;
+
+      auto *dwarf_cu = llvm::dyn_cast<DWARFCompileUnit>(parent.GetCU());
+      if (!dwarf_cu)
+        return true;
+
+      sc.comp_unit = GetCompUnitForDWARFCompUnit(*dwarf_cu);
+
+      for (DWARFDIE die = parent.GetFirstChild(); die.IsValid();
+           die = die.GetSibling()) {
+        // Try parsing the entry as a static const member.
+        if (auto var_sp = ParseStaticConstMemberDIE(sc, die)) {
+          if (var_sp->GetUnqualifiedName().GetStringRef() != basename)
+            continue;
+
+          // There can be only one member with a given name.
+          variables.AddVariableIfUnique(var_sp);
+          break;
+        }
+      }
+
+      return variables.GetSize() - original_size < max_matches;
+    });
+  }
+
   // Return the number of variable that were appended to the list
   const uint32_t num_matches = variables.GetSize() - original_size;
   if (log && num_matches > 0) {
@@ -3371,6 +3413,93 @@ size_t SymbolFileDWARF::ParseVariablesForContext(const SymbolContext &sc) {
   return 0;
 }
 
+VariableSP SymbolFileDWARF::ParseStaticConstMemberDIE(
+    const lldb_private::SymbolContext &sc, const DWARFDIE &die) {
+  if (die.GetDWARF() != this)
+    return die.GetDWARF()->ParseStaticConstMemberDIE(sc, die);
+
+  // Look only for members, ignore all other types of entries.
+  if (die.Tag() != DW_TAG_member)
+    return nullptr;
+
+  if (VariableSP var_sp = GetDIEToVariable()[die.GetDIE()])
+    return var_sp; // Already been parsed!
+
+  const char *name = nullptr;
+  const char *mangled = nullptr;
+  Declaration decl;
+  DWARFExpression location;
+  DWARFFormValue type_die_form;
+  DWARFFormValue const_value_form;
+
+  DWARFAttributes attributes = die.GetAttributes();
+  const size_t num_attributes = attributes.Size();
+
+  for (size_t i = 0; i < num_attributes; ++i) {
+    dw_attr_t attr = attributes.AttributeAtIndex(i);
+    DWARFFormValue form_value;
+
+    if (!attributes.ExtractFormValueAtIndex(i, form_value))
+      continue;
+
+    switch (attr) {
+    case DW_AT_decl_file:
+      decl.SetFile(sc.comp_unit->GetSupportFiles().GetFileSpecAtIndex(
+          form_value.Unsigned()));
+      break;
+    case DW_AT_decl_line:
+      decl.SetLine(form_value.Unsigned());
+      break;
+    case DW_AT_decl_column:
+      decl.SetColumn(form_value.Unsigned());
+      break;
+    case DW_AT_name:
+      name = form_value.AsCString();
+      break;
+    case DW_AT_type:
+     type_die_form = form_value;
+      break;
+    case DW_AT_const_value:
+      const_value_form = form_value;
+      break;
+    default:
+      break;
+    }
+  }
+
+  // Look only for static const members with const values.
+  if (!DWARFFormValue::IsDataForm(const_value_form.Form()))
+    return nullptr;
+
+  SymbolFileTypeSP type_sp = std::make_shared<SymbolFileType>(
+     *this, type_die_form.Reference().GetID());
+
+  if (type_sp->GetType()) {
+    location.UpdateValue(const_value_form.Unsigned(),
+                         type_sp->GetType()->GetByteSize(nullptr).value_or(0),
+                         die.GetCU()->GetAddressByteSize());
+  }
+
+  if (Language::LanguageIsCPlusPlus(GetLanguage(*die.GetCU())))
+    mangled =
+        die.GetDWARFDeclContext().GetQualifiedNameAsConstString().GetCString();
+
+  ValueType scope = eValueTypeVariableGlobal;
+  Variable::RangeList scope_ranges;
+
+  DWARFExpressionList location_list(GetObjectFile()->GetModule(), location, die.GetCU());
+  VariableSP var_sp = std::make_shared<Variable>(
+      die.GetID(), name, mangled, type_sp, scope, sc.comp_unit, scope_ranges,
+      &decl, location_list, /*is_external*/ true, /*is_artificial*/ false,
+      /*is_static_member*/ true);
+  var_sp->SetLocationIsConstantValueData(true);
+
+  // Cache this variable, so we don't parse it over and over again.
+  GetDIEToVariable()[die.GetDIE()] = var_sp;
+
+  return var_sp;
+}
+
 VariableSP SymbolFileDWARF::ParseVariableDIECached(const SymbolContext &sc,
                                                    const DWARFDIE &die) {
   if (!die)
diff --git a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
index 4967b37d753a09..0565dd0a666826 100644
--- a/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
+++ b/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.h
@@ -408,6 +408,10 @@ class SymbolFileDWARF : public SymbolFileCommon {
   bool ParseSupportFiles(DWARFUnit &dwarf_cu, const lldb::ModuleSP &module,
                          SupportFileList &support_files);
 
+  lldb::VariableSP
+  ParseStaticConstMemberDIE(const SymbolContext &sc,
+                            const DWARFDIE &die);
+
   lldb::VariableSP ParseVariableDIE(const SymbolContext &sc,
                                     const DWARFDIE &die,
                                     const lldb::addr_t func_low_pc);
diff --git a/lldb/test/API/python_api/frame/globals/Makefile b/lldb/test/API/python_api/frame/globals/Makefile
new file mode 100644
index 00000000000000..c24259ec8c9d93
--- /dev/null
+++ b/lldb/test/API/python_api/frame/globals/Makefile
@@ -0,0 +1,4 @@
+CXX_SOURCES := main.cpp
+CXXFLAGS_EXTRAS := -gdwarf-4
+
+include Makefile.rules
diff --git a/lldb/test/API/python_api/frame/globals/TestTargetGlobals.py b/lldb/test/API/python_api/frame/globals/TestTargetGlobals.py
new file mode 100644
index 00000000000000..edbd97ceadaf67
--- /dev/null
+++ b/lldb/test/API/python_api/frame/globals/TestTargetGlobals.py
@@ -0,0 +1,42 @@
+"""
+Test SBTarget::FindGlobalVariables API.
+"""
+
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+
+class TargetAPITestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    @add_test_categories(['pyapi'])
+    def test_find_global_variables(self):
+        """Exercise SBTarget.FindGlobalVariables() API."""
+        self.build()
+
+        # Don't need to launch a process, since we're only interested in
+        # looking up global variables.
+        target = self.dbg.CreateTarget(self.getBuildArtifact())
+
+        def test_global_var(query, name, type_name, value):
+            value_list = target.FindGlobalVariables(query, 1)
+            self.assertEqual(value_list.GetSize(), 1)
+            var = value_list.GetValueAtIndex(0)
+            self.DebugSBValue(var)
+            self.assertTrue(var)
+            self.assertEqual(var.GetName(), name)
+            self.assertEqual(var.GetTypeName(), type_name)
+            self.assertEqual(var.GetValue(), value)
+
+        test_global_var(
+            "Vars::inline_static",
+            "Vars::inline_static", "double", "1.5")
+        test_global_var(
+            "Vars::static_constexpr",
+            "Vars::static_constexpr", "const int", "2")
+        test_global_var(
+            "Vars::static_const_out_out_class",
+            "Vars::static_const_out_out_class", "const int", "3")
+        test_global_var(
+            "global_var_of_char_type",
+            "::global_var_of_char_type", "char", "'X'")
diff --git a/lldb/test/API/python_api/frame/globals/main.cpp b/lldb/test/API/python_api/frame/globals/main.cpp
new file mode 100644
index 00000000000000..e2095e80082677
--- /dev/null
+++ b/lldb/test/API/python_api/frame/globals/main.cpp
@@ -0,0 +1,12 @@
+class Vars {
+public:
+  inline static double inline_static = 1.5;
+  static constexpr int static_constexpr = 2;
+  static const int static_const_out_out_class;
+};
+
+const int Vars::static_const_out_out_class = 3;
+
+char global_var_of_char_type = 'X';
+
+int main() {}

@kuilpd
Copy link
Contributor Author

kuilpd commented Oct 10, 2024

This patch aims to retrieve static const members specifically from debug information versions DWARF 4 and earlier, where such members are marked DW_TAG_member. In DWARF 5 they are marked DW_TAG_variable and get retrieved normally without this patch.
This was initially created to retrieve debug info in Unreal Engine, which still uses DWARF 4 on Linux: https://reviews.llvm.org/D92643

Right now there is a data formatter that has to use expression evaluation to recalculate something that have already been done during compile time, but it cannot access it because it's a static constexpr member of a struct.

https://github.com/EpicGames/UnrealEngine/blob/40eea367040d50aadd9f030ed5909fc890c159c2/Engine/Extras/LLDBDataFormatters/UEDataFormatters_2ByteChars.py#L84

Copy link

github-actions bot commented Oct 10, 2024

✅ With the latest revision this PR passed the Python code formatter.

Copy link

github-actions bot commented Oct 10, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

Static const members initialized inside a class definition might not have a corresponding DW_TAG_variable (DWARF 4 and earlier), so they're not indexed by ManualDWARFIndex.

Add an additional lookup in FindGlobalVariables. Try looking up the enclosing type (e.g. foo::bar for foo::bar::A) and then searching for a static const member (A) within this type.
@kuilpd kuilpd force-pushed the lookup-static-const-members branch from 5563ba0 to 4c394ec Compare October 10, 2024 16:00
Copy link
Collaborator

@labath labath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a very interesting patch and, at some point, I wanted to do something similar as well. At the same time, I am somewhat worried about the performance impact of the extra lookups.
I think we may want to implement this in a slightly different matter: find the unique definition DIE for the given type (we use that concept when parsing types) and then search for child inside that -- instead of searching through all (possibly one per CU) definitions of that type.

However, before we go down that path, I want to note that something like what you want is already possible using the python API:

(lldb) target var Vars::static_constexpr
error: can't find global variable 'Vars::static_constexpr'
(lldb) script lldb.target.FindFirstType("Vars").GetStaticFieldWithName("static_constexpr").GetConstantValue(lldb.target)
(const int) static_constexpr = 2

It would still be nice if the "target variable" command worked as well, but:

  • I'm a little less concerned about that given that it works for dwarf v5 (clang default)
  • you should be using the python API (instead of expression evaluation) in the data formatter anyway, as it's faster and more reliable

@kuilpd
Copy link
Contributor Author

kuilpd commented Oct 16, 2024

I think we may want to implement this in a slightly different matter: find the unique definition DIE for the given type (we use that concept when parsing types) and then search for child inside that -- instead of searching through all (possibly one per CU) definitions of that type.

I am a bit confused here, don't we need to look through all the debug information to find the type anyway? Could you please point me to the code with type parsing you are referring to?

However, before we go down that path, I want to note that something like what you want is already possible using the python API:

(lldb) target var Vars::static_constexpr
error: can't find global variable 'Vars::static_constexpr'
(lldb) script lldb.target.FindFirstType("Vars").GetStaticFieldWithName("static_constexpr").GetConstantValue(lldb.target)
(const int) static_constexpr = 2

The custom formatter's expression is an example of how I found this bug, but I think it should be fixed regardless, this function can be used from API in other circumstances.

It would still be nice if the "target variable" command worked as well, but:

  • I'm a little less concerned about that given that it works for dwarf v5 (clang default)

  • you should be using the python API (instead of expression evaluation) in the data formatter anyway, as it's faster and more reliable

Would checking the DWARF version and doing the search only on v4 and earlier improve the situation somewhat?

@labath
Copy link
Collaborator

labath commented Oct 18, 2024

I think we may want to implement this in a slightly different matter: find the unique definition DIE for the given type (we use that concept when parsing types) and then search for child inside that -- instead of searching through all (possibly one per CU) definitions of that type.

I am a bit confused here, don't we need to look through all the debug information to find the type anyway? Could you please point me to the code with type parsing you are referring to?

Not really. You can stop the as soon as you find the first definition of that type (even though the definition may be copied into every compile unit out there.

The code I'm referring to is the UniqueDWARFASTTypeMap class (and the code surrounding it). We use that to determine whether to DIEs describe the same type. My idea was something like: a) look up the type that is supposed to contain the variable we're searching for (using FindTypes or something like that); b) get the DWARF DIE describing that type; c) look at the children of that DIE for the variable

However, before we go down that path, I want to note that something like what you want is already possible using the python API:

(lldb) target var Vars::static_constexpr
error: can't find global variable 'Vars::static_constexpr'
(lldb) script lldb.target.FindFirstType("Vars").GetStaticFieldWithName("static_constexpr").GetConstantValue(lldb.target)
(const int) static_constexpr = 2

The custom formatter's expression is an example of how I found this bug, but I think it should be fixed regardless, this function can be used from API in other circumstances.

I agree. It's just that I think this has a slightly lower priority because the problem will presumably "fix" itself over time as people migrate to the new version. I was mentioning this in case your primary (only?) motivation is to fix the data formatter, as that approach would be faster (in any DWARF version).

It would still be nice if the "target variable" command worked as well, but:

  • I'm a little less concerned about that given that it works for dwarf v5 (clang default)
  • you should be using the python API (instead of expression evaluation) in the data formatter anyway, as it's faster and more reliable

Would checking the DWARF version and doing the search only on v4 and earlier improve the situation somewhat?

Somewhat, yes, but I'm not sure how we'd go about this, as every compile unit in a file can be compiled with a different version (and you won't know which one it is until you actually search for it).

It might actually be better to fix this problem during indexing, in the ManualDWARFIndex -- basically, if we're indexing a v4 unit and we run into a static const(expr?) variable (can we detect that reliably by looking at the DIE alone?), then we add it to the index, just as if it was a v5 DW_TAG_variable.

I'm normally want to avoid solving lookup issues by modifying the contents of the index (as that means things behave differently depending on which index you're using), but since in this case we, in a way, bringing the manual index in line with the compiler-generated debug names index (and debug_names does not support DWARF v4), I think this would be okay.

@kuilpd
Copy link
Contributor Author

kuilpd commented Oct 18, 2024

The code I'm referring to is the UniqueDWARFASTTypeMap class (and the code surrounding it). We use that to determine whether to DIEs describe the same type. My idea was something like: a) look up the type that is supposed to contain the variable we're searching for (using FindTypes or something like that); b) get the DWARF DIE describing that type; c) look at the children of that DIE for the variable

Thank you, I will look into it.

It might actually be better to fix this problem during indexing, in the ManualDWARFIndex -- basically, if we're indexing a v4 unit and we run into a static const(expr?) variable (can we detect that reliably by looking at the DIE alone?), then we add it to the index, just as if it was a v5 DW_TAG_variable.

This sounds like the best solution indeed. I will try doing this first and see if it's possible at all to detect such a case. However, there might be other cases where global variables are not tagged as DW_TAG_variable, but to be honest I don't know any other example other than static constexpr anyway.

@@ -0,0 +1,43 @@
"""
Test SBTarget::FindGlobalVariables API.
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we re-use TestConstStaticIntegralMember.py? Surprised it doesn't already have the tests added here XFAILed

@kuilpd
Copy link
Contributor Author

kuilpd commented Oct 24, 2024

It might actually be better to fix this problem during indexing, in the ManualDWARFIndex -- basically, if we're indexing a v4 unit and we run into a static const(expr?) variable (can we detect that reliably by looking at the DIE alone?), then we add it to the index, just as if it was a v5 DW_TAG_variable.

Implemented this as best as I could understand from the DWARF specs, but not sure if there are some cases where the checks I made can be false positive.

Could we re-use TestConstStaticIntegralMember.py? Surprised it doesn't already have the tests added here XFAILed

Thank you, I expanded this test instead of making a new one.

parent->Tag() == DW_TAG_class_type ||
parent->Tag() == DW_TAG_union_type;
}
if (!parent_is_class_type || !is_external || !is_declaration)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanted to note that we have this block here in DWARFASTParserClang.cpp:

// Handle static members, which are typically members without
// locations. However, GCC doesn't emit DW_AT_data_member_location
// for any union members (regardless of linkage).
// Non-normative text pre-DWARFv5 recommends marking static
// data members with an DW_AT_external flag. Clang emits this consistently
// whereas GCC emits it only for static data members if not part of an
// anonymous namespace. The flag that is consistently emitted for static
// data members is DW_AT_declaration, so we check it instead.
// The following block is only necessary to support DWARFv4 and earlier.
// Starting with DWARFv5, static data members are marked DW_AT_variable so we
// can consistently detect them on both GCC and Clang without below heuristic.
if (attrs.member_byte_offset == UINT32_MAX &&
attrs.data_bit_offset == UINT64_MAX && attrs.is_declaration) {
CreateStaticMemberVariable(die, attrs, class_clang_type);
return;
}

Which describes an edge-case with how GCC emits static data members (i.e., it doesn't consistently emit the DW_AT_external flag). Not sure we want to account for that, but also, it would be nice if we had a single place where we did the "isPreDWARFv5StaticDataMember` check. But feel free to ignore

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good finding. I think it would be good to match whatever that code does. (making it a single function for it would be somewhat tricky, as the code here needs to be very fast, so we don't want to have the helper function do another lookup for the attributes and stuff).

@@ -408,6 +408,9 @@ class SymbolFileDWARF : public SymbolFileCommon {
bool ParseSupportFiles(DWARFUnit &dwarf_cu, const lldb::ModuleSP &module,
SupportFileList &support_files);

lldb::VariableSP ParseStaticConstMemberDIE(const SymbolContext &sc,
const DWARFDIE &die);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused

@@ -0,0 +1,12 @@
class Vars {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets remove this new test given we adjusted the existing ones

@@ -3490,7 +3490,7 @@ VariableSP SymbolFileDWARF::ParseVariableDIE(const SymbolContext &sc,
ModuleSP module = GetObjectFile()->GetModule();

if (tag != DW_TAG_variable && tag != DW_TAG_constant &&
(tag != DW_TAG_formal_parameter || !sc.function))
tag != DW_TAG_member && (tag != DW_TAG_formal_parameter || !sc.function))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we bail out (or at least assert) if we detect a DW_AT_data_member_location? Though I guess technically nothing should be calling ParseVariableDIE with such a DIE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an assert, but maybe I should just add that as a condition for detecting a static const member to begin with?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that should be an assert unless some piece of code (where?) alredy makes sure that these kinds of DIEs don't make their way here. Debug info can come from all kinds of compilers (including those from the past and the future), so we shouldn't be crashing just because we've ran into debug info (aka "user input") that we don't understand. Logging something or reporting a warning might be more appropriate.

Checking for this member when detecting a static const member might be okay, but it doesn't really count as the check I mention above, since it only works for the manual index. The other indexes come straight from the compiler (past, future, etc.) so they can't be relied upon in the same way. For this reason (and because we want to make the indexing step as fast as possible), I'd put the check into the indexing step only if it's necessary to properly detect static members.

Comment on lines 377 to 378
if (unit.GetVersion() >= 5)
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put this into the first switch statement (something like case DW_TAG_member: if (unit.GetVersion() >= 5) continue; else break;), so that we avoid parsing the attributes when we know we're not going to use them.

Comment on lines 379 to 384
bool parent_is_class_type = false;
if (auto parent = die.GetParent()) {
parent_is_class_type = parent->Tag() == DW_TAG_structure_type ||
parent->Tag() == DW_TAG_class_type ||
parent->Tag() == DW_TAG_union_type;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bool parent_is_class_type = false;
if (auto parent = die.GetParent()) {
parent_is_class_type = parent->Tag() == DW_TAG_structure_type ||
parent->Tag() == DW_TAG_class_type ||
parent->Tag() == DW_TAG_union_type;
}
bool parent_is_class_type = false;
if (auto parent = die.GetParent())
parent_is_class_type = DWARFDIE(&unit, parent).IsStructUnionOrClass();

@@ -3490,7 +3490,7 @@ VariableSP SymbolFileDWARF::ParseVariableDIE(const SymbolContext &sc,
ModuleSP module = GetObjectFile()->GetModule();

if (tag != DW_TAG_variable && tag != DW_TAG_constant &&
(tag != DW_TAG_formal_parameter || !sc.function))
tag != DW_TAG_member && (tag != DW_TAG_formal_parameter || !sc.function))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that should be an assert unless some piece of code (where?) alredy makes sure that these kinds of DIEs don't make their way here. Debug info can come from all kinds of compilers (including those from the past and the future), so we shouldn't be crashing just because we've ran into debug info (aka "user input") that we don't understand. Logging something or reporting a warning might be more appropriate.

Checking for this member when detecting a static const member might be okay, but it doesn't really count as the check I mention above, since it only works for the manual index. The other indexes come straight from the compiler (past, future, etc.) so they can't be relied upon in the same way. For this reason (and because we want to make the indexing step as fast as possible), I'd put the check into the indexing step only if it's necessary to properly detect static members.

parent->Tag() == DW_TAG_class_type ||
parent->Tag() == DW_TAG_union_type;
}
if (!parent_is_class_type || !is_external || !is_declaration)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good finding. I think it would be good to match whatever that code does. (making it a single function for it would be somewhat tricky, as the code here needs to be very fast, so we don't want to have the helper function do another lookup for the attributes and stuff).

@kuilpd kuilpd changed the title [lldb] Lookup static const members in FindGlobalVariables [lldb] Index static const members of classes, structs and unions as global variables in DWARF 4 and earlier Nov 4, 2024
// `DW_TAG_variable`.
bool parent_is_class_type = false;
if (auto parent = die.GetParent())
parent_is_class_type = DWARFDIE(&unit, parent).IsStructUnionOrClass();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this just be:

Suggested change
parent_is_class_type = DWARFDIE(&unit, parent).IsStructUnionOrClass();
parent_is_class_type = parent->IsStructUnionOrClass();

?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because parent is a DWARFDebugInfoEntry, not a DWARFDIE :P

It's a (not entirely fortunate) result of trying to avoid any high level (expensive) operations, and of the fact that lldb's DWARFDebugInfoEntry (unlike the llvm) fork actually provides some functionality instead of being a simple data holder.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, yea that's unfortunate

Copy link
Collaborator

@labath labath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good now. Is there anything you'd like to add, Michael?

// `DW_TAG_variable`.
bool parent_is_class_type = false;
if (auto parent = die.GetParent())
parent_is_class_type = DWARFDIE(&unit, parent).IsStructUnionOrClass();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because parent is a DWARFDebugInfoEntry, not a DWARFDIE :P

It's a (not entirely fortunate) result of trying to avoid any high level (expensive) operations, and of the fact that lldb's DWARFDebugInfoEntry (unlike the llvm) fork actually provides some functionality instead of being a simple data holder.

Copy link
Member

@Michael137 Michael137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM too, thanks!

Given the pre-DWARFv5 spec isn't very specific about how to differentiate static vs non-static members, leaving out the check/assert for DW_TAG_data_member_location as Pavel suggested seems correct.

@kuilpd kuilpd merged commit 1361c19 into llvm:main Nov 7, 2024
7 checks passed
DavidSpickett added a commit that referenced this pull request Nov 7, 2024
Added by #111859

Due to:
https://lab.llvm.org/buildbot/#/builders/141/builds/3691

This is not uncommon with DWARF testing on Windows. We may be
discarding the required information during linking.

I will look into it next week.
@DavidSpickett
Copy link
Collaborator

I've disabled the new tests on Windows but can't look into the reason for the failures until next week.

@Michael137
Copy link
Member

Michael137 commented Nov 7, 2024

This also fails on the macOS buildbots: https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/14840/execution/node/97/log/

FAIL: test_inline_static_members_dwarf4_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 153, in test_inline_static_members_dwarf4
    self.check_inline_static_members("-gdwarf-4")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1

Could you take a look? Not sure why the XFAIL isn't catching this.

Michael137 added a commit to Michael137/llvm-project that referenced this pull request Nov 8, 2024
…on Darwin

llvm#111859 fixed these
tests for DWARFv4 on Linux by adjusting the manual index.
As part of the change we unXFAILed these tests for DWARFv4
on all platforms. However, the manual index isn't used on macOS
so they're still broken. This patch reverts the XFAIL on Darwin
for DWARFv4.

Example CI failure:
```
FAIL: test_inline_static_members_dwarf4_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 153, in test_inline_static_members_dwarf4
    self.check_inline_static_members("-gdwarf-4")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1
```
@Michael137
Copy link
Member

#115401

Michael137 added a commit that referenced this pull request Nov 8, 2024
…on Darwin (#115401)

#111859 fixed these tests for
DWARFv4 on Linux by adjusting the manual index. As part of the change we
unXFAILed these tests for DWARFv4 on all platforms. However, the manual
index isn't used on macOS so they're still broken. This patch reverts
the XFAIL on Darwin for DWARFv4.

Example CI failure:
```
FAIL: test_inline_static_members_dwarf4_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 153, in test_inline_static_members_dwarf4
    self.check_inline_static_members("-gdwarf-4")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1
```
Michael137 added a commit that referenced this pull request Nov 8, 2024
…der compiler verions

Follow-up to #111859. Prior
to this PR we would never run these tests with DWARFv5 on older Clang
versions (since default wasn't DWARFv5 on macOS until recently). The
patch explicitly started running some of these tests with DWARFv5.
These were failing on the macOS matrix bot (with Clang-15/Clang-17).

```
======================================================================
FAIL: test_inline_static_members_dwarf5_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 150, in test_inline_static_members_dwarf5
    self.check_inline_static_members("-gdwarf-5")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1
```
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Nov 15, 2024
Added by llvm#111859

Due to:
https://lab.llvm.org/buildbot/#/builders/141/builds/3691

This is not uncommon with DWARF testing on Windows. We may be
discarding the required information during linking.

I will look into it next week.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Nov 15, 2024
…on Darwin (llvm#115401)

llvm#111859 fixed these tests for
DWARFv4 on Linux by adjusting the manual index. As part of the change we
unXFAILed these tests for DWARFv4 on all platforms. However, the manual
index isn't used on macOS so they're still broken. This patch reverts
the XFAIL on Darwin for DWARFv4.

Example CI failure:
```
FAIL: test_inline_static_members_dwarf4_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 153, in test_inline_static_members_dwarf4
    self.check_inline_static_members("-gdwarf-4")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1
```
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Nov 15, 2024
…der compiler verions

Follow-up to llvm#111859. Prior
to this PR we would never run these tests with DWARFv5 on older Clang
versions (since default wasn't DWARFv5 on macOS until recently). The
patch explicitly started running some of these tests with DWARFv5.
These were failing on the macOS matrix bot (with Clang-15/Clang-17).

```
======================================================================
FAIL: test_inline_static_members_dwarf5_dsym (TestConstStaticIntegralMember.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1769, in test_method
    return attrvalue(self)
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 150, in test_inline_static_members_dwarf5
    self.check_inline_static_members("-gdwarf-5")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 129, in check_inline_static_members
    self.check_global_var("A::int_val", "const int", "1")
  File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/lang/cpp/const_static_integral_member/TestConstStaticIntegralMember.py", line 118, in check_global_var
    self.assertGreaterEqual(len(var_list), 1)
AssertionError: 0 not greater than or equal to 1
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants