-
Notifications
You must be signed in to change notification settings - Fork 14.3k
[lldb][progress] Add discrete boolean flag to progress reports #69516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,6 +55,11 @@ namespace lldb_private { | |
|
||
class Progress { | ||
public: | ||
/// Enum that indicates the type of progress report | ||
enum class ProgressReportType { | ||
eAggregateProgressReport, | ||
eNonAggregateProgressReport | ||
}; | ||
Comment on lines
+59
to
+62
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This might need to belong in lldb-enumerations if we want |
||
/// Construct a progress object that will report information. | ||
/// | ||
/// The constructor will create a unique progress reporting object and | ||
|
@@ -63,13 +68,30 @@ class Progress { | |
/// | ||
/// @param [in] title The title of this progress activity. | ||
/// | ||
/// @param [in] total The total units of work to be done if specified, if | ||
/// set to UINT64_MAX then an indeterminate progress indicator should be | ||
/// @param [in] report_type Enum value indicating how the progress is being | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Normally these start with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've clang-formatted these files and that didn't change this comment style so I guess clang-format is just going with was already here, strange 🤔 |
||
/// reported. Progress reports considered "aggregate" are reports done for | ||
/// operations that may happen multiple times during a debug session. | ||
/// | ||
/// For example, when a debug session is first started it needs to parse the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this the case that I attach to a running process and I find 100 binaries loaded in the process. I create Module/ObjectFile's for each binary, including parsing the symbol table, possibly looking for a dSYM or a dwp symbol file with debug information for each of those. If dSYMs are an example of additional binaries that may need to be scanned (and report progress as they're being scanned), is it sufficient to simply mention how each of the hundred binaries in the process will report a status update as its Module is created, and not mention dSYMs specifically? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'd want to use this enum to indicate that parsing symbol tables and loading DWARF indices. I used the term dSYM in this example which makes it look like this is a macOS specific issue when it's more generic so I'll change this, thanks for noticing! |
||
/// symbol tables for all files that were initially included and this | ||
/// operation will deliver progress reports. If new symbol tables need to be | ||
/// parsed later during the session then these will also be parsed and deliver | ||
/// more progress reports. This type of operation would use the | ||
/// eAggregateProgressReport enum. Using this enum would allow these progress | ||
/// reports to be grouped together as one, even though their reports are | ||
/// happening individually. | ||
/// | ||
/// @param [in] total Optional total units of work to be done if specified, if | ||
/// unset then an indeterminate progress indicator should be | ||
/// displayed. | ||
/// | ||
/// @param [in] debugger An optional debugger pointer to specify that this | ||
/// progress is to be reported only to specific debuggers. | ||
Progress(std::string title, uint64_t total = UINT64_MAX, | ||
/// | ||
Progress(std::string title, | ||
ProgressReportType report_type = | ||
ProgressReportType::eNonAggregateProgressReport, | ||
chelcassanova marked this conversation as resolved.
Show resolved
Hide resolved
|
||
std::optional<uint64_t> total = std::nullopt, | ||
lldb_private::Debugger *debugger = nullptr); | ||
|
||
/// Destroy the progress object. | ||
|
@@ -97,12 +119,17 @@ class Progress { | |
/// The title of the progress activity. | ||
std::string m_title; | ||
std::mutex m_mutex; | ||
/// Set to eAggregateProgressReport if the progress event is aggregate; | ||
/// meaning it will happen multiple times during a debug session as individual | ||
/// progress events | ||
ProgressReportType m_report_type = | ||
ProgressReportType::eNonAggregateProgressReport; | ||
/// A unique integer identifier for progress reporting. | ||
const uint64_t m_id; | ||
/// How much work ([0...m_total]) that has been completed. | ||
uint64_t m_completed; | ||
/// Total amount of work, UINT64_MAX for non deterministic progress. | ||
const uint64_t m_total; | ||
/// Total amount of work, use a std::nullopt for non deterministic progress. | ||
const std::optional<uint64_t> m_total; | ||
/// The optional debugger ID to report progress to. If this has no value then | ||
/// all debuggers will receive this event. | ||
std::optional<lldb::user_id_t> m_debugger_id; | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2225,7 +2225,8 @@ void ObjectFileMachO::ParseSymtab(Symtab &symtab) { | |
const char *file_name = file.GetFilename().AsCString("<Unknown>"); | ||
LLDB_SCOPED_TIMERF("ObjectFileMachO::ParseSymtab () module = %s", file_name); | ||
LLDB_LOG(log, "Parsing symbol table for {0}", file_name); | ||
Progress progress(llvm::formatv("Parsing symbol table for {0}", file_name)); | ||
Progress progress(llvm::formatv("Parsing symbol table for {0}", file_name), | ||
Progress::ProgressReportType::eAggregateProgressReport); | ||
Comment on lines
+2228
to
+2229
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could add a progress category as a string instead of a report type? Then this would be easy to group all of symbol table parsing into one section.
I used a local variable for the category for clarity There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When the functionality to add details to a progress report was added in LLDB we wanted to group operations under large progress reports and then incrementally update them but we found that there was no easy place to do that for symbol table and debug info operations so for now, we want to us a flag to indicate to a UI how these operations are reported. We did find that it was possible to group Swift-based operations into their own large progress reports (swiftlang#7769) and so we'd want to have these operations be the "non-aggregate/umbrella" operations. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding a string to represent the category (one of a possible vector of strings for each operation even?) looks like a good idea though. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Seems to me that a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The intention is that if Instead of using a string as a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When we looked at how to group progress notifications by That being said, this does group long-running LLDB notifications as all being too spammy to show since we couldn't group them together under umbrella progress reports. We don't want to keep it this way forever as ultimately we'd like to group all categories of progress reports under their own umbrella We also thought about going with the timeout approach that you suggested but ran into similar issues as you mentioned wherein if many reports were too short then they wouldn't get displayed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So we do want all of the reports to be sent, this seems like a GUI / display issue that can be solved with category names. For you guys the DWARF isn't a big contributor and we want to always see those since on linux a lot of DWARF is not indexed and that is a major contributor to our startup times. Symbol table parsing takes time on mac and linux as well. So everyone is going to classify what progress is interesting in their own way. With a single enum for what Apple users might consider "aggregate" doesn't match up with what Linux or other users consider aggregate and we are now going to be hard coded this into the progress objectsd and that is what is worrying me about the current approach. The user interface is what needs to figure out how to display these things efficiently and the category string seems to allow user interfaces to do more grouping in more efficient ways. The boolean of is_aggregate is a bit too coarse. With the category strings like "Parsing symbol tables" or "Manually Indexing DWARF" or "Loading DWARF accelerator tables", it would allow spammy stuff to be noticed on the, and then instead of displaying individual entries, it would display a single entry with the category name and a spinning progress GUI. Of course you could hard code things based on the category name all the time in Xcode so that "Parsing symbol tables" and "Loading DWARF accelerator tables" would always be grouped if desired to keep the noise down. If no one else agrees with me here, please let me know There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @clayborg I understand your point but how would a category string be different from filtering out the progress events based on the progress title string (or parts of it) ? I see how certain progress reports could be relevant to linux where we wouldn't want to surface it on macOS. The reason we came up with this approach is precisely because we prefer having LLDB tell its clients what progress should be displayed, rather than having the filtering logic be done in the client. May we could keep a list of blacklisted progress title / categories for every platform and either submit progress reports if they're not in that list or have an SBAPI to fetch the list of blacklisted progresses from the client ? What do you think about this ? (also cc @JDevlieghere since he will probably have some opinions about this). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
It would stop clients from having to add logic to try and look for common substrings, but other than that not much different. It would allow progress titles to vary quite a bit more, but they don't right now.
Not sure I follow? Are you saying if
One ideas is if we have categories, we could have settings that could control if the progress reports for a given category should be displayed. Those could easily be set after initializing a debugger object.
Sounds good The entire idea of the progress dialogs is to provide insight for when the debugger seems to be stalled, but it is actually doing work. If we have any progress items that are not useful, we should remove them. But parsing symbol tables, even on Mac, can sometimes take a few seconds on larger binaries, so I find this useful when running LLDB from the command line. Indexing DWARF is a huge time sync for really large projects which often can take many minutes to complete and when I load them on the command line, seeing the progress in the terminal is very useful. So a few things that seem to stand out to me:
I don't see any other uses in the upstream LLDB sources. But my main point is we make Progress dialogs for a reason and we should make sure they are useful and that everyone wants them. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And if we want to allow different platforms to say what is useful and what isn't we can provide commands to control these if needed. We can have an internal enumeration for the categories if needed, or we can add a string as the category to help say "please disable all symbol table parsing" progress dialogs. |
||
|
||
llvm::MachO::symtab_command symtab_load_command = {0, 0, 0, 0, 0, 0}; | ||
llvm::MachO::linkedit_data_command function_starts_load_command = {0, 0, 0, 0}; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if it might be a little easier to intuit if the enums were eAggregateProgressReport and eIndividualProgressReport? Then someone might look at this and say "oh I have a single thing I'm providing progress on, I probably want eIndividualProgressReport". I don't feel strongly that I'm correct here, just throwing it out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe eSolitaryProgressReport
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eUmbrellaProgressReport may work here since that's what's happening when we use a single progress report that's being incrementally updated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is happening multiple times a session? We only index the DWARF one time where we have N DWARF units that we index, but that doesn't happen at all for dSYM files. Symbol tables are parsed only once, but we do parse the symbol tables for each library. What is happening multiple times and how are we going to be able to group these?
Would it be better to use progress category as a string instead of a bool? I will make some inline comments to clarify this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To my understanding something like symbol table parsing can happen multiple times per session if new symbol tables need to be parsed due to new libraries being added. We want to send the progress events to the IDE team who can then filter the events based on this enum.