[trace][intelpt] Introduce instruction Ids

In order to support quick arbitrary access to instructions in the trace, we need
each instruction to have an id. It could be an index or any other value that the
trace plugin defines.

This will be useful for reverse debugging or for creating callstacks, as each
frame will need an instruction id associated with them.

I've updated the `thread trace dump instructions` command accordingly. It now
prints the instruction id instead of relative offset. I've also added a new --id
argument that allows starting the dump from an arbitrary position.

Differential Revision: https://reviews.llvm.org/D122254

Author: Walter Erquinigo

lldb/include/lldb/Target/TraceCursor.h

@@ -74,9 +74,10 @@ class TraceCursor {
 public:
   /// Helper enum to indicate the reference point when invoking
   /// \a TraceCursor::Seek().
+  /// The following values are inspired by \a std::istream::seekg.
   enum class SeekType {
     /// The beginning of the trace, i.e the oldest item.
-    Set = 0,
+    Beginning = 0,
     /// The current position in the trace.
     Current,
     /// The end of the trace, i.e the most recent item.
@@ -133,6 +134,51 @@ class TraceCursor {
   ///     \b true if the cursor effectively moved, \b false otherwise.
   virtual bool Next() = 0;
 
+  /// Instruction identifiers:
+  ///
+  /// When building complex higher level tools, fast random accesses in the
+  /// trace might be needed, for which each instruction requires a unique
+  /// identifier within its thread trace. For example, a tool might want to
+  /// repeatedly inspect random consecutive portions of a trace. This means that
+  /// it will need to first move quickly to the beginning of each section and
+  /// then start its iteration. Given that the number of instructions can be in
+  /// the order of hundreds of millions, fast random access is necessary.
+  ///
+  /// An example of such a tool could be an inspector of the call graph of a
+  /// trace, where each call is represented with its start and end instructions.
+  /// Inspecting all the instructions of a call requires moving to its first
+  /// instruction and then iterating until the last instruction, which following
+  /// the pattern explained above.
+  ///
+  /// Instead of using 0-based indices as identifiers, each Trace plug-in can
+  /// decide the nature of these identifiers and thus no assumptions can be made
+  /// regarding their ordering and sequentiality. The reason is that an
+  /// instruction might be encoded by the plug-in in a way that hides its actual
+  /// 0-based index in the trace, but it's still possible to efficiently find
+  /// it.
+  ///
+  /// Requirements:
+  /// - For a given thread, no two instructions have the same id.
+  /// - In terms of efficiency, moving the cursor to a given id should be as
+  ///   fast as possible, but not necessarily O(1). That's why the recommended
+  ///   way to traverse sequential instructions is to use the \a
+  ///   TraceCursor::Next() method and only use \a TraceCursor::GoToId(id)
+  ///   sparingly.
+
+  /// Make the cursor point to the item whose identifier is \p id.
+  ///
+  /// \return
+  ///     \b true if the given identifier exists and the cursor effectively
+  ///     moved. Otherwise, \b false is returned and the cursor doesn't change
+  ///     its position.
+  virtual bool GoToId(lldb::user_id_t id) = 0;
+
+  /// \return
+  ///     A unique identifier for the instruction or error this cursor is
+  ///     pointing to.
+  virtual lldb::user_id_t GetId() const = 0;
+  /// \}
+
   /// Make the cursor point to an item in the trace based on an origin point and
   /// an offset. This API doesn't distinguishes instruction types nor errors in
   /// the trace, unlike the \a TraceCursor::Next() method.
@@ -152,7 +198,7 @@ class TraceCursor {
   ///
   /// \return
   ///     The number of trace items moved from the origin.
-  virtual size_t Seek(ssize_t offset, SeekType origin) = 0;
+  virtual uint64_t Seek(int64_t offset, SeekType origin) = 0;
 
   /// \return
   ///   The \a ExecutionContextRef of the backing thread from the creation time

lldb/include/lldb/Target/TraceInstructionDumper.h

@@ -13,6 +13,26 @@
 
 namespace lldb_private {
 
+/// Class that holds the configuration used by \a TraceInstructionDumper for
+/// traversing and dumping instructions.
+struct TraceInstructionDumperOptions {
+  /// If \b true, the cursor will be iterated forwards starting from the
+  /// oldest instruction. Otherwise, the iteration starts from the most
+  /// recent instruction.
+  bool forwards = false;
+  /// Dump only instruction addresses without disassembly nor symbol
+  /// information.
+  bool raw = false;
+  /// For each instruction, print the corresponding timestamp counter if
+  /// available.
+  bool show_tsc = false;
+  /// Optional custom id to start traversing from.
+  llvm::Optional<uint64_t> id = llvm::None;
+  /// Optional number of instructions to skip from the starting position
+  /// of the cursor.
+  llvm::Optional<size_t> skip = llvm::None;
+};
+
 /// Class used to dump the instructions of a \a TraceCursor using its current
 /// state and granularity.
 class TraceInstructionDumper {
@@ -22,52 +42,46 @@ class TraceInstructionDumper {
   /// \param[in] cursor
   ///     The cursor whose instructions will be dumped.
   ///
-  /// \param[in] initial_index
-  ///     Presentation index to use for referring to the current instruction
-  ///     of the cursor. If the direction is forwards, the index will increase,
-  ///     and if the direction is backwards, the index will decrease.
-  ///
-  /// \param[in] raw
-  ///     Dump only instruction addresses without disassembly nor symbol
-  ///     information.
+  /// \param[in] s
+  ///     The stream where to dump the instructions to.
   ///
-  /// \param[in] show_tsc
-  ///     For each instruction, print the corresponding timestamp counter if
-  ///     available.
-  TraceInstructionDumper(lldb::TraceCursorUP &&cursor_up, int initial_index = 0,
-                         bool raw = false, bool show_tsc = false);
+  /// \param[in] options
+  ///     Additional options for configuring the dumping.
+  TraceInstructionDumper(lldb::TraceCursorUP &&cursor_up, Stream &s,
+                         const TraceInstructionDumperOptions &options);
 
   /// Dump \a count instructions of the thread trace starting at the current
   /// cursor position.
   ///
   /// This effectively moves the cursor to the next unvisited position, so that
   /// a subsequent call to this method continues where it left off.
   ///
-  /// \param[in] s
-  ///     The stream object where the instructions are printed.
-  ///
   /// \param[in] count
   ///     The number of instructions to print.
-  void DumpInstructions(Stream &s, size_t count);
-
-  /// Indicate the dumper that no more data is available in the trace.
-  void SetNoMoreData();
+  ///
+  /// \return
+  ///     The instruction id of the last traversed instruction, or \b llvm::None
+  ///     if no instructions were visited.
+  llvm::Optional<lldb::user_id_t> DumpInstructions(size_t count);
 
   /// \return
   ///     \b true if there's still more data to traverse in the trace.
   bool HasMoreData();
 
 private:
+  /// Indicate to the dumper that no more data is available in the trace.
+  /// This will prevent further iterations.
+  void SetNoMoreData();
+
   /// Move the cursor one step.
   ///
   /// \return
   ///     \b true if the cursor moved.
   bool TryMoveOneStep();
 
   lldb::TraceCursorUP m_cursor_up;
-  int m_index;
-  bool m_raw;
-  bool m_show_tsc;
+  TraceInstructionDumperOptions m_options;
+  Stream &m_s;
   /// If \b true, all the instructions have been traversed.
   bool m_no_more_data = false;
 };

lldb/source/Commands/CommandObjectThread.cpp

@@ -2099,8 +2099,7 @@ class CommandObjectTraceStop : public CommandObjectMultipleThreads {
 #define LLDB_OPTIONS_thread_trace_dump_instructions
 #include "CommandOptions.inc"
 
-class CommandObjectTraceDumpInstructions
-    : public CommandObjectIterateOverThreads {
+class CommandObjectTraceDumpInstructions : public CommandObjectParsed {
 public:
   class CommandOptions : public Options {
   public:
@@ -2132,19 +2131,29 @@ class CommandObjectTraceDumpInstructions
               "invalid integer value for option '%s'",
               option_arg.str().c_str());
         else
-          m_skip = skip;
+          m_dumper_options.skip = skip;
+        break;
+      }
+      case 'i': {
+        uint64_t id;
+        if (option_arg.empty() || option_arg.getAsInteger(0, id))
+          error.SetErrorStringWithFormat(
+              "invalid integer value for option '%s'",
+              option_arg.str().c_str());
+        else
+          m_dumper_options.id = id;
         break;
       }
       case 'r': {
-        m_raw = true;
+        m_dumper_options.raw = true;
         break;
       }
       case 'f': {
-        m_forwards = true;
+        m_dumper_options.forwards = true;
         break;
       }
       case 't': {
-        m_show_tsc = true;
+        m_dumper_options.show_tsc = true;
         break;
       }
       case 'C': {
@@ -2159,11 +2168,8 @@ class CommandObjectTraceDumpInstructions
 
     void OptionParsingStarting(ExecutionContext *execution_context) override {
       m_count = kDefaultCount;
-      m_skip = 0;
-      m_raw = false;
-      m_forwards = false;
-      m_show_tsc = false;
       m_continue = false;
+      m_dumper_options = {};
     }
 
     llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
@@ -2174,23 +2180,19 @@ class CommandObjectTraceDumpInstructions
 
     // Instance variables to hold the values for command options.
     size_t m_count;
-    size_t m_skip;
-    bool m_raw;
-    bool m_forwards;
-    bool m_show_tsc;
-    bool m_continue;
+    size_t m_continue;
+    TraceInstructionDumperOptions m_dumper_options;
   };
 
   CommandObjectTraceDumpInstructions(CommandInterpreter &interpreter)
-      : CommandObjectIterateOverThreads(
+      : CommandObjectParsed(
             interpreter, "thread trace dump instructions",
-            "Dump the traced instructions for one or more threads. If no "
-            "threads are specified, show the current thread. Use the "
-            "thread-index \"all\" to see all threads.",
+            "Dump the traced instructions for one thread. If no "
+            "thread is specified, show the current thread.",
             nullptr,
-            eCommandRequiresProcess | eCommandTryTargetAPILock |
-                eCommandProcessMustBeLaunched | eCommandProcessMustBePaused |
-                eCommandProcessMustBeTraced) {}
+            eCommandRequiresProcess | eCommandRequiresThread |
+                eCommandTryTargetAPILock | eCommandProcessMustBeLaunched |
+                eCommandProcessMustBePaused | eCommandProcessMustBeTraced) {}
 
   ~CommandObjectTraceDumpInstructions() override = default;
 
@@ -2200,57 +2202,63 @@ class CommandObjectTraceDumpInstructions
                                                uint32_t index) override {
     std::string cmd;
     current_command_args.GetCommandString(cmd);
-    if (cmd.find("--continue") == std::string::npos)
+    if (cmd.find(" --continue") == std::string::npos)
       cmd += " --continue";
     return cmd;
   }
 
 protected:
-  bool DoExecute(Args &args, CommandReturnObject &result) override {
-    if (!m_options.m_continue)
-      m_dumpers.clear();
+  ThreadSP GetThread(Args &args, CommandReturnObject &result) {
+    if (args.GetArgumentCount() == 0)
+      return m_exe_ctx.GetThreadSP();
 
-    return CommandObjectIterateOverThreads::DoExecute(args, result);
-  }
-
-  bool HandleOneThread(lldb::tid_t tid, CommandReturnObject &result) override {
-    Stream &s = result.GetOutputStream();
+    const char *arg = args.GetArgumentAtIndex(0);
+    uint32_t thread_idx;
 
-    const TraceSP &trace_sp = m_exe_ctx.GetTargetSP()->GetTrace();
+    if (!llvm::to_integer(arg, thread_idx)) {
+      result.AppendErrorWithFormat("invalid thread specification: \"%s\"\n",
+                                   arg);
+      return nullptr;
+    }
     ThreadSP thread_sp =
-        m_exe_ctx.GetProcessPtr()->GetThreadList().FindThreadByID(tid);
-
-    if (!m_dumpers.count(thread_sp->GetID())) {
-      lldb::TraceCursorUP cursor_up = trace_sp->GetCursor(*thread_sp);
-      // Set up the cursor and return the presentation index of the first
-      // instruction to dump after skipping instructions.
-      auto setUpCursor = [&]() {
-        cursor_up->SetForwards(m_options.m_forwards);
-        if (m_options.m_forwards)
-          return cursor_up->Seek(m_options.m_skip, TraceCursor::SeekType::Set);
-        return -cursor_up->Seek(-m_options.m_skip, TraceCursor::SeekType::End);
-      };
-
-      int initial_index = setUpCursor();
+        m_exe_ctx.GetProcessRef().GetThreadList().FindThreadByIndexID(
+            thread_idx);
+    if (!thread_sp)
+      result.AppendErrorWithFormat("no thread with index: \"%s\"\n", arg);
+    return thread_sp;
+  }
 
-      auto dumper = std::make_unique<TraceInstructionDumper>(
-          std::move(cursor_up), initial_index, m_options.m_raw,
-          m_options.m_show_tsc);
+  bool DoExecute(Args &args, CommandReturnObject &result) override {
+    ThreadSP thread_sp = GetThread(args, result);
+    if (!thread_sp)
+      return false;
 
-      // This happens when the seek value was more than the number of available
-      // instructions.
-      if (std::abs(initial_index) < (int)m_options.m_skip)
-        dumper->SetNoMoreData();
+    Stream &s = result.GetOutputStream();
+    s.Printf("thread #%u: tid = %" PRIu64 "\n", thread_sp->GetIndexID(),
+             thread_sp->GetID());
 
-      m_dumpers[thread_sp->GetID()] = std::move(dumper);
+    if (m_options.m_continue) {
+      if (!m_last_id) {
+        result.AppendMessage("    no more data\n");
+        return true;
+      }
+      // We set up the options to continue one instruction past where
+      // the previous iteration stopped.
+      m_options.m_dumper_options.skip = 1;
+      m_options.m_dumper_options.id = m_last_id;
     }
 
-    m_dumpers[thread_sp->GetID()]->DumpInstructions(s, m_options.m_count);
+    const TraceSP &trace_sp = m_exe_ctx.GetTargetSP()->GetTrace();
+    TraceInstructionDumper dumper(trace_sp->GetCursor(*thread_sp), s,
+                                  m_options.m_dumper_options);
+    m_last_id = dumper.DumpInstructions(m_options.m_count);
     return true;
   }
 
   CommandOptions m_options;
-  std::map<lldb::tid_t, std::unique_ptr<TraceInstructionDumper>> m_dumpers;
+  // Last traversed id used to continue a repeat command. None means
+  // that all the trace has been consumed.
+  llvm::Optional<lldb::user_id_t> m_last_id;
 };
 
 // CommandObjectTraceDumpInfo

lldb/source/Commands/Options.td

@@ -1097,25 +1097,30 @@ let Command = "thread plan list" in {
 }
 
 let Command = "thread trace dump instructions" in {
-  def thread_trace_dump_instructions_forwards: Option<"forwards", "f">, Group<1>,
+  def thread_trace_dump_instructions_forwards: Option<"forwards", "f">,
+    Group<1>,
     Desc<"If specified, the trace is traversed forwards chronologically "
     "starting at the oldest instruction. Otherwise, it starts at the most "
     "recent one and the traversal is backwards.">;
   def thread_trace_dump_instructions_count : Option<"count", "c">, Group<1>,
     Arg<"Count">,
     Desc<"The number of instructions to display starting at the most recent "
     "instruction, or the oldest if --forwards is provided.">;
-  def thread_trace_dump_instructions_skip: Option<"skip", "s">,
-    Group<1>,
+  def thread_trace_dump_instructions_id: Option<"id", "i">, Group<1>,
     Arg<"Index">,
-    Desc<"How many instruction to skip from the end of the trace to start "
-    "dumping instructions, or from the beginning if --forwards is provided">;
+    Desc<"Custom starting instruction id from where to start traversing. This "
+    "id can be provided in decimal or hexadecimal representation.">;
+  def thread_trace_dump_instructions_skip: Option<"skip", "s">, Group<1>,
+    Arg<"Index">,
+    Desc<"How many instruction to skip from the starting position of the trace "
+    "before starting the traversal.">;
   def thread_trace_dump_instructions_raw : Option<"raw", "r">,
     Group<1>,
-    Desc<"Dump only instruction address without disassembly nor symbol information.">;
-  def thread_trace_dump_instructions_show_tsc : Option<"tsc", "t">,
-    Group<1>,
-    Desc<"For each instruction, print the corresponding timestamp counter if available.">;
+    Desc<"Dump only instruction address without disassembly nor symbol "
+    "information.">;
+  def thread_trace_dump_instructions_show_tsc : Option<"tsc", "t">, Group<1>,
+    Desc<"For each instruction, print the corresponding timestamp counter if "
+    "available.">;
   def thread_trace_dump_instructions_continue: Option<"continue", "C">,
     Group<1>,
     Desc<"Continue dumping instructions right where the previous invocation of this "