[lldb] Make SBProcess thread related actions listen to StopLocker (#134339)

# Summary

This PR updates `SBProcess::GetNumThreads()` and
`SBProcess::GetThreadAtIndex()` to listen to the stop locker.
`SBProcess::GetNumThreads()` will return 0 if the process is running.

## Problem Description

Recently upon debugging a program with thousands of threads in VS Code,
lldb-dap would hang at a `threads` request sent right after receiving
the `configurationDone` response. Soon after it will end the debug
session with the following error
```
Process <pid> exited with status = -1 (0xffffffff) lost connection
```

This is because LLDB is still in the middle of resuming all the threads.
And requesting threads will end up interrupt the process on Linux. From
the gdb-remote log it ended up getting `lldb::StateType::eStateInvalid`
and just exit with status -1.

I don't think it's reasonable to allow getting threads from a running
process. There are a few approaches to fix this:
1) Send the stopped event to IDE after `configurationDone`. This aligns
with the CLI behavior.
2) However, the above approach will break the existing user facing
behavior. The alternative will be reject the `threads` request if the
process is not stopped.
3) Improve the run lock. This is a synchronize issue where process was
in the middle of resuming while lldb-dap attempts to interrupt it.

**This PR implements the option 3**

## HOWEVER

This fixed the "lost connection" issue below but new issue has surfaced.
From testing, and also from checking the [VSCode source
code](https://github.com/microsoft/vscode/blob/174af221c9ea2ccdb64abe4aab8e1a805e77beae/src/vs/workbench/contrib/debug/browser/debugSession.ts#L791),
it expects having threadID to perform `pause`. So after attaching,
without any threads reported to the client, the user will not be able to
pause the attached process. `setBreakpoint` will still work and once we
make a stop at the bp (or any stop that will report threads, client can
perform pause again.

## NEXT
1) Made an attempt to return initial thread list so that VSCode can
pause (second commit in the PR)
2) Investigate why threads will trigger unwinding the second frame of a
thread, which leads to sending the interrupt
3) Decided if we want to support `stopOnEntry` for attaching, given
  i. This is not an official specification
ii. If enable stopOnEntry, we need to fix attaching on Linux, to send
only one stopped event. Currently, all threads upon attaching will have
stop reason `SIGSTOP` and lldb-dap will send `stopped` event for each
one of them. Every `stopped` will trigger the client request for
threads.
iii. Alternatively, we can support auto continue correspond to `(lldb)
process attach --continue`. This require the ii above.


### Additionally

lldb-dap will not send a `continued` event after `configurationDone`
because it checks `dap.focus_tid == LLDB_INVALID_THREAD_ID` (so that we
don't send it for `launch` request). Notice `dap.focus_tid` will only
get assigned when handling stop or stepping.

According to DAP

> Please note: a debug adapter is not expected to send this event in
response to a request that implies that execution continues, e.g. launch
or continue.
It is only necessary to send a continued event if there was no previous
request that implied this.

So I guess we are not violating DAP if we don't send `continued` event.
But I'd like to get some sense about this.


## Test Plan
Used following program for testing:
https://gist.github.com/kusmour/1729d2e07b7b1063897db77de194e47d
**NOTE: Utilize stdin to get pid and attach AFTER hitting enter. Attach
should happen when all the threads start running.**

DAP messages before the change
<img width="1165" alt="image"
src="https://github.com/user-attachments/assets/a9ad85fb-81ce-419c-95e5-612639905c66"
/>

DAP message after the change - report zero threads after attaching
<img width="1165" alt="image"
src="https://github.com/user-attachments/assets/a1179e18-6844-437a-938c-0383702294cd"
/>

---------

Co-authored-by: Jonas Devlieghere <[email protected]>

Author: kusmour

lldb/packages/Python/lldbsuite/test/tools/lldb-dap/dap_server.py

@@ -649,6 +649,10 @@ def request_configurationDone(self):
         response = self.send_recv(command_dict)
         if response:
             self.configuration_done_sent = True
+            # Client requests the baseline of currently existing threads after
+            # a successful launch or attach.
+            # Kick off the threads request that follows
+            self.request_threads()
         return response
 
     def _process_stopped(self):

lldb/source/API/SBProcess.cpp

@@ -193,10 +193,11 @@ uint32_t SBProcess::GetNumThreads() {
   if (process_sp) {
     Process::StopLocker stop_locker;
 
-    const bool can_update = stop_locker.TryLock(&process_sp->GetRunLock());
-    std::lock_guard<std::recursive_mutex> guard(
-        process_sp->GetTarget().GetAPIMutex());
-    num_threads = process_sp->GetThreadList().GetSize(can_update);
+    if (stop_locker.TryLock(&process_sp->GetRunLock())) {
+      std::lock_guard<std::recursive_mutex> guard(
+          process_sp->GetTarget().GetAPIMutex());
+      num_threads = process_sp->GetThreadList().GetSize();
+    }
   }
 
   return num_threads;
@@ -393,11 +394,12 @@ SBThread SBProcess::GetThreadAtIndex(size_t index) {
   ProcessSP process_sp(GetSP());
   if (process_sp) {
     Process::StopLocker stop_locker;
-    const bool can_update = stop_locker.TryLock(&process_sp->GetRunLock());
-    std::lock_guard<std::recursive_mutex> guard(
-        process_sp->GetTarget().GetAPIMutex());
-    thread_sp = process_sp->GetThreadList().GetThreadAtIndex(index, can_update);
-    sb_thread.SetThread(thread_sp);
+    if (stop_locker.TryLock(&process_sp->GetRunLock())) {
+      std::lock_guard<std::recursive_mutex> guard(
+          process_sp->GetTarget().GetAPIMutex());
+      thread_sp = process_sp->GetThreadList().GetThreadAtIndex(index, false);
+      sb_thread.SetThread(thread_sp);
+    }
   }
 
   return sb_thread;

lldb/test/API/tools/lldb-dap/attach/TestDAP_attach.py

@@ -1,5 +1,5 @@
 """
-Test lldb-dap setBreakpoints request
+Test lldb-dap attach request
 """
 
 

lldb/tools/lldb-dap/DAP.h

@@ -211,6 +211,9 @@ struct DAP {
   /// The set of features supported by the connected client.
   llvm::DenseSet<ClientFeature> clientFeatures;
 
+  /// The initial thread list upon attaching.
+  std::optional<llvm::json::Array> initial_thread_list;
+
   /// Creates a new DAP sessions.
   ///
   /// \param[in] log

lldb/tools/lldb-dap/Handler/ConfigurationDoneRequestHandler.cpp

@@ -44,6 +44,7 @@ namespace lldb_dap {
 //             just an acknowledgement, so no body field is required."
 //             }]
 // },
+
 void ConfigurationDoneRequestHandler::operator()(
     const llvm::json::Object &request) const {
   llvm::json::Object response;
@@ -52,8 +53,15 @@ void ConfigurationDoneRequestHandler::operator()(
   dap.configuration_done_sent = true;
   if (dap.stop_at_entry)
     SendThreadStoppedEvent(dap);
-  else
+  else {
+    // Client requests the baseline of currently existing threads after
+    // a successful launch or attach by sending a 'threads' request
+    // right after receiving the configurationDone response.
+    // Obtain the list of threads before we resume the process
+    dap.initial_thread_list =
+        GetThreads(dap.target.GetProcess(), dap.thread_format);
     dap.target.GetProcess().Continue();
+  }
 }
 
 } // namespace lldb_dap

lldb/tools/lldb-dap/Handler/ThreadsRequestHandler.cpp

@@ -50,16 +50,23 @@ namespace lldb_dap {
 // }
 void ThreadsRequestHandler::operator()(
     const llvm::json::Object &request) const {
-  lldb::SBProcess process = dap.target.GetProcess();
   llvm::json::Object response;
   FillResponse(request, response);
 
-  const uint32_t num_threads = process.GetNumThreads();
   llvm::json::Array threads;
-  for (uint32_t thread_idx = 0; thread_idx < num_threads; ++thread_idx) {
-    lldb::SBThread thread = process.GetThreadAtIndex(thread_idx);
-    threads.emplace_back(CreateThread(thread, dap.thread_format));
+  // Client requests the baseline of currently existing threads after
+  // a successful launch or attach by sending a 'threads' request
+  // right after receiving the configurationDone response.
+  // If no thread has reported to the client, it prevents something
+  // like the pause request from working in the running state.
+  // Return the cache of initial threads as the process might have resumed
+  if (dap.initial_thread_list) {
+    threads = dap.initial_thread_list.value();
+    dap.initial_thread_list.reset();
+  } else {
+    threads = GetThreads(dap.target.GetProcess(), dap.thread_format);
   }
+
   if (threads.size() == 0) {
     response["success"] = llvm::json::Value(false);
   }

lldb/tools/lldb-dap/JSONUtils.cpp

@@ -870,6 +870,19 @@ llvm::json::Value CreateThread(lldb::SBThread &thread, lldb::SBFormat &format) {
   return llvm::json::Value(std::move(object));
 }
 
+llvm::json::Array GetThreads(lldb::SBProcess process, lldb::SBFormat &format) {
+  lldb::SBMutex lock = process.GetTarget().GetAPIMutex();
+  std::lock_guard<lldb::SBMutex> guard(lock);
+
+  llvm::json::Array threads;
+  const uint32_t num_threads = process.GetNumThreads();
+  for (uint32_t thread_idx = 0; thread_idx < num_threads; ++thread_idx) {
+    lldb::SBThread thread = process.GetThreadAtIndex(thread_idx);
+    threads.emplace_back(CreateThread(thread, format));
+  }
+  return threads;
+}
+
 // "StoppedEvent": {
 //   "allOf": [ { "$ref": "#/definitions/Event" }, {
 //     "type": "object",

lldb/tools/lldb-dap/JSONUtils.h

@@ -414,6 +414,8 @@ llvm::json::Value CreateExtendedStackFrameLabel(lldb::SBThread &thread,
 ///     definition outlined by Microsoft.
 llvm::json::Value CreateThread(lldb::SBThread &thread, lldb::SBFormat &format);
 
+llvm::json::Array GetThreads(lldb::SBProcess process, lldb::SBFormat &format);
+
 /// Create a "StoppedEvent" object for a LLDB thread object.
 ///
 /// This function will fill in the following keys in the returned