Increase timeout to reduce test failure rate.

[rastogishubham]

The timeout for this test was set to 1.0s which is very low, it should be a default of 10s and be increased by a factor of 10 if ASAN is enabled. This will help reduce the falkiness of the test, especially in ASAN builds.

[llvmbot]

@llvm/pr-subscribers-lldb

Author: Shubham Sandeep Rastogi (rastogishubham)

Changes

The timeout for this test was set to 1.0s which is very low, it should be a default of 10s and be increased by a factor of 10 if ASAN is enabled. This will help reduce the falkiness of the test, especially in ASAN builds.

---

Full diff: https://github.com/llvm/llvm-project/pull/83312.diff

1 Files Affected:

• (modified) lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py (+9-6)

diff --git a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
index 04d741c1d47201..635ec4f0baf190 100644
--- a/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
+++ b/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
@@ -44,7 +44,8 @@ def test_termination(self):
         self.dap_server.request_disconnect()
 
         # Wait until the underlying lldb-dap process dies.
-        self.dap_server.process.wait(timeout=10)
+        timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+        self.dap_server.process.wait(timeout=timeoutval)
 
         # Check the return code
         self.assertEqual(self.dap_server.process.poll(), 0)
@@ -334,14 +335,15 @@ def test_commands(self):
         # Get output from the console. This should contain both the
         # "stopCommands" that were run after the first breakpoint was hit
         self.continue_to_breakpoints(breakpoint_ids)
-        output = self.get_console(timeout=1.0)
+        timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+        output = self.get_console(timeout=timeoutval)
         self.verify_commands("stopCommands", output, stopCommands)
 
         # Continue again and hit the second breakpoint.
         # Get output from the console. This should contain both the
         # "stopCommands" that were run after the second breakpoint was hit
         self.continue_to_breakpoints(breakpoint_ids)
-        output = self.get_console(timeout=1.0)
+        output = self.get_console(timeout=timeoutval)
         self.verify_commands("stopCommands", output, stopCommands)
 
         # Continue until the program exits
@@ -402,21 +404,22 @@ def test_extra_launch_commands(self):
         self.verify_commands("launchCommands", output, launchCommands)
         # Verify the "stopCommands" here
         self.continue_to_next_stop()
-        output = self.get_console(timeout=1.0)
+        timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+        output = self.get_console(timeout=timeoutval)
         self.verify_commands("stopCommands", output, stopCommands)
 
         # Continue and hit the second breakpoint.
         # Get output from the console. This should contain both the
         # "stopCommands" that were run after the first breakpoint was hit
         self.continue_to_next_stop()
-        output = self.get_console(timeout=1.0)
+        output = self.get_console(timeout=timeoutval)
         self.verify_commands("stopCommands", output, stopCommands)
 
         # Continue until the program exits
         self.continue_to_exit()
         # Get output from the console. This should contain both the
         # "exitCommands" that were run after the second breakpoint was hit
-        output = self.get_console(timeout=1.0)
+        output = self.get_console(timeout=timeoutval)
         self.verify_commands("exitCommands", output, exitCommands)
 
     @skipIfWindows

[github-actions]

⚠️ Python code formatter, darker found issues in your code. ⚠️

You can test this locally with the following command:

darker --check --diff -r c1b8c6cf41df4a148e7a89c3a3c7e8049b0a47af...cef723783edd5ba89fd5b9b0ec800fa1439ffb90 lldb/packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py

View the diff from darker here.

--- packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py	2024-02-29 21:11:31.000000 +0000
+++ packages/Python/lldbsuite/test/tools/lldb-dap/lldbdap_testcase.py	2024-02-29 21:14:13.400696 +0000
@@ -6,11 +6,11 @@
 
 
 class DAPTestCaseBase(TestBase):
     # set timeout based on whether ASAN was enabled or not. Increase
     # timeout by a factor of 10 if ASAN is enabled.
-    timeoutval = 10 * (10 if ('ASAN_OPTIONS' in os.environ) else 1)
+    timeoutval = 10 * (10 if ("ASAN_OPTIONS" in os.environ) else 1)
     NO_DEBUG_INFO_TESTCASE = True
 
     def create_debug_adaptor(self, lldbDAPEnv=None):
         """Create the Visual Studio Code debug adaptor"""
         self.assertTrue(
--- test/API/tools/lldb-dap/launch/TestDAP_launch.py	2024-02-29 21:11:31.000000 +0000
+++ test/API/tools/lldb-dap/launch/TestDAP_launch.py	2024-02-29 21:14:13.545884 +0000
@@ -42,11 +42,13 @@
         # The lldb-dap process should finish even though
         # we didn't close the communication socket explicitly
         self.dap_server.request_disconnect()
 
         # Wait until the underlying lldb-dap process dies.
-        self.dap_server.process.wait(timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval)
+        self.dap_server.process.wait(
+            timeout=lldbdap_testcase.DAPTestCaseBase.timeoutval
+        )
 
         # Check the return code
         self.assertEqual(self.dap_server.process.poll(), 0)
 
     @skipIfWindows

[walter-erquinigo]

could you create instead a variable at the base test class level with a suggested timeout that can be used by other DAP tests when setting timeouts? I'm pretty sure at least one other test file uses timeouts.

[walter-erquinigo]

Other than a missing comment, this LGTM. Thanks!