[compiler-rt][rtsan] Use Die instead of exit, define cf.exitcode

[cjappl]

This is typical sanitizer behavior.

Specifically, what die does is aborts on mac by default, and exits on other platforms. This gives the ability to collect very nice stack traces (see info here https://discourse.llvm.org/t/why-do-sanitizers-abort-on-error-by-default-on-mac-and-android/80807)

With this, we need to do a few things:

• Turn off abort_on_error on our unit tests, both lit and unit. I grabbed code from asan that does this, along with helpful comments as to why it is being done.
• Set an error code that is unique to rtsan.

A note on the error code. It must be < 255 on mac at least, I chose the 808 drum machine as a little nod to audio tech. Happy to change this if you have another favorite number.

[cjappl]

CC for review @davidtrevelyan

[llvmbot]

@llvm/pr-subscribers-compiler-rt-sanitizer

Author: Chris Apple (cjappl)

Changes

This is typical sanitizer behavior.

Specifically, what die does is aborts on mac by default, and exits on other platforms. This gives the ability to collect very nice stack traces (see info here https://discourse.llvm.org/t/why-do-sanitizers-abort-on-error-by-default-on-mac-and-android/80807)

With this, we need to do a few things:

• Turn off abort_on_error on our unit tests, both lit and unit. I grabbed code from asan that does this, along with helpful comments as to why it is being done.
• Set an error code that is unique to rtsan.

A note on the error code. It must be < 255 on mac at least, I chose the 808 drum machine as a little nod to audio tech. Happy to change this if you have another favorite number.

---

Full diff: https://github.com/llvm/llvm-project/pull/107635.diff

6 Files Affected:

• (modified) compiler-rt/lib/rtsan/rtsan_context.cpp (+1-1)
• (modified) compiler-rt/lib/rtsan/rtsan_flags.cpp (+1)
• (modified) compiler-rt/lib/rtsan/tests/rtsan_test_main.cpp (+17)
• (modified) compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h (+2-2)
• (modified) compiler-rt/test/rtsan/basic.cpp (+1-1)
• (modified) compiler-rt/test/rtsan/lit.cfg.py (+16)

diff --git a/compiler-rt/lib/rtsan/rtsan_context.cpp b/compiler-rt/lib/rtsan/rtsan_context.cpp
index a49b70360babbd..5efb0aa99a86e2 100644
--- a/compiler-rt/lib/rtsan/rtsan_context.cpp
+++ b/compiler-rt/lib/rtsan/rtsan_context.cpp
@@ -62,7 +62,7 @@ static __rtsan::Context &GetContextForThisThreadImpl() {
     Until then, and to keep the first PRs small, only the exit mode
     is available.
 */
-static void InvokeViolationDetectedAction() { exit(EXIT_FAILURE); }
+static void InvokeViolationDetectedAction() { Die(); }
 
 __rtsan::Context::Context() = default;
 
diff --git a/compiler-rt/lib/rtsan/rtsan_flags.cpp b/compiler-rt/lib/rtsan/rtsan_flags.cpp
index beab2a2fc5d895..70ac5c6dae2f40 100644
--- a/compiler-rt/lib/rtsan/rtsan_flags.cpp
+++ b/compiler-rt/lib/rtsan/rtsan_flags.cpp
@@ -35,6 +35,7 @@ void __rtsan::InitializeFlags() {
   {
     CommonFlags cf;
     cf.CopyFrom(*common_flags());
+    cf.exitcode = 43; // (TR-)808 % 255 = 43
     cf.external_symbolizer_path = GetEnv("RTSAN_SYMBOLIZER_PATH");
     OverrideCommonFlags(cf);
   }
diff --git a/compiler-rt/lib/rtsan/tests/rtsan_test_main.cpp b/compiler-rt/lib/rtsan/tests/rtsan_test_main.cpp
index 255ac9497103e9..50c726e09f287f 100644
--- a/compiler-rt/lib/rtsan/tests/rtsan_test_main.cpp
+++ b/compiler-rt/lib/rtsan/tests/rtsan_test_main.cpp
@@ -8,8 +8,25 @@
 //
 //===----------------------------------------------------------------------===//
 
+#include "sanitizer_common/sanitizer_platform.h"
 #include "sanitizer_test_utils.h"
 
+// Default RTSAN_OPTIONS for the unit tests.
+extern "C" const char *__rtsan_default_options() {
+#if SANITIZER_APPLE
+  // On Darwin, we default to `abort_on_error=1`, which would make tests run
+  // much slower. Let's override this and run lit tests with 'abort_on_error=0'
+  // and make sure we do not overwhelm the syslog while testing. Also, let's
+  // turn symbolization off to speed up testing, especially when not running
+  // with llvm-symbolizer but with atos.
+  return "symbolize=false:abort_on_error=0:log_to_syslog=0";
+#else
+  // Let's turn symbolization off to speed up testing (more than 3 times speedup
+  // observed).
+  return "symbolize=false";
+#endif
+}
+
 int main(int argc, char **argv) {
   testing::GTEST_FLAG(death_test_style) = "threadsafe";
   testing::InitGoogleTest(&argc, argv);
diff --git a/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h b/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h
index 6ca09cf6570940..3d2a30954ce58c 100644
--- a/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h
+++ b/compiler-rt/lib/rtsan/tests/rtsan_test_utilities.h
@@ -36,8 +36,8 @@ void ExpectRealtimeDeath(Function &&Func,
                : "";
   };
 
-  EXPECT_EXIT(RealtimeInvoke(std::forward<Function>(Func)),
-              ExitedWithCode(EXIT_FAILURE), GetExpectedErrorSubstring());
+  EXPECT_EXIT(RealtimeInvoke(std::forward<Function>(Func)), ExitedWithCode(43),
+              GetExpectedErrorSubstring());
 }
 
 template <typename Function> void ExpectNonRealtimeSurvival(Function &&Func) {
diff --git a/compiler-rt/test/rtsan/basic.cpp b/compiler-rt/test/rtsan/basic.cpp
index c7cbfcda31562e..4e0768cce6d916 100644
--- a/compiler-rt/test/rtsan/basic.cpp
+++ b/compiler-rt/test/rtsan/basic.cpp
@@ -1,5 +1,5 @@
 // RUN: %clangxx -fsanitize=realtime %s -o %t
-// RUN: not %run %t 2>&1 | FileCheck %s
+// RUN: %env_rtsan_opts=abort_on_error=0 not %run %t 2>&1 | FileCheck %s
 // UNSUPPORTED: ios
 
 // Intent: Ensure that an intercepted call in a [[clang::nonblocking]] function
diff --git a/compiler-rt/test/rtsan/lit.cfg.py b/compiler-rt/test/rtsan/lit.cfg.py
index b262ecfa7fb4bb..ad3e46dba04730 100644
--- a/compiler-rt/test/rtsan/lit.cfg.py
+++ b/compiler-rt/test/rtsan/lit.cfg.py
@@ -3,6 +3,22 @@
 # Setup config name.
 config.name = "RTSAN" + config.name_suffix
 
+
+default_rtsan_opts = ""
+
+if config.host_os == "Darwin":
+    # On Darwin, we default to `abort_on_error=1`, which would make tests run
+    # much slower. Let's override this and run lit tests with 'abort_on_error=0'.
+    default_rtsan_opts += ":abort_on_error=0"
+
+if default_rtsan_opts:
+    config.environment["RTSAN_OPTIONS"] = default_rtsan_opts
+    default_rtsan_opts += ":"
+
+config.substitutions.append(
+    ("%env_rtsan_opts=", "env RTSAN_OPTIONS=" + default_rtsan_opts)
+)
+
 # Setup source root.
 config.test_source_root = os.path.dirname(__file__)
 

[cjappl]

This does fairly drastically speed up our unit tests. The difference is very noticeable

[fmayer]

A note on the error code. It must be < 255 on mac at least

just FYI, on Linux, I would make it < 128 (which you do) to not get into the signal exit codes.

[cjappl]

Excellent, thanks for the review @fmayer