Skip to content

[fuzzer] fix clang-cl build fuzzer lit test failure #112339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

yingcong-wu
Copy link
Contributor

@yingcong-wu yingcong-wu commented Oct 15, 2024

The check-fuzzer runs fine with cl build llvm, but the following lit tests fail with clang-cl build llvm

********************
Timed Out Tests (2):
  libFuzzer-x86_64-default-Windows :: fork-ubsan.test
  libFuzzer-x86_64-default-Windows :: fuzzer-oom.test

********************
Failed Tests (22):
  libFuzzer-x86_64-default-Windows :: acquire-crash-state.test
  libFuzzer-x86_64-default-Windows :: cross_over_copy.test
  libFuzzer-x86_64-default-Windows :: cross_over_insert.test
  libFuzzer-x86_64-default-Windows :: exit_on_src_pos.test
  libFuzzer-x86_64-default-Windows :: fuzzer-alignment-assumption.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-integer-sign-change.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation-or-sign-change.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-unsigned-integer-truncation.test
  libFuzzer-x86_64-default-Windows :: fuzzer-printcovpcs.test
  libFuzzer-x86_64-default-Windows :: fuzzer-timeout.test
  libFuzzer-x86_64-default-Windows :: fuzzer-ubsan.test
  libFuzzer-x86_64-default-Windows :: minimize_crash.test
  libFuzzer-x86_64-default-Windows :: minimize_two_crashes.test
  libFuzzer-x86_64-default-Windows :: null-deref-on-empty.test
  libFuzzer-x86_64-default-Windows :: null-deref.test
  libFuzzer-x86_64-default-Windows :: print-func.test
  libFuzzer-x86_64-default-Windows :: stack-overflow-with-asan.test
  libFuzzer-x86_64-default-Windows :: trace-malloc-2.test
  libFuzzer-x86_64-default-Windows :: trace-malloc-unbalanced.test
  libFuzzer-x86_64-default-Windows :: trace-malloc.test

The related commits are 53a81d4 and e31efd8. Following the change in e31efd8 can fix these failures.

As for the issue mentioned in the comment that alternatename support in clang not good enough(https://bugs.llvm.org/show_bug.cgi?id=40218). I find that using __builtin_function_start(func) instead of directly using func would make it work as intended.

@llvmbot
Copy link
Member

llvmbot commented Oct 15, 2024

@llvm/pr-subscribers-compiler-rt-sanitizer

Author: Wu Yingcong (yingcong-wu)

Changes

The check-fuzzer runs fine with cl build llvm, but the following lit tests fail with clang-cl build llvm

********************
Timed Out Tests (2):
  libFuzzer-x86_64-default-Windows :: fork-ubsan.test
  libFuzzer-x86_64-default-Windows :: fuzzer-oom.test

********************
Failed Tests (22):
  libFuzzer-x86_64-default-Windows :: acquire-crash-state.test
  libFuzzer-x86_64-default-Windows :: cross_over_copy.test
  libFuzzer-x86_64-default-Windows :: cross_over_insert.test
  libFuzzer-x86_64-default-Windows :: exit_on_src_pos.test
  libFuzzer-x86_64-default-Windows :: fuzzer-alignment-assumption.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-integer-sign-change.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation-or-sign-change.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-signed-integer-truncation.test
  libFuzzer-x86_64-default-Windows :: fuzzer-implicit-unsigned-integer-truncation.test
  libFuzzer-x86_64-default-Windows :: fuzzer-printcovpcs.test
  libFuzzer-x86_64-default-Windows :: fuzzer-timeout.test
  libFuzzer-x86_64-default-Windows :: fuzzer-ubsan.test
  libFuzzer-x86_64-default-Windows :: minimize_crash.test
  libFuzzer-x86_64-default-Windows :: minimize_two_crashes.test
  libFuzzer-x86_64-default-Windows :: null-deref-on-empty.test
  libFuzzer-x86_64-default-Windows :: null-deref.test
  libFuzzer-x86_64-default-Windows :: print-func.test
  libFuzzer-x86_64-default-Windows :: stack-overflow-with-asan.test
  libFuzzer-x86_64-default-Windows :: trace-malloc-2.test
  libFuzzer-x86_64-default-Windows :: trace-malloc-unbalanced.test
  libFuzzer-x86_64-default-Windows :: trace-malloc.test

The related commits are 53a81d4 and e31efd8. Following the change in e31efd8 can fix these failures.

As for the issue mentioned in the comment that alternatename support in clang not good enough(https://bugs.llvm.org/show_bug.cgi?id=40218). I find that using __builtin_function_start() would make it works as intended.


Full diff: https://github.com/llvm/llvm-project/pull/112339.diff

1 Files Affected:

  • (modified) compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp (+14-14)
diff --git a/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp b/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
index 688bad1d51ca5b..dfc32ac9db2979 100644
--- a/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
+++ b/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
@@ -22,6 +22,11 @@ using namespace fuzzer;
 #define STRINGIFY(A) STRINGIFY_(A)
 
 #if LIBFUZZER_MSVC
+#define GET_FUNCTION_ADDRESS(fn) &fn
+#else
+#define GET_FUNCTION_ADDRESS(fn) __builtin_function_start(fn)
+#endif // LIBFUZER_MSVC
+
 // Copied from compiler-rt/lib/sanitizer_common/sanitizer_win_defs.h
 #if defined(_M_IX86) || defined(__i386__)
 #define WIN_SYM_PREFIX "_"
@@ -31,17 +36,9 @@ using namespace fuzzer;
 
 // Declare external functions as having alternativenames, so that we can
 // determine if they are not defined.
-#define EXTERNAL_FUNC(Name, Default)                                   \
-  __pragma(comment(linker, "/alternatename:" WIN_SYM_PREFIX STRINGIFY( \
+#define EXTERNAL_FUNC(Name, Default)                                           \
+  __pragma(comment(linker, "/alternatename:" WIN_SYM_PREFIX STRINGIFY(         \
                                Name) "=" WIN_SYM_PREFIX STRINGIFY(Default)))
-#else
-// Declare external functions as weak to allow them to default to a specified
-// function if not defined explicitly. We must use weak symbols because clang's
-// support for alternatename is not 100%, see
-// https://bugs.llvm.org/show_bug.cgi?id=40218 for more details.
-#define EXTERNAL_FUNC(Name, Default) \
-  __attribute__((weak, alias(STRINGIFY(Default))))
-#endif  // LIBFUZZER_MSVC
 
 extern "C" {
 #define EXT_FUNC(NAME, RETURN_TYPE, FUNC_SIG, WARN)         \
@@ -57,20 +54,23 @@ extern "C" {
 }
 
 template <typename T>
-static T *GetFnPtr(T *Fun, T *FunDef, const char *FnName, bool WarnIfMissing) {
+static T *GetFnPtr(void *Fun, void *FunDef, const char *FnName,
+                   bool WarnIfMissing) {
   if (Fun == FunDef) {
     if (WarnIfMissing)
       Printf("WARNING: Failed to find function \"%s\".\n", FnName);
     return nullptr;
   }
-  return Fun;
+  return (T *)Fun;
 }
 
 namespace fuzzer {
 
 ExternalFunctions::ExternalFunctions() {
-#define EXT_FUNC(NAME, RETURN_TYPE, FUNC_SIG, WARN) \
-  this->NAME = GetFnPtr<decltype(::NAME)>(::NAME, ::NAME##Def, #NAME, WARN);
+#define EXT_FUNC(NAME, RETURN_TYPE, FUNC_SIG, WARN)                            \
+  this->NAME = GetFnPtr<decltype(::NAME)>(GET_FUNCTION_ADDRESS(::NAME),        \
+                                          GET_FUNCTION_ADDRESS(::NAME##Def),   \
+                                          #NAME, WARN);
 
 #include "FuzzerExtFunctions.def"
 

@yingcong-wu yingcong-wu merged commit d54953e into llvm:main Oct 17, 2024
11 checks passed
@Zentrik
Copy link
Contributor

Zentrik commented Oct 17, 2024

This broke building for me on x64 and i686 using mingw to build as a shared object. Logs at https://github.com/Zentrik/llvm_julia_tester/actions/runs/11392238393/job/31697854988#step:9:5200. Unfortunately, I'm not going to have time to look into this until next week, but I'll see if I can figure out what's going on then.

@yingcong-wu
Copy link
Contributor Author

Okay, I will take a look when I have time too. @Zentrik Please keep me posted for any updates on this. Thank you.

@yingcong-wu
Copy link
Contributor Author

I can see it fails with

2024-10-17T20:15:06.2079276Z [20:15:06] ninja: job failed: /opt/bin/x86_64-w64-mingw32-libgfortran5-cxx11/x86_64-w64-mingw32-clang++ --target=x86_64-w64-mingw32 --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/ -D_DEBUG -D_FILE_OFFSET_BITS=64 -D_GLIBCXX_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/workspace/build/projects/compiler-rt/lib/fuzzer -I/workspace/srcdir/llvm-project/compiler-rt/lib/fuzzer -I/workspace/build/include -I/workspace/srcdir/llvm-project/llvm/include -I/workspace/srcdir/llvm-project/compiler-rt/lib/fuzzer/../../include -remap -D__USING_SJLJ_EXCEPTIONS__ -D__CRT__NO_INLINE -pthread -DMLIR_CAPI_ENABLE_WINDOWS_DLL_DECLSPEC -Dmlir_arm_sme_abi_stubs_EXPORTS -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -ffunction-sections -fdata-sections -Wall -O3 -DNDEBUG -m64 -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -fms-extensions -fno-omit-frame-pointer -std=c++17 -MD -MT projects/compiler-rt/lib/fuzzer/CMakeFiles/RTfuzzer.x86_64.dir/FuzzerExtFunctionsWindows.cpp.obj -MF projects/compiler-rt/lib/fuzzer/CMakeFiles/RTfuzzer.x86_64.dir/FuzzerExtFunctionsWindows.cpp.obj.d -o projects/compiler-rt/lib/fuzzer/CMakeFiles/RTfuzzer.x86_64.dir/FuzzerExtFunctionsWindows.cpp.obj -c /workspace/srcdir/llvm-project/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp

But no any detailed error information is provided.

We can guard this change to exclude MinGW. But I think it is best for us to root-cause this problem first.

@yingcong-wu yingcong-wu deleted the yc/1015-fuzzer-clang-build-failure branch November 13, 2024 08:49
@Zentrik
Copy link
Contributor

Zentrik commented Jan 28, 2025

diff --git a/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp b/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
index dfc32ac9db29..7d0c7573f2e0 100644
--- a/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
+++ b/compiler-rt/lib/fuzzer/FuzzerExtFunctionsWindows.cpp
@@ -24,7 +24,7 @@ using namespace fuzzer;
 #if LIBFUZZER_MSVC
 #define GET_FUNCTION_ADDRESS(fn) &fn
 #else
-#define GET_FUNCTION_ADDRESS(fn) __builtin_function_start(fn)
+#define GET_FUNCTION_ADDRESS(fn) (void *)(&fn)
 #endif // LIBFUZER_MSVC
 
 // Copied from compiler-rt/lib/sanitizer_common/sanitizer_win_defs.h

Fixed the build issue for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants