Fix diagnostics-dsym.test on mac-arm64

[zeroomega]

The check ordering of diagnostics-dsym.test is wrong and it causes
test failure when running on mac-arm64 machine. This patch fixes it.

[zeroomega]

Example of the failure:

******************** TEST 'Clang :: InstallAPI/diagnostics-dsym.test' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 4: rm -rf /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp
+ rm -rf /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp
RUN: at line 5: split-file /Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp
+ split-file /Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp
RUN: at line 8: /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/clang --target=arm64-apple-macos11 -g -dynamiclib /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.c  -current_version 1 -compatibility_version 1 -L/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/usr/lib  -save-temps -dynamiclib  -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib -install_name /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib
+ /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/clang --target=arm64-apple-macos11 -g -dynamiclib /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.c -current_version 1 -compatibility_version 1 -L/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/usr/lib -save-temps -dynamiclib -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib -install_name /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib
RUN: at line 12: dsymutil /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dSYM
+ dsymutil /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dSYM
RUN: at line 14: not /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/clang-installapi -x c++ --target=arm64-apple-macos11  -install_name /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib   -current_version 1 -compatibility_version 1  -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/output.tbd -dynamiclib  --verify-against=/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib --dsym=/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dSYM  --verify-mode=Pedantic 2>&1 | /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/FileCheck /Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test
+ not /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/clang-installapi -x c++ --target=arm64-apple-macos11 -install_name /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib -current_version 1 -compatibility_version 1 -o /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/output.tbd -dynamiclib --verify-against=/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dylib --dsym=/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.dSYM --verify-mode=Pedantic
+ /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/bin/FileCheck /Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test
/Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test:23:10: error: CHECK: expected string not found in input
; CHECK: foo.c:1:0: error: no declaration found for exported symbol 'foo' in dynamic library
         ^
<stdin>:3:205: note: scanning from here
/Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.c:5:0: error: no declaration found for exported symbol 'bar' in dynamic library
                                                                                                                                                                                                            ^

Input file: <stdin>
Check file: /Volumes/Work/s/w/ir/x/w/llvm-llvm-project/clang/test/InstallAPI/diagnostics-dsym.test

-dump-input=help explains the following input dump.

Input was:
<<<<<<
          1: warning: violations found for arm64 [-Winstallapi-violation] 
          2: /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.c:1:0: error: no declaration found for exported symbol 'foo' in dynamic library 
          3: /Volumes/Work/s/w/ir/x/w/llvm_build/tools/clang/stage2-bins/tools/clang/test/InstallAPI/Output/diagnostics-dsym.test.tmp/foo.c:5:0: error: no declaration found for exported symbol 'bar' in dynamic library 
check:23                                                                                                                                                                                                                 X error: no match found
>>>>>>

--

********************

After this patch. It passes.

[llvmbot]

@llvm/pr-subscribers-clang

Author: Haowei (zeroomega)

Changes

The check ordering of diagnostics-dsym.test is wrong and it causes
test failure when running on mac-arm64 machine. This patch fixes it.

---

Full diff: https://github.com/llvm/llvm-project/pull/99399.diff

1 Files Affected:

• (modified) clang/test/InstallAPI/diagnostics-dsym.test (+1-1)

diff --git a/clang/test/InstallAPI/diagnostics-dsym.test b/clang/test/InstallAPI/diagnostics-dsym.test
index 42fa67a1f9b1e..fb760d1858ba3 100644
--- a/clang/test/InstallAPI/diagnostics-dsym.test
+++ b/clang/test/InstallAPI/diagnostics-dsym.test
@@ -19,8 +19,8 @@
 ; RUN: --verify-mode=Pedantic 2>&1 | FileCheck %s
 
 ; CHECK: violations found for arm64 
-; CHECK: foo.c:5:0: error: no declaration found for exported symbol 'bar' in dynamic library
 ; CHECK: foo.c:1:0: error: no declaration found for exported symbol 'foo' in dynamic library
+; CHECK: foo.c:5:0: error: no declaration found for exported symbol 'bar' in dynamic library
 
 ;--- foo.c
 int foo(void) {

[cyndyishida]

Thank you @zeroomega ! I am also running on arm64-mac and that test is passing. What is your cmake invocation? I'd like to dig into the underlying cause of non-stable ordering. It looks like the darwin-aarch64 is passing the test as well https://lab.llvm.org/buildbot/#/builders/190
In general, though, I'm not too worried about the ordering of diagnostics so maybe a CHECK-DAG would make the test resilient

[zeroomega]

Thank you @zeroomega ! I am also running on arm64-mac and that test is passing. What is your cmake invocation? I'd like to dig into the underlying cause of non-stable ordering. It looks like the darwin-aarch64 is passing the test as well https://lab.llvm.org/buildbot/#/builders/190 In general, though, I'm not too worried about the ordering of diagnostics so maybe a CHECK-DAG would make the test resilient

Our build is a bit complicated to setup since we use a clang (near tip of the tree) which linked to llvm-libcxx, as the host toolchain. But when I try to build the clang with the toolchain came with the xcode, the original test file passed, and after I swapped the ordering, it failed. So apparently the output ordering is a bit non-deterministic. And since bar is used later than foo, so natually foo should comes first and bar comes later. My guess is that underlying data structure for the symbols is probably using a C++ standard library and it has different implementation from libcxx and whatever was provided by xcode.

Let me test the CHECK-DAG solution locally and update my patch. I think that is a better solution.

[zeroomega]

@cyndyishida I updated the patch to use CHECK-DAG and confirm the test now pass under both XCode toolchain and our in house one. Please take a look. Thx.

[cyndyishida]

Thank you for looking into & resolving this again!