Skip to content

Reapply "[llvm-jitlink] Use concurrent linking by default." with fixes. #120958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 23, 2024

Conversation

lhames
Copy link
Contributor

@lhames lhames commented Dec 23, 2024

Reapplies commit edca1d9 which was reverted in 34531cf while I investigated bot failures, (e.g.
https://lab.llvm.org/buildbot/#/builders/137/builds/10791).

Commit 158a600 should address the -check failures on the bots, which were caused by checks running earlier under the concurrent linking scheme, before all files referenced by the checks had been fully linked. This patch also fixes the -threads option failure by renaming the option to -num-threads to avoid clashing with the ThreadCount cl::opt variable defined in ThinLTOCodeGenerator.cpp.

Reapplies commit edca1d9 which was reverted in 34531cf while I
investigated bot failures, (e.g.
https://lab.llvm.org/buildbot/#/builders/137/builds/10791).

Commit 158a600 should address the -check failures on the bots, which were
caused by checks running earlier under the concurrent linking scheme, before
all files referenced by the checks had been fully linked. This patch also
fixes the -threads option failure by renaming the option to -num-threads to
avoid clashing with the ThreadCount cl::opt variable defined in
ThinLTOCodeGenerator.cpp.
@lhames lhames force-pushed the jitlink-enable-concurrency branch from 8bdbcda to 38eb730 Compare December 23, 2024 22:12
@lhames lhames merged commit 93d4b1f into llvm:main Dec 23, 2024
8 checks passed
@lhames lhames deleted the jitlink-enable-concurrency branch December 23, 2024 23:19
@llvm-ci
Copy link
Collaborator

llvm-ci commented Dec 23, 2024

LLVM Buildbot has detected a new failure on builder clang-debian-cpp20 running on clang-debian-cpp20 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/108/builds/7490

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 2: /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/test/ExecutionEngine/JITLink/x86-64/Output/LocalDependencyPropagation.s.tmp /vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s
+ /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/test/ExecutionEngine/JITLink/x86-64/Output/LocalDependencyPropagation.s.tmp /vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s
RUN: at line 3: /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/llvm-jitlink -debug-only=orc -noexec -abs _external_func=0x1    -entry=_foo /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/test/ExecutionEngine/JITLink/x86-64/Output/LocalDependencyPropagation.s.tmp 2>&1 | /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/FileCheck /vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s
+ /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/FileCheck /vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s
+ /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/bin/llvm-jitlink -debug-only=orc -noexec -abs _external_func=0x1 -entry=_foo /vol/worker/clang-debian-cpp20/clang-debian-cpp20/build/test/ExecutionEngine/JITLink/x86-64/Output/LocalDependencyPropagation.s.tmp
/vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s:17:10: error: CHECK: expected string not found in input
# CHECK: Simplified dependencies:
         ^
<stdin>:85:65: note: scanning from here
 Symbols: { _foo }, Dependencies: { (main, { _external_func }) }
                                                                ^
<stdin>:93:13: note: possible intended match here
Destroying JITDylib Process
            ^

Input file: <stdin>
Check file: /vol/worker/clang-debian-cpp20/clang-debian-cpp20/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/LocalDependencyPropagation.s

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            .
            .
            .
           80:  Symbols: { }, Dependencies: { } 
           81:  _foo }, Dependencies: { (main, { _external_func }) } 
           82:  Symbols: { _baz }, Dependencies: { (main, { _foo }) } 
           83:  Simplified dependencies: 
           84:  Symbols: { _baz }, Dependencies: { (main, { _external_func }) } 
           85:  Symbols: { _foo }, Dependencies: { (main, { _external_func }) } 
check:17'0                                                                     X error: no match found
           86: Dispatching: Execute query complete callback for { ("_foo": 0x7c4d80001000 [Callable]) } 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           87: Ending ExecutionSession 0x5b6b42e42da0 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           88: In main removing tracker 0x5b6b42e43940 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           89: In main destroying tracker 0x5b6b42e43940 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           90: In Process removing tracker 0x5b6b42e46410 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           91: In Process destroying tracker 0x5b6b42e46410 
check:17'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           92: Destroying JITDylib main 
...

@chapuni
Copy link
Contributor

chapuni commented Dec 24, 2024

chapuni added a commit that referenced this pull request Dec 24, 2024
…ith fixes. (#120958)"

Caused random failures.

This reverts commit 93d4b1f.
(llvmorg-20-init-16299-g93d4b1f7a72f)
@lhames
Copy link
Contributor Author

lhames commented Dec 24, 2024

Thanks for the revert @chapuni! I'll take a look when I get back to my desk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants