Skip to content

[PassBuilder][FatLTO] Expose FatLTO pipeline via pipeline string #146048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 30, 2025

Conversation

nikic
Copy link
Contributor

@nikic nikic commented Jun 27, 2025

Expose the FatLTO pipeline via -passes="fatlto-pre-link<Ox>", similar to all the other optimization pipelines. This is to allow reproducing it outside clang. (Possibly also useful for C API users.)

@nikic nikic requested review from ilovepi and aeubanks June 27, 2025 10:10
Expose the FatLTO pipeline via `-passes="fatlto-pre-link<Ox>"`,
similar to all the other optimization pipelines. This is to allow
reproducing it outside clang. (Possibly also useful for C API users.)
Copy link
Contributor

@ilovepi ilovepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I wish I had thought of this earlier.

@nikic nikic merged commit d7a3bdf into llvm:main Jun 30, 2025
7 checks passed
@nikic nikic deleted the fatlto-pipeline branch June 30, 2025 10:04
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 30, 2025

LLVM Buildbot has detected a new failure on builder ml-opt-rel-x86-64 running on ml-opt-rel-x86-64-b1 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/185/builds/21136

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: Other/fatlto.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
/b/ml-opt-rel-x86-64-b1/build/bin/opt -debug-pass-manager -passes='fatlto-pre-link<O2>' -disable-output /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll 2>&1 | /b/ml-opt-rel-x86-64-b1/build/bin/FileCheck /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll # RUN: at line 1
+ /b/ml-opt-rel-x86-64-b1/build/bin/opt -debug-pass-manager '-passes=fatlto-pre-link<O2>' -disable-output /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
+ /b/ml-opt-rel-x86-64-b1/build/bin/FileCheck /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
/b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll:6:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: FatLtoCleanup on [module]
              ^
<stdin>:36:1: note: 'next' match was here
Running pass: FatLtoCleanup on [module]
^
<stdin>:31:43: note: previous match ended here
Running pass: EmbedBitcodePass on [module]
                                          ^
<stdin>:32:1: note: non-matching line after previous match is here
Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module]
^

Input file: <stdin>
Check file: /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       31: Running pass: EmbedBitcodePass on [module] 
       32: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
       33: Invalidating analysis: CallGraphAnalysis on [module] 
       34: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       35: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::Module> on [module] 
       36: Running pass: FatLtoCleanup on [module] 
next:6     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       37: Running pass: LowerTypeTestsPass on [module] 
       38: Running pass: EliminateAvailableExternallyPass on [module] 
       39: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       40: Running analysis: LazyCallGraphAnalysis on [module] 
       41: Running analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
        .
        .
        .
>>>>>>

--

...

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 30, 2025

LLVM Buildbot has detected a new failure on builder ml-opt-devrel-x86-64 running on ml-opt-devrel-x86-64-b2 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/175/builds/21273

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: Other/fatlto.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
/b/ml-opt-devrel-x86-64-b1/build/bin/opt -debug-pass-manager -passes='fatlto-pre-link<O2>' -disable-output /b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll 2>&1 | /b/ml-opt-devrel-x86-64-b1/build/bin/FileCheck /b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll # RUN: at line 1
+ /b/ml-opt-devrel-x86-64-b1/build/bin/opt -debug-pass-manager '-passes=fatlto-pre-link<O2>' -disable-output /b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
+ /b/ml-opt-devrel-x86-64-b1/build/bin/FileCheck /b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
/b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll:6:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: FatLtoCleanup on [module]
              ^
<stdin>:36:1: note: 'next' match was here
Running pass: FatLtoCleanup on [module]
^
<stdin>:31:43: note: previous match ended here
Running pass: EmbedBitcodePass on [module]
                                          ^
<stdin>:32:1: note: non-matching line after previous match is here
Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module]
^

Input file: <stdin>
Check file: /b/ml-opt-devrel-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       31: Running pass: EmbedBitcodePass on [module] 
       32: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
       33: Invalidating analysis: CallGraphAnalysis on [module] 
       34: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       35: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::Module> on [module] 
       36: Running pass: FatLtoCleanup on [module] 
next:6     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       37: Running pass: LowerTypeTestsPass on [module] 
       38: Running pass: EliminateAvailableExternallyPass on [module] 
       39: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       40: Running analysis: LazyCallGraphAnalysis on [module] 
       41: Running analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
        .
        .
        .
>>>>>>

--

...

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 30, 2025

LLVM Buildbot has detected a new failure on builder ml-opt-dev-x86-64 running on ml-opt-dev-x86-64-b1 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/137/builds/21339

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: Other/fatlto.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
/b/ml-opt-dev-x86-64-b1/build/bin/opt -debug-pass-manager -passes='fatlto-pre-link<O2>' -disable-output /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll 2>&1 | /b/ml-opt-dev-x86-64-b1/build/bin/FileCheck /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll # RUN: at line 1
+ /b/ml-opt-dev-x86-64-b1/build/bin/FileCheck /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
+ /b/ml-opt-dev-x86-64-b1/build/bin/opt -debug-pass-manager '-passes=fatlto-pre-link<O2>' -disable-output /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll
/b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll:6:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: FatLtoCleanup on [module]
              ^
<stdin>:36:1: note: 'next' match was here
Running pass: FatLtoCleanup on [module]
^
<stdin>:31:43: note: previous match ended here
Running pass: EmbedBitcodePass on [module]
                                          ^
<stdin>:32:1: note: non-matching line after previous match is here
Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module]
^

Input file: <stdin>
Check file: /b/ml-opt-dev-x86-64-b1/llvm-project/llvm/test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       31: Running pass: EmbedBitcodePass on [module] 
       32: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
       33: Invalidating analysis: CallGraphAnalysis on [module] 
       34: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       35: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::Module> on [module] 
       36: Running pass: FatLtoCleanup on [module] 
next:6     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       37: Running pass: LowerTypeTestsPass on [module] 
       38: Running pass: EliminateAvailableExternallyPass on [module] 
       39: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       40: Running analysis: LazyCallGraphAnalysis on [module] 
       41: Running analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
        .
        .
        .
>>>>>>

--

...

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 30, 2025

LLVM Buildbot has detected a new failure on builder lld-x86_64-ubuntu-fast running on as-builder-4 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/33/builds/19263

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: Other/fatlto.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
/home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/build/bin/opt -debug-pass-manager -passes='fatlto-pre-link<O2>' -disable-output /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll 2>&1 | /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/build/bin/FileCheck /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll # RUN: at line 1
+ /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/build/bin/opt -debug-pass-manager '-passes=fatlto-pre-link<O2>' -disable-output /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll
+ /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/build/bin/FileCheck /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll
/home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll:6:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: FatLtoCleanup on [module]
              ^
<stdin>:36:1: note: 'next' match was here
Running pass: FatLtoCleanup on [module]
^
<stdin>:31:43: note: previous match ended here
Running pass: EmbedBitcodePass on [module]
                                          ^
<stdin>:32:1: note: non-matching line after previous match is here
Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module]
^

Input file: <stdin>
Check file: /home/buildbot/worker/as-builder-4/ramdisk/lld-x86_64/llvm-project/llvm/test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       31: Running pass: EmbedBitcodePass on [module] 
       32: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
       33: Invalidating analysis: CallGraphAnalysis on [module] 
       34: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       35: Invalidating analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::LazyCallGraph::SCC, llvm::LazyCallGraph&>, llvm::Module> on [module] 
       36: Running pass: FatLtoCleanup on [module] 
next:6     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       37: Running pass: LowerTypeTestsPass on [module] 
       38: Running pass: EliminateAvailableExternallyPass on [module] 
       39: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       40: Running analysis: LazyCallGraphAnalysis on [module] 
       41: Running analysis: InnerAnalysisManagerProxy<llvm::AnalysisManager<llvm::Function>, llvm::Module> on [module] 
        .
        .
        .
>>>>>>

--

...

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jun 30, 2025

LLVM Buildbot has detected a new failure on builder llvm-clang-x86_64-expensive-checks-debian running on gribozavr4 while building llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/16/builds/21783

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM :: Other/fatlto.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
/b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/opt -debug-pass-manager -passes='fatlto-pre-link<O2>' -disable-output /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll 2>&1 | /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/FileCheck /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll # RUN: at line 1
+ /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/FileCheck /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll
+ /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/opt -debug-pass-manager '-passes=fatlto-pre-link<O2>' -disable-output /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll
/b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll:6:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: FatLtoCleanup on [module]
              ^
<stdin>:39:1: note: 'next' match was here
Running pass: FatLtoCleanup on [module]
^
<stdin>:32:43: note: previous match ended here
Running pass: EmbedBitcodePass on [module]
                                          ^
<stdin>:33:1: note: non-matching line after previous match is here
Invalidating analysis: InnerAnalysisManagerProxy<llvm::FunctionAnalysisManager, llvm::Module> on [module]
^

Input file: <stdin>
Check file: /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       34: Invalidating analysis: PreservedModuleHashAnalysis on [module] 
       35: Invalidating analysis: CallGraphAnalysis on [module] 
       36: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       37: Invalidating analysis: InnerAnalysisManagerProxy<llvm::CGSCCAnalysisManager, llvm::Module> on [module] 
       38: Running analysis: InnerAnalysisManagerProxy<llvm::FunctionAnalysisManager, llvm::Module> on [module] 
       39: Running pass: FatLtoCleanup on [module] 
next:6     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       40: Running analysis: PreservedModuleHashAnalysis on [module] 
       41: Running pass: LowerTypeTestsPass on [module] 
       42: Running pass: EliminateAvailableExternallyPass on [module] 
       43: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       44: Running analysis: LazyCallGraphAnalysis on [module] 
        .
        .
        .
>>>>>>

--

...

@mikaelholmen
Copy link
Collaborator

With EXPENSIVE_CHECKS the fatlto.ll testcase fails because there is an unexpected
Running analysis: PreservedModuleHashAnalysis on [module].
We get

test/Other/fatlto.ll:7:15: error: CHECK-NEXT: is not on the line after the previous match
; CHECK-NEXT: Running pass: LowerTypeTestsPass on [module]
              ^
<stdin>:41:1: note: 'next' match was here
Running pass: LowerTypeTestsPass on [module]
^
<stdin>:39:40: note: previous match ended here
Running pass: FatLtoCleanup on [module]
                                       ^
<stdin>:40:1: note: non-matching line after previous match is here
Running analysis: PreservedModuleHashAnalysis on [module]
^

Input file: <stdin>
Check file: test/Other/fatlto.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
        .
        .
        .
       36: Invalidating analysis: LazyCallGraphAnalysis on [module] 
       37: Invalidating analysis: InnerAnalysisManagerProxy<CGSCCAnalysisManager, Module> on [module] 
       38: Running analysis: InnerAnalysisManagerProxy<FunctionAnalysisManager, Module> on [module] 
       39: Running pass: FatLtoCleanup on [module] 
       40: Running analysis: PreservedModuleHashAnalysis on [module] 
       41: Running pass: LowerTypeTestsPass on [module] 
next:7     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  error: match on wrong line
       42: Running pass: EliminateAvailableExternallyPass on [module] 
       43: Running pass: ReversePostOrderFunctionAttrsPass on [module] 
       44: Running analysis: LazyCallGraphAnalysis on [module] 
       45: Running pass: RecomputeGlobalsAAPass on [module] 
       46: Running analysis: CallGraphAnalysis on [module] 
        .
        .
        .
>>>>>>

I suppose a bit more relaxation is needed.

nikic added a commit that referenced this pull request Jul 1, 2025
To fix test failure with expensive checks reports at:
#146048 (comment)
@nikic
Copy link
Contributor Author

nikic commented Jul 1, 2025

@mikaelholmen Thanks, I believe 102c22c should fix it.

@mikaelholmen
Copy link
Collaborator

@mikaelholmen Thanks, I believe 102c22c should fix it.

Yep, thanks!

llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Jul 1, 2025
rlavaee pushed a commit to rlavaee/llvm-project that referenced this pull request Jul 1, 2025
…m#146048)

Expose the FatLTO pipeline via `-passes="fatlto-pre-link<Ox>"`, similar
to all the other optimization pipelines. This is to allow reproducing it
outside clang. (Possibly also useful for C API users.)
rlavaee pushed a commit to rlavaee/llvm-project that referenced this pull request Jul 1, 2025
rlavaee pushed a commit to rlavaee/llvm-project that referenced this pull request Jul 1, 2025
…m#146048)

Expose the FatLTO pipeline via `-passes="fatlto-pre-link<Ox>"`, similar
to all the other optimization pipelines. This is to allow reproducing it
outside clang. (Possibly also useful for C API users.)
rlavaee pushed a commit to rlavaee/llvm-project that referenced this pull request Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants