[clang-repl] Fix error recovery while PTU cleanup #127467

anutosh491 · 2025-02-17T10:08:01Z

What is seen

clang-repl> int x = 42;
clang-repl> auto capture = [&]() { return x * 2; };
In file included from <<< inputs >>>:1:
input_line_4:1:17: error: non-local lambda expression cannot have a capture-default
    1 | auto capture = [&]() { return x * 2; };
      |                 ^
zsh: segmentation fault  clang-repl --Xcc="-v"

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
  * frame #0: 0x0000000107b4f8b8 libclang-cpp.19.1.dylib`clang::IncrementalParser::CleanUpPTU(clang::PartialTranslationUnit&) + 988
    frame #1: 0x0000000107b4f1b4 libclang-cpp.19.1.dylib`clang::IncrementalParser::ParseOrWrapTopLevelDecl() + 416
    frame #2: 0x0000000107b4fb94 libclang-cpp.19.1.dylib`clang::IncrementalParser::Parse(llvm::StringRef) + 612
    frame #3: 0x0000000107b52fec libclang-cpp.19.1.dylib`clang::Interpreter::ParseAndExecute(llvm::StringRef, clang::Value*) + 180
    frame #4: 0x0000000100003498 clang-repl`main + 3560
    frame #5: 0x000000018d39a0e0 dyld`start + 2360

Though the error is justified, we shouldn't be interested in exiting through a segfault in such cases.

The issue is that empty named decls weren't being taken care of resulting into this assert

llvm-project/clang/include/clang/AST/DeclarationName.h

Line 503 in c1a2292

assert(getPtr() && "getFETokenInfo on an empty DeclarationName!");

Can also be seen when the example is attempted through xeus-cpp-lite.

anutosh491 · 2025-02-17T10:08:22Z

Should be fixed now

clang-repl> int x = 5; auto capture = [&]() { return x * 2; };
In file included from <<< inputs >>>:1:
input_line_1:1:28: error: non-local lambda expression cannot have a capture-default
    1 | int x = 5; auto capture = [&]() { return x * 2; };
      |                            ^
error: Parsing failed.

llvmbot · 2025-02-17T10:08:40Z

@llvm/pr-subscribers-clang

Author: Anutosh Bhat (anutosh491)

Changes

Fixes #123300

What is seen

clang-repl&gt; int x = 42;
clang-repl&gt; auto capture = [&amp;]() { return x * 2; };
In file included from &lt;&lt;&lt; inputs &gt;&gt;&gt;:1:
input_line_4:1:17: error: non-local lambda expression cannot have a capture-default
    1 | auto capture = [&amp;]() { return x * 2; };
      |                 ^
zsh: segmentation fault  clang-repl --Xcc="-v"

Though the error is justified, we shouldn't be interested in exiting through a segfault in such cases.

The issue is that empty named decls weren't being taken care of resulting into this assert

llvm-project/clang/include/clang/AST/DeclarationName.h

Line 503 in c1a2292

assert(getPtr() && "getFETokenInfo on an empty DeclarationName!");

Can also be seen when the example is attempted through xeus-cpp-lite.

Full diff: https://github.com/llvm/llvm-project/pull/127467.diff

2 Files Affected:

(modified) clang/lib/Interpreter/IncrementalParser.cpp (+2-2)
(modified) clang/unittests/Interpreter/InterpreterTest.cpp (+7)

diff --git a/clang/lib/Interpreter/IncrementalParser.cpp b/clang/lib/Interpreter/IncrementalParser.cpp
index e43cea1baf43a..1ebef0e434b3d 100644
--- a/clang/lib/Interpreter/IncrementalParser.cpp
+++ b/clang/lib/Interpreter/IncrementalParser.cpp
@@ -178,8 +178,8 @@ void IncrementalParser::CleanUpPTU(TranslationUnitDecl *MostRecentTU) {
     if (!ND)
       continue;
     // Check if we need to clean up the IdResolver chain.
-    if (ND->getDeclName().getFETokenInfo() && !D->getLangOpts().ObjC &&
-        !D->getLangOpts().CPlusPlus)
+    if (!ND->getDeclName().isEmpty() && ND->getDeclName().getFETokenInfo() &&
+        !D->getLangOpts().ObjC && !D->getLangOpts().CPlusPlus)
       S.IdResolver.RemoveDecl(ND);
   }
 }
diff --git a/clang/unittests/Interpreter/InterpreterTest.cpp b/clang/unittests/Interpreter/InterpreterTest.cpp
index 578f1d4c0eac6..56ab155ebf5a4 100644
--- a/clang/unittests/Interpreter/InterpreterTest.cpp
+++ b/clang/unittests/Interpreter/InterpreterTest.cpp
@@ -114,6 +114,13 @@ TEST_F(InterpreterTest, Errors) {
 
   RecoverErr = Interp->Parse("var1 = 424;");
   EXPECT_TRUE(!!RecoverErr);
+
+  Err = Interp->Parse("int x = 5; auto capture = [&]() { return x * 2; };").takeError();
+  EXPECT_THAT(DiagnosticOutput, HasSubstr("error: non-local lambda expression cannot have a capture-default"));
+  EXPECT_EQ("Parsing failed.", llvm::toString(std::move(Err)));
+
+  RecoverErr = Interp->Parse("int validVar = 10;");
+  EXPECT_TRUE(!!RecoverErr);
 }
 
 // Here we test whether the user can mix declarations and statements. The

anutosh491 · 2025-02-17T10:10:52Z

cc @ferdymercury @vgvassilev

github-actions · 2025-02-17T10:11:25Z

✅ With the latest revision this PR passed the C/C++ code formatter.

anutosh491 · 2025-02-17T10:13:56Z

Hmmm, the code formatter has a suggestion. Just increases the lines on a simple test. Not sure if the code formatter should be respected here !

ferdymercury · 2025-02-17T10:17:43Z

Thanks! fyi also @hahnjo

anutosh491 · 2025-02-17T10:23:00Z

For anyone interested in seeing the error at realtime, I am just leaving the static link for xeus-cpp-lite here.

https://compiler-research.org/xeus-cpp/lab/index.html

Possibly trying something like

int x = 5;
auto capture = [&]() { return x * 2; };

Should display the assertion error in the console.

anutosh491 · 2025-02-26T09:49:39Z

cc @vgvassilev

clang/lib/Interpreter/IncrementalParser.cpp

clang/unittests/Interpreter/InterpreterTest.cpp

clang/test/Interpreter/lambda.cpp

vgvassilev

LGTM!

anutosh491 · 2025-03-13T04:20:15Z

Thanks for the review. The test lamda.cpp passes (https://github.com/llvm/llvm-project/actions/runs/13810830928/job/38631851756?pr=127467#step:4:20415) and I think this should be ready now.

vgvassilev

Lgtm!

clang/test/Interpreter/lambda.cpp

vgvassilev

LGTM! Thank you.

anutosh491 · 2025-06-02T17:40:09Z

/cherry-pick 3b4c51b

llvmbot · 2025-06-02T17:46:20Z

/pull-request #142445

Fixes llvm#123300 What is seen ``` clang-repl> int x = 42; clang-repl> auto capture = [&]() { return x * 2; }; In file included from <<< inputs >>>:1: input_line_4:1:17: error: non-local lambda expression cannot have a capture-default 1 | auto capture = [&]() { return x * 2; }; | ^ zsh: segmentation fault clang-repl --Xcc="-v" (lldb) bt * thread llvm#1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) * frame #0: 0x0000000107b4f8b8 libclang-cpp.19.1.dylib`clang::IncrementalParser::CleanUpPTU(clang::PartialTranslationUnit&) + 988 frame llvm#1: 0x0000000107b4f1b4 libclang-cpp.19.1.dylib`clang::IncrementalParser::ParseOrWrapTopLevelDecl() + 416 frame llvm#2: 0x0000000107b4fb94 libclang-cpp.19.1.dylib`clang::IncrementalParser::Parse(llvm::StringRef) + 612 frame llvm#3: 0x0000000107b52fec libclang-cpp.19.1.dylib`clang::Interpreter::ParseAndExecute(llvm::StringRef, clang::Value*) + 180 frame llvm#4: 0x0000000100003498 clang-repl`main + 3560 frame llvm#5: 0x000000018d39a0e0 dyld`start + 2360 ``` Though the error is justified, we shouldn't be interested in exiting through a segfault in such cases. The issue is that empty named decls weren't being taken care of resulting into this assert https://github.com/llvm/llvm-project/blob/c1a229252617ed58f943bf3f4698bd8204ee0f04/clang/include/clang/AST/DeclarationName.h#L503 Can also be seen when the example is attempted through xeus-cpp-lite. ![image](https://github.com/user-attachments/assets/9b0e6ead-138e-4b06-9ad9-fcb9f8d5bf6e)

DavidSpickett · 2025-06-10T12:00:25Z

After this change the test labda.cpp is failing on Linaro's Windows on Arm clang bots. They were stuck for other reasons and only just got back up to date, so here's a new build showing the failure:
https://lab.llvm.org/buildbot/#/builders/161/builds/6473

cat C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp | c:\users\tcwg\llvm-worker\clang-arm64-windows-msvc\build\bin\clang-repl.exe -Xcc -Xclang -Xcc -verify -Xcc -O2 | c:\users\tcwg\llvm-worker\clang-arm64-windows-msvc\build\bin\filecheck.exe C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp
# executed command: cat 'C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp'
# executed command: 'c:\users\tcwg\llvm-worker\clang-arm64-windows-msvc\build\bin\clang-repl.exe' -Xcc -Xclang -Xcc -verify -Xcc -O2
# .---command stderr------------
# | error: Parsing failed.
# `-----------------------------
# executed command: 'c:\users\tcwg\llvm-worker\clang-arm64-windows-msvc\build\bin\filecheck.exe' 'C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp'
# .---command stderr------------
# | C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp:26:11: error: CHECK: expected string not found in input
# | // CHECK: x = 42
# |           ^
# | <stdin>:4:28: note: scanning from here
# | clang-repl> clang-repl> TWO
# |                            ^
# | <stdin>:5:9: note: possible intended match here
# | clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> 
# |         ^
# | 
# | Input file: <stdin>
# | Check file: C:\Users\tcwg\llvm-worker\clang-arm64-windows-msvc\llvm-project\clang\test\Interpreter\lambda.cpp
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             1: x = 42 
# |             2: clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> ONE 
# |             3: clang-repl> clang-repl> TWO 
# |             4: clang-repl> clang-repl> TWO 
# | check:26'0                                X error: no match found
# |             5: clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl>  
# | check:26'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:26'1             ?                                                                                                                                                     possible intended match
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1
--

I've confirmed locally that the test passed before this change.

The first RUN line is ok:

C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build>cat C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\test\Interpreter\lambda.cpp | .\bin\clang-repl.exe
clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> ONE
clang-repl> clang-repl> TWO
clang-repl> clang-repl> TWO
clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> In file included from <<< inputs >>>:1:
input_line_21:1:17: error: non-local lambda expression cannot have a capture-default
    1 | auto capture = [&]() { return x * 2; };
      |                 ^
error: Parsing failed.
clang-repl> clang-repl> clang-repl> x = 42
clang-repl> clang-repl> clang-repl>

The second one fails to parse the lambda:

C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build>cat C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\test\Interpreter\lambda.cpp | .\bin\clang-repl.exe -Xcc -Xclang -Xcc -verify -Xcc -O2
clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> ONE
clang-repl> clang-repl> TWO
clang-repl> clang-repl> TWO
clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> clang-repl> error: Parsing failed.
clang-repl> clang-repl> clang-repl> x = 42
clang-repl> clang-repl> clang-repl>

(also the repeated clang-repl> is strange but maybe that is expected?)

How can I debug this further, is there a way to enable some verbose parsing / verification? (seems more likely that verification is actually the failing part)

DavidSpickett · 2025-06-10T12:09:27Z

Also failing on Windows x64, but this time the error is better:
https://lab.llvm.org/buildbot/#/builders/63/builds/7023

And now I see the actual problem, all I had to do was scroll up :)

# .---command stderr------------
# | In file included from <<< inputs >>>:1:
# | input_line_22:1:17: error: non-local lambda expression cannot have a capture-default
# |     1 | auto capture = [&]() { return x * 2; };
# |       |                 ^
# | error: Parsing failed.
# `-----------------------------

Puzzling that we don't see this on other platforms though.

But this is the error message you are expecting. So it is doing what you expect but the test isn't finding the output.

anutosh491 · 2025-06-10T12:24:13Z

Hi,

On phone right now so shall have a better look at the error in sometime but just for some clarification, the parsing error is expected (we had a seg fault on master for this whereas a parsing error is expected and that's what the Pr is trying to achieve)

Edit: We weren't notified by any failures by the bot or the CI so it's the first time I'm seeing this. Thanks for the report.

DavidSpickett · 2025-06-10T12:27:24Z

Thanks for confirming, yes, your patch is working as expected but something about Windows' output is tripping up FileCheck I think.

Linux does not print the clang-repl> (maybe because it can tell it's outputting to a non-interactive pipe):

$ cat ../llvm-project/clang/test/Interpreter/lambda.cpp | ./bin/clang-repl -Xcc -Xclang -Xcc -verify -Xcc -O2
ONE
TWO
TWO
error: Parsing failed.
x = 42

Which should not be a problem in theory.

I'll figure it out.

#143547 for details. For some reason the order of the printf output is not as expected when piped. None of the "fixes" I have are good, so only run this RUN on non-Windows. Test added by #127467.

DavidSpickett · 2025-06-10T15:15:45Z

I've disabled the failing RUN line on Windows and opened #143547 with details of what's happening.

Let me know if anything stands out to you. Also would be useful if I can see the code produced at the different optimisation levels.

llvm/llvm-project#143547 for details. For some reason the order of the printf output is not as expected when piped. None of the "fixes" I have are good, so only run this RUN on non-Windows. Test added by llvm/llvm-project#127467.

llvm#143547 for details. For some reason the order of the printf output is not as expected when piped. None of the "fixes" I have are good, so only run this RUN on non-Windows. Test added by llvm#127467.

Fixes llvm#123300 What is seen ``` clang-repl> int x = 42; clang-repl> auto capture = [&]() { return x * 2; }; In file included from <<< inputs >>>:1: input_line_4:1:17: error: non-local lambda expression cannot have a capture-default 1 | auto capture = [&]() { return x * 2; }; | ^ zsh: segmentation fault clang-repl --Xcc="-v" (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) * frame #0: 0x0000000107b4f8b8 libclang-cpp.19.1.dylib`clang::IncrementalParser::CleanUpPTU(clang::PartialTranslationUnit&) + 988 frame #1: 0x0000000107b4f1b4 libclang-cpp.19.1.dylib`clang::IncrementalParser::ParseOrWrapTopLevelDecl() + 416 frame #2: 0x0000000107b4fb94 libclang-cpp.19.1.dylib`clang::IncrementalParser::Parse(llvm::StringRef) + 612 frame #3: 0x0000000107b52fec libclang-cpp.19.1.dylib`clang::Interpreter::ParseAndExecute(llvm::StringRef, clang::Value*) + 180 frame #4: 0x0000000100003498 clang-repl`main + 3560 frame #5: 0x000000018d39a0e0 dyld`start + 2360 ``` Though the error is justified, we shouldn't be interested in exiting through a segfault in such cases. The issue is that empty named decls weren't being taken care of resulting into this assert https://github.com/llvm/llvm-project/blob/c1a229252617ed58f943bf3f4698bd8204ee0f04/clang/include/clang/AST/DeclarationName.h#L503 Can also be seen when the example is attempted through xeus-cpp-lite. ![image](https://github.com/user-attachments/assets/9b0e6ead-138e-4b06-9ad9-fcb9f8d5bf6e)

This cherry-picks 3b4c51b and beffd15 to 20.x. Which are: [clang-repl] Fix error recovery while PTU cleanup (llvm#127467) [clang][Interpreter] Disable part of lambda test on Windows The latter commit avoids a test failure seen in Windows builds. On main, I turned off one of the RUN lines for Windows, but reviewers on the cherry-pick preferred UNSUPPORTED to disable the whole test. So I have used UNSUPPORTED in this version for 20.x.

llvm#143547 for details. For some reason the order of the printf output is not as expected when piped. None of the "fixes" I have are good, so only run this RUN on non-Windows. Test added by llvm#127467.

llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Feb 17, 2025

hahnjo requested a review from vgvassilev February 17, 2025 10:30

anutosh491 changed the title ~~Fix error recovery while PTU cleanup~~ [clang-repl] Fix error recovery while PTU cleanup Feb 18, 2025

vgvassilev reviewed Feb 26, 2025

View reviewed changes

clang/lib/Interpreter/IncrementalParser.cpp Outdated Show resolved Hide resolved

vgvassilev reviewed Feb 26, 2025

View reviewed changes

clang/unittests/Interpreter/InterpreterTest.cpp Outdated Show resolved Hide resolved

Fix error recovery while PTU cleanup

bd1b0b2

anutosh491 force-pushed the captures branch from 387103d to bd1b0b2 Compare March 11, 2025 06:24

vgvassilev reviewed Mar 11, 2025

View reviewed changes

clang/test/Interpreter/lambda.cpp Outdated Show resolved Hide resolved

Fix test as per review

1963cc3

vgvassilev approved these changes Mar 12, 2025

View reviewed changes

fix failing lambda.cpp

45924cd

vgvassilev approved these changes Apr 1, 2025

View reviewed changes

vgvassilev reviewed Apr 1, 2025

View reviewed changes

clang/test/Interpreter/lambda.cpp Show resolved Hide resolved

Merge branch 'main' into captures

9749583

anutosh491 requested a review from vgvassilev May 13, 2025 09:07

anutosh491 added 2 commits June 2, 2025 16:55

Merge branch 'main' into captures

8663875

print the value for x after the parsing failed error on capture

056555c

vgvassilev approved these changes Jun 2, 2025

View reviewed changes

anutosh491 merged commit 3b4c51b into llvm:main Jun 2, 2025
11 checks passed

anutosh491 deleted the captures branch June 2, 2025 14:47

anutosh491 added this to the LLVM 20.X Release milestone Jun 2, 2025

github-project-automation bot added this to LLVM Release Status Jun 2, 2025

github-project-automation bot moved this to Needs Triage in LLVM Release Status Jun 2, 2025

llvmbot moved this from Needs Triage to Done in LLVM Release Status Jun 2, 2025

DavidSpickett mentioned this pull request Jun 10, 2025

clang-repl output differences on Windows when piped into something else #143547

Open

DavidSpickett mentioned this pull request Jun 12, 2025

[clan-reply] Backport PTU error recovery to 20.x #143893

Merged

[clang-repl] Fix error recovery while PTU cleanup #127467

[clang-repl] Fix error recovery while PTU cleanup #127467

Uh oh!

Conversation

anutosh491 commented Feb 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anutosh491 commented Feb 17, 2025

Uh oh!

llvmbot commented Feb 17, 2025

Uh oh!

anutosh491 commented Feb 17, 2025

Uh oh!

github-actions bot commented Feb 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anutosh491 commented Feb 17, 2025

Uh oh!

ferdymercury commented Feb 17, 2025

Uh oh!

anutosh491 commented Feb 17, 2025

Uh oh!

anutosh491 commented Feb 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vgvassilev left a comment

Choose a reason for hiding this comment

Uh oh!

anutosh491 commented Mar 13, 2025

Uh oh!

vgvassilev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vgvassilev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anutosh491 commented Jun 2, 2025

Uh oh!

llvmbot commented Jun 2, 2025

Uh oh!

DavidSpickett commented Jun 10, 2025

Uh oh!

DavidSpickett commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anutosh491 commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidSpickett commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidSpickett commented Jun 10, 2025

Uh oh!

Uh oh!

anutosh491 commented Feb 17, 2025 •

edited

Loading

github-actions bot commented Feb 17, 2025 •

edited

Loading

DavidSpickett commented Jun 10, 2025 •

edited

Loading

anutosh491 commented Jun 10, 2025 •

edited

Loading

DavidSpickett commented Jun 10, 2025 •

edited

Loading