[clang][deps] Respect Lexer::cutOffLexing()

[jansvoboda11]

This is crucial when recovering from fatal loader errors. Without it, the Lexer keeps yielding more tokens and the compiler may access invalid ASTReader state.

rdar://133388373

[llvmbot]

@llvm/pr-subscribers-clang

Author: Jan Svoboda (jansvoboda11)

Changes

This is crucial when recovering from fatal loader errors. Without it, the Lexer keeps yielding more tokens and the compiler may access invalid ASTReader state.

rdar://133388373

---

Full diff: https://github.com/llvm/llvm-project/pull/134404.diff

2 Files Affected:

• (modified) clang/lib/Lex/Lexer.cpp (+3)
• (added) clang/test/ClangScanDeps/modules-relocated-mm-macro.c (+42)

diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index 3128627490e28..93200458f04b4 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -4543,6 +4543,9 @@ bool Lexer::LexDependencyDirectiveToken(Token &Result) {
 
   using namespace dependency_directives_scan;
 
+  if (BufferPtr == BufferEnd)
+    return LexEndOfFile(Result, BufferPtr);
+
   while (NextDepDirectiveTokenIndex == DepDirectives.front().Tokens.size()) {
     if (DepDirectives.front().Kind == pp_eof)
       return LexEndOfFile(Result, BufferEnd);
diff --git a/clang/test/ClangScanDeps/modules-relocated-mm-macro.c b/clang/test/ClangScanDeps/modules-relocated-mm-macro.c
new file mode 100644
index 0000000000000..17f479d9e0046
--- /dev/null
+++ b/clang/test/ClangScanDeps/modules-relocated-mm-macro.c
@@ -0,0 +1,42 @@
+// This test checks that we don't crash when we load two conflicting PCM files
+// and instead emit the appropriate diagnostics.
+
+// RUN: rm -rf %t
+// RUN: split-file %s %t
+
+// RUN: mkdir %t/frameworks1
+
+// RUN: clang-scan-deps -format experimental-full -o %t/deps1.json -- \
+// RUN:   %clang -fmodules -fmodules-cache-path=%t/cache \
+// RUN:   -F %t/frameworks1 -F %t/frameworks2 \
+// RUN:   -c %t/tu1.m -o %t/tu1.o
+
+// RUN: cp -r %t/frameworks2/A.framework %t/frameworks1
+
+// RUN: not clang-scan-deps -format experimental-full -o %t/deps2.json -- \
+// RUN:   %clang -fmodules -fmodules-cache-path=%t/cache \
+// RUN:   -F %t/frameworks1 -F %t/frameworks2 \
+// RUN:   -c %t/tu2.m -o %t/tu2.o \
+// RUN:     2>&1 | FileCheck %s
+
+// CHECK: fatal error: module 'A' is defined in both '{{.*}}.pcm' and '{{.*}}.pcm'
+
+//--- frameworks2/A.framework/Modules/module.modulemap
+framework module A { header "A.h" }
+//--- frameworks2/A.framework/Headers/A.h
+#define MACRO_A 1
+
+//--- frameworks2/B.framework/Modules/module.modulemap
+framework module B { header "B.h" }
+//--- frameworks2/B.framework/Headers/B.h
+#include <A/A.h>
+
+//--- tu1.m
+#include <B/B.h>
+
+//--- tu2.m
+#include <A/A.h>
+#include <B/B.h> // This results in a conflict and a fatal loader error.
+
+#if MACRO_A // This crashes with lexer that does not respect `cutOfLexing()`.
+#endif