[flang][preprocessor] Handle initial "MACRO&" with no space

[klausler]

The prescanner checks lines that begin with a keyword macro name to see whether they should be treated as a comment or compiler directive instead of a source line. This fails when the potential keyword macro name is extended with identifier characters via Fortran line continuation. Disable line continuation during this check.

[llvmbot]

@llvm/pr-subscribers-flang-parser

Author: Peter Klausler (klausler)

Changes

The prescanner checks lines that begin with a keyword macro name to see whether they should be treated as a comment or compiler directive instead of a source line. This fails when the potential keyword macro name is extended with identifier characters via Fortran line continuation. Disable line continuation during this check.

---

Full diff: https://github.com/llvm/llvm-project/pull/98684.diff

2 Files Affected:

• (modified) flang/lib/Parser/prescan.cpp (+12-4)
• (modified) flang/test/Preprocessing/directive-contin-with-pp.F90 (+14-3)

diff --git a/flang/lib/Parser/prescan.cpp b/flang/lib/Parser/prescan.cpp
index ca6a823999244..aa45548408aa9 100644
--- a/flang/lib/Parser/prescan.cpp
+++ b/flang/lib/Parser/prescan.cpp
@@ -189,11 +189,19 @@ void Prescanner::Statement() {
       // a comment marker or directive sentinel.  If so, disable line
       // continuation, so that NextToken() won't consume anything from
       // following lines.
-      if (IsLegalIdentifierStart(*at_) && NextToken(tokens) &&
-          tokens.SizeInTokens() > 0) {
-        if (CharBlock id{tokens.TokenAt(0)}; preprocessor_.IsNameDefined(id) &&
+      if (IsLegalIdentifierStart(*at_)) {
+        // TODO: Only bother with these cases when any keyword macro has
+        // been defined with replacement text that could begin a comment
+        // or directive sentinel.
+        const char *p{at_};
+        while (IsLegalInIdentifier(*++p)) {
+        }
+        CharBlock id{at_, static_cast<std::size_t>(p - at_)};
+        if (preprocessor_.IsNameDefined(id) &&
             !preprocessor_.IsFunctionLikeDefinition(id)) {
-          if (auto replaced{preprocessor_.MacroReplacement(tokens, *this)}) {
+          TokenSequence toks;
+          toks.Put(id, GetProvenance(at_));
+          if (auto replaced{preprocessor_.MacroReplacement(toks, *this)}) {
             auto newLineClass{ClassifyLine(*replaced, GetCurrentProvenance())};
             disableSourceContinuation_ =
                 newLineClass.kind != LineClassification::Kind::Source;
diff --git a/flang/test/Preprocessing/directive-contin-with-pp.F90 b/flang/test/Preprocessing/directive-contin-with-pp.F90
index 64f1dc43f72b4..544c6619f6b53 100644
--- a/flang/test/Preprocessing/directive-contin-with-pp.F90
+++ b/flang/test/Preprocessing/directive-contin-with-pp.F90
@@ -11,7 +11,7 @@
 
 module m
  contains
-  subroutine s(x1, x2, x3, x4, x5, x6, x7)
+  subroutine s1(x1, x2, x3, x4, x5, x6, x7)
 
 !dir$ ignore_tkr x1
 
@@ -51,11 +51,18 @@ subroutine s(x1, x2, x3, x4, x5, x6, x7)
     do j3 = 1, n
     end do
   end
-end
+
+COMMENT &
+  subroutine s2
+  end subroutine
+COMMENT&
+  subroutine s3
+  end subroutine
+end module
 
 !CHECK: MODULE m
 !CHECK: CONTAINS
-!CHECK:  SUBROUTINE s (x1, x2, x3, x4, x5, x6, x7)
+!CHECK:  SUBROUTINE s1 (x1, x2, x3, x4, x5, x6, x7)
 !CHECK:   !DIR$ IGNORE_TKR x1
 !CHECK:   !DIR$ IGNORE_TKR x2
 !CHECK:   !DIR$ IGNORE_TKR x3
@@ -73,4 +80,8 @@ subroutine s(x1, x2, x3, x4, x5, x6, x7)
 !CHECK:   DO j3=1_4,n
 !CHECK:   END DO
 !CHECK:  END SUBROUTINE
+!CHECK:  SUBROUTINE s2
+!CHECK:  END SUBROUTINE
+!CHECK:  SUBROUTINE s3
+!CHECK:  END SUBROUTINE
 !CHECK: END MODULE

[DavidTruby]

LGTM and fixes the issue in quantum-espresso. Thanks!