[flang] Move parser invocations into ParserActions #130309

mrkajetanp · 2025-03-07T17:27:43Z

FrontendActions.cpp is currently one of the biggest compilation units in all of flang. Measuring its compilation gives the following metrics:

User time (seconds): 139.21
System time (seconds): 4.65
Maximum resident set size (kbytes): 5891440 (5.61 GB)

This commit separates out explicit invocations of the parser into a separate compilation unit - ParserActions.cpp - through helper functions in order to decrease the maximum compilation time and memory usage of a single unit.
After the split, the measurements of FrontendActions.cpp are as follows:

User time (seconds): 70.08
System time (seconds): 3.16
Maximum resident set size (kbytes): 3961492 (3.7 GB)

While the ones for the newly created ParserActions.cpp as follows:

User time (seconds): 104.33
System time (seconds): 3.37
Maximum resident set size (kbytes): 4185600 (3.99 GB)

FrontendActions.cpp is currently one of the biggest compilation units in all of flang. Measuring its compilation gives the following metrics: User time (seconds): 139.21 System time (seconds): 4.65 Maximum resident set size (kbytes): 5891440 (5.61 GB) This commit separates out explicit invocations of the parser into a separate compilation unit - ParserActions.cpp - through helper functions in order to decrease the maximum compilation time and memory usage of a single unit. After the split, the measurements of FrontendActions.cpp are as follows: User time (seconds): 70.08 System time (seconds): 3.16 Maximum resident set size (kbytes): 3961492 (3.7 GB) While the ones for the newly created ParserActions.cpp as follows: User time (seconds): 104.33 System time (seconds): 3.37 Maximum resident set size (kbytes): 4185600 (3.99 GB) Signed-off-by: Kajetan Puchalski <[email protected]>

github-actions · 2025-03-07T17:28:00Z

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

llvmbot · 2025-03-07T17:28:34Z

@llvm/pr-subscribers-flang-driver

Author: Kajetan Puchalski (mrkajetanp)

Changes

FrontendActions.cpp is currently one of the biggest compilation units in all of flang. Measuring its compilation gives the following metrics:

User time (seconds): 139.21
System time (seconds): 4.65
Maximum resident set size (kbytes): 5891440 (5.61 GB)

This commit separates out explicit invocations of the parser into a separate compilation unit - ParserActions.cpp - through helper functions in order to decrease the maximum compilation time and memory usage of a single unit.
After the split, the measurements of FrontendActions.cpp are as follows:

User time (seconds): 70.08
System time (seconds): 3.16
Maximum resident set size (kbytes): 3961492 (3.7 GB)

While the ones for the newly created ParserActions.cpp as follows:

User time (seconds): 104.33
System time (seconds): 3.37
Maximum resident set size (kbytes): 4185600 (3.99 GB)

Patch is 24.08 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/130309.diff

10 Files Affected:

(modified) flang/include/flang/Frontend/CompilerInstance.h (+1-2)
(modified) flang/include/flang/Frontend/CompilerInvocation.h (+2-2)
(modified) flang/include/flang/Frontend/FrontendActions.h (+1-18)
(added) flang/include/flang/Frontend/ParserActions.h (+65)
(added) flang/include/flang/Parser/options.h (+43)
(modified) flang/include/flang/Parser/parsing.h (+1-24)
(modified) flang/lib/Frontend/CMakeLists.txt (+1)
(modified) flang/lib/Frontend/FrontendAction.cpp (+1)
(modified) flang/lib/Frontend/FrontendActions.cpp (+16-131)
(added) flang/lib/Frontend/ParserActions.cpp (+152)

diff --git a/flang/include/flang/Frontend/CompilerInstance.h b/flang/include/flang/Frontend/CompilerInstance.h
index 509c9f4b9e91a..d34c128d01794 100644
--- a/flang/include/flang/Frontend/CompilerInstance.h
+++ b/flang/include/flang/Frontend/CompilerInstance.h
@@ -16,8 +16,7 @@
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/FrontendAction.h"
 #include "flang/Frontend/PreprocessorOptions.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Parser/provenance.h"
+#include "flang/Frontend/ParserActions.h"
 #include "flang/Semantics/runtime-type-info.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Support/StringOstream.h"
diff --git a/flang/include/flang/Frontend/CompilerInvocation.h b/flang/include/flang/Frontend/CompilerInvocation.h
index 9e6724be33033..d6ee1511cdb4b 100644
--- a/flang/include/flang/Frontend/CompilerInvocation.h
+++ b/flang/include/flang/Frontend/CompilerInvocation.h
@@ -18,7 +18,7 @@
 #include "flang/Frontend/PreprocessorOptions.h"
 #include "flang/Frontend/TargetOptions.h"
 #include "flang/Lower/LoweringOptions.h"
-#include "flang/Parser/parsing.h"
+#include "flang/Parser/options.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Support/LangOptions.h"
 #include "mlir/Support/Timing.h"
@@ -29,7 +29,7 @@
 
 namespace llvm {
 class TargetMachine;
-}
+} // namespace llvm
 
 namespace Fortran::frontend {
 
diff --git a/flang/include/flang/Frontend/FrontendActions.h b/flang/include/flang/Frontend/FrontendActions.h
index 4e3d3cb2657db..f9a45bd6c0a56 100644
--- a/flang/include/flang/Frontend/FrontendActions.h
+++ b/flang/include/flang/Frontend/FrontendActions.h
@@ -13,10 +13,8 @@
 #ifndef FORTRAN_FRONTEND_FRONTENDACTIONS_H
 #define FORTRAN_FRONTEND_FRONTENDACTIONS_H
 
-#include "flang/Frontend/CodeGenOptions.h"
 #include "flang/Frontend/FrontendAction.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Semantics/semantics.h"
+#include "flang/Frontend/ParserActions.h"
 
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/OwningOpRef.h"
@@ -26,21 +24,6 @@
 
 namespace Fortran::frontend {
 
-// TODO: This is a copy from f18.cpp. It doesn't really belong here and should
-// be moved to a more suitable place in future.
-struct MeasurementVisitor {
-  template <typename A>
-  bool Pre(const A &) {
-    return true;
-  }
-  template <typename A>
-  void Post(const A &) {
-    ++objects;
-    bytes += sizeof(A);
-  }
-  size_t objects{0}, bytes{0};
-};
-
 //===----------------------------------------------------------------------===//
 // Custom Consumer Actions
 //===----------------------------------------------------------------------===//
diff --git a/flang/include/flang/Frontend/ParserActions.h b/flang/include/flang/Frontend/ParserActions.h
new file mode 100644
index 0000000000000..9c76f2bc9b1ed
--- /dev/null
+++ b/flang/include/flang/Frontend/ParserActions.h
@@ -0,0 +1,65 @@
+//===- ParserActions.h -------------------------------------------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_PARSER_ACTIONS_H_
+#define FORTRAN_PARSER_ACTIONS_H_
+
+#include <string>
+
+namespace llvm {
+    class raw_string_ostream;
+    class raw_ostream;
+    class StringRef;
+} // namespace llvm
+
+namespace Fortran::lower {
+    class LoweringBridge;
+} // namespace Fortran::lower
+
+namespace Fortran::parser {
+    class Parsing;
+    class AllCookedSources;
+} // namespace Fortran::parser
+
+namespace lower::pft {
+    class Program;
+} // namespace lower::pft
+
+//=== Frontend Parser helpers ===
+
+namespace Fortran::frontend {
+class CompilerInstance;
+
+Fortran::parser::AllCookedSources& getAllCooked(CompilerInstance &ci);
+
+void parseAndLowerTree(CompilerInstance &ci, Fortran::lower::LoweringBridge &lb);
+
+void dumpTree(CompilerInstance &ci);
+
+void dumpProvenance(CompilerInstance &ci);
+
+void dumpPreFIRTree(CompilerInstance &ci);
+
+void formatOrDumpPrescanner(std::string &buf, llvm::raw_string_ostream &outForPP, CompilerInstance &ci);
+
+void debugMeasureParseTree(CompilerInstance &ci, llvm::StringRef filename);
+
+void debugUnparseNoSema(CompilerInstance &ci, llvm::raw_ostream &out);
+
+void debugUnparseWithSymbols(CompilerInstance &ci);
+
+void debugUnparseWithModules(CompilerInstance &ci);
+
+void debugDumpParsingLog(CompilerInstance &ci);
+} // namespace Fortran::frontend
+
+#endif // FORTRAN_PARSER_ACTIONS_H_
diff --git a/flang/include/flang/Parser/options.h b/flang/include/flang/Parser/options.h
new file mode 100644
index 0000000000000..a9f93221ffdf1
--- /dev/null
+++ b/flang/include/flang/Parser/options.h
@@ -0,0 +1,43 @@
+//===-- include/flang/Parser/options.h --------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_PARSER_OPTIONS_H_
+#define FORTRAN_PARSER_OPTIONS_H_
+
+#include <optional>
+#include <string>
+#include <vector>
+
+#include "characters.h"
+#include "flang/Support/Fortran-features.h"
+
+namespace Fortran::parser {
+
+struct Options {
+  Options() {}
+
+  using Predefinition = std::pair<std::string, std::optional<std::string>>;
+
+  bool isFixedForm{false};
+  int fixedFormColumns{72};
+  common::LanguageFeatureControl features;
+  std::vector<std::string> searchDirectories;
+  std::vector<std::string> intrinsicModuleDirectories;
+  std::vector<Predefinition> predefinitions;
+  bool instrumentedParse{false};
+  bool isModuleFile{false};
+  bool needProvenanceRangeToCharBlockMappings{false};
+  Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
+  bool prescanAndReformat{false}; // -E
+  bool expandIncludeLinesInPreprocessedOutput{true};
+  bool showColors{false};
+};
+
+} // namespace Fortran::parser
+
+#endif // FORTRAN_PARSER_OPTIONS_H_
diff --git a/flang/include/flang/Parser/parsing.h b/flang/include/flang/Parser/parsing.h
index 116b6bd6f191f..3b4f5f5145a58 100644
--- a/flang/include/flang/Parser/parsing.h
+++ b/flang/include/flang/Parser/parsing.h
@@ -9,41 +9,18 @@
 #ifndef FORTRAN_PARSER_PARSING_H_
 #define FORTRAN_PARSER_PARSING_H_
 
-#include "characters.h"
 #include "instrumented-parser.h"
 #include "message.h"
 #include "parse-tree.h"
 #include "provenance.h"
+#include "options.h"
 #include "flang/Parser/preprocessor.h"
-#include "flang/Support/Fortran-features.h"
 #include "llvm/Support/raw_ostream.h"
 #include <optional>
 #include <string>
-#include <utility>
-#include <vector>
 
 namespace Fortran::parser {
 
-struct Options {
-  Options() {}
-
-  using Predefinition = std::pair<std::string, std::optional<std::string>>;
-
-  bool isFixedForm{false};
-  int fixedFormColumns{72};
-  common::LanguageFeatureControl features;
-  std::vector<std::string> searchDirectories;
-  std::vector<std::string> intrinsicModuleDirectories;
-  std::vector<Predefinition> predefinitions;
-  bool instrumentedParse{false};
-  bool isModuleFile{false};
-  bool needProvenanceRangeToCharBlockMappings{false};
-  Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
-  bool prescanAndReformat{false}; // -E
-  bool expandIncludeLinesInPreprocessedOutput{true};
-  bool showColors{false};
-};
-
 class Parsing {
 public:
   explicit Parsing(AllCookedSources &);
diff --git a/flang/lib/Frontend/CMakeLists.txt b/flang/lib/Frontend/CMakeLists.txt
index 80d63fca6fb76..c80373799b015 100644
--- a/flang/lib/Frontend/CMakeLists.txt
+++ b/flang/lib/Frontend/CMakeLists.txt
@@ -5,6 +5,7 @@ add_flang_library(flangFrontend
   CompilerInstance.cpp
   CompilerInvocation.cpp
   CodeGenOptions.cpp
+  ParserActions.cpp
   FrontendAction.cpp
   FrontendActions.cpp
   FrontendOptions.cpp
diff --git a/flang/lib/Frontend/FrontendAction.cpp b/flang/lib/Frontend/FrontendAction.cpp
index 9a555bc7cd23b..5813f776efad4 100644
--- a/flang/lib/Frontend/FrontendAction.cpp
+++ b/flang/lib/Frontend/FrontendAction.cpp
@@ -17,6 +17,7 @@
 #include "flang/Frontend/FrontendPluginRegistry.h"
 #include "clang/Basic/DiagnosticFrontend.h"
 #include "llvm/Support/Errc.h"
+#include "flang/Parser/parsing.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 using namespace Fortran::frontend;
diff --git a/flang/lib/Frontend/FrontendActions.cpp b/flang/lib/Frontend/FrontendActions.cpp
index 76d329d043731..69d7cf6cc9e7a 100644
--- a/flang/lib/Frontend/FrontendActions.cpp
+++ b/flang/lib/Frontend/FrontendActions.cpp
@@ -14,9 +14,8 @@
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/FrontendOptions.h"
-#include "flang/Frontend/PreprocessorOptions.h"
+#include "flang/Frontend/ParserActions.h"
 #include "flang/Lower/Bridge.h"
-#include "flang/Lower/PFTBuilder.h"
 #include "flang/Lower/Support/Verifier.h"
 #include "flang/Optimizer/Dialect/Support/FIRContext.h"
 #include "flang/Optimizer/Dialect/Support/KindMapping.h"
@@ -25,13 +24,7 @@
 #include "flang/Optimizer/Support/InitFIR.h"
 #include "flang/Optimizer/Support/Utils.h"
 #include "flang/Optimizer/Transforms/Passes.h"
-#include "flang/Parser/dump-parse-tree.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Parser/provenance.h"
-#include "flang/Parser/source.h"
-#include "flang/Parser/unparse.h"
 #include "flang/Semantics/runtime-type-info.h"
-#include "flang/Semantics/semantics.h"
 #include "flang/Semantics/unparse-with-symbols.h"
 #include "flang/Support/default-kinds.h"
 #include "flang/Tools/CrossToolHelpers.h"
@@ -318,7 +311,7 @@ bool CodeGenAction::beginSourceFileAction() {
       *mlirCtx, ci.getSemanticsContext(), defKinds,
       ci.getSemanticsContext().intrinsics(),
       ci.getSemanticsContext().targetCharacteristics(),
-      ci.getParsing().allCooked(), ci.getInvocation().getTargetOpts().triple,
+      Fortran::frontend::getAllCooked(ci), ci.getInvocation().getTargetOpts().triple,
       kindMap, ci.getInvocation().getLoweringOpts(),
       ci.getInvocation().getFrontendOpts().envDefaults,
       ci.getInvocation().getFrontendOpts().features, targetMachine,
@@ -333,8 +326,7 @@ bool CodeGenAction::beginSourceFileAction() {
   }
 
   // Create a parse tree and lower it to FIR
-  Fortran::parser::Program &parseTree{*ci.getParsing().parseTree()};
-  lb.lower(parseTree, ci.getSemanticsContext());
+  Fortran::frontend::parseAndLowerTree(ci, lb);
 
   // Fetch module from lb, so we can set
   mlirModule = lb.getModuleAndRelease();
@@ -431,19 +423,8 @@ void PrintPreprocessedAction::executeAction() {
   std::string buf;
   llvm::raw_string_ostream outForPP{buf};
 
-  // Format or dump the prescanner's output
   CompilerInstance &ci = this->getInstance();
-  if (ci.getInvocation().getPreprocessorOpts().showMacros) {
-    ci.getParsing().EmitPreprocessorMacros(outForPP);
-  } else if (ci.getInvocation().getPreprocessorOpts().noReformat) {
-    ci.getParsing().DumpCookedChars(outForPP);
-  } else {
-    ci.getParsing().EmitPreprocessedSource(
-        outForPP, !ci.getInvocation().getPreprocessorOpts().noLineDirectives);
-  }
-
-  // Print getDiagnostics from the prescanner
-  ci.getParsing().messages().Emit(llvm::errs(), ci.getAllCookedSources());
+  Fortran::frontend::formatOrDumpPrescanner(buf, outForPP, ci);
 
   // If a pre-defined output stream exists, dump the preprocessed content there
   if (!ci.isOutputStreamNull()) {
@@ -463,60 +444,31 @@ void PrintPreprocessedAction::executeAction() {
 }
 
 void DebugDumpProvenanceAction::executeAction() {
-  this->getInstance().getParsing().DumpProvenance(llvm::outs());
+  Fortran::frontend::dumpProvenance(this->getInstance());
 }
 
 void ParseSyntaxOnlyAction::executeAction() {}
 
 void DebugUnparseNoSemaAction::executeAction() {
-  auto &invoc = this->getInstance().getInvocation();
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // TODO: Options should come from CompilerInvocation
-  Unparse(llvm::outs(), *parseTree,
-          /*encoding=*/Fortran::parser::Encoding::UTF_8,
-          /*capitalizeKeywords=*/true, /*backslashEscapes=*/false,
-          /*preStatement=*/nullptr,
-          invoc.getUseAnalyzedObjectsForUnparse() ? &invoc.getAsFortran()
-                                                  : nullptr);
+  Fortran::frontend::debugUnparseNoSema(this->getInstance(), llvm::outs());
 }
 
 void DebugUnparseAction::executeAction() {
-  auto &invoc = this->getInstance().getInvocation();
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
   CompilerInstance &ci = this->getInstance();
   auto os{ci.createDefaultOutputFile(
       /*Binary=*/false, /*InFile=*/getCurrentFileOrBufferName())};
 
-  // TODO: Options should come from CompilerInvocation
-  Unparse(*os, *parseTree,
-          /*encoding=*/Fortran::parser::Encoding::UTF_8,
-          /*capitalizeKeywords=*/true, /*backslashEscapes=*/false,
-          /*preStatement=*/nullptr,
-          invoc.getUseAnalyzedObjectsForUnparse() ? &invoc.getAsFortran()
-                                                  : nullptr);
-
-  // Report fatal semantic errors
+  Fortran::frontend::debugUnparseNoSema(ci, *os);
   reportFatalSemanticErrors();
 }
 
 void DebugUnparseWithSymbolsAction::executeAction() {
-  auto &parseTree{*getInstance().getParsing().parseTree()};
-
-  Fortran::semantics::UnparseWithSymbols(
-      llvm::outs(), parseTree, /*encoding=*/Fortran::parser::Encoding::UTF_8);
-
-  // Report fatal semantic errors
+  Fortran::frontend::debugUnparseWithSymbols(this->getInstance());
   reportFatalSemanticErrors();
 }
 
 void DebugUnparseWithModulesAction::executeAction() {
-  auto &parseTree{*getInstance().getParsing().parseTree()};
-  CompilerInstance &ci{getInstance()};
-  Fortran::semantics::UnparseWithModules(
-      llvm::outs(), ci.getSemantics().context(), parseTree,
-      /*encoding=*/Fortran::parser::Encoding::UTF_8);
+  Fortran::frontend::debugUnparseWithModules(this->getInstance());
   reportFatalSemanticErrors();
 }
 
@@ -540,12 +492,7 @@ void DebugDumpAllAction::executeAction() {
   CompilerInstance &ci = this->getInstance();
 
   // Dump parse tree
-  auto &parseTree{getInstance().getParsing().parseTree()};
-  llvm::outs() << "========================";
-  llvm::outs() << " Flang: parse tree dump ";
-  llvm::outs() << "========================\n";
-  Fortran::parser::DumpTree(llvm::outs(), parseTree,
-                            &ci.getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(ci);
 
   if (!ci.getRtTyTables().schemata) {
     unsigned diagID = ci.getDiagnostics().getCustomDiagID(
@@ -564,21 +511,11 @@ void DebugDumpAllAction::executeAction() {
 }
 
 void DebugDumpParseTreeNoSemaAction::executeAction() {
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // Dump parse tree
-  Fortran::parser::DumpTree(
-      llvm::outs(), parseTree,
-      &this->getInstance().getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(this->getInstance());
 }
 
 void DebugDumpParseTreeAction::executeAction() {
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // Dump parse tree
-  Fortran::parser::DumpTree(
-      llvm::outs(), parseTree,
-      &this->getInstance().getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(this->getInstance());
 
   // Report fatal semantic errors
   reportFatalSemanticErrors();
@@ -586,62 +523,20 @@ void DebugDumpParseTreeAction::executeAction() {
 
 void DebugMeasureParseTreeAction::executeAction() {
   CompilerInstance &ci = this->getInstance();
-
-  // Parse. In case of failure, report and return.
-  ci.getParsing().Parse(llvm::outs());
-
-  if ((ci.getParsing().parseTree().has_value() &&
-       !ci.getParsing().consumedWholeFile()) ||
-      (!ci.getParsing().messages().empty() &&
-       (ci.getInvocation().getWarnAsErr() ||
-        ci.getParsing().messages().AnyFatalError()))) {
-    unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-        clang::DiagnosticsEngine::Error, "Could not parse %0");
-    ci.getDiagnostics().Report(diagID) << getCurrentFileOrBufferName();
-
-    ci.getParsing().messages().Emit(llvm::errs(),
-                                    this->getInstance().getAllCookedSources());
-    return;
-  }
-
-  // Report the getDiagnostics from parsing
-  ci.getParsing().messages().Emit(llvm::errs(), ci.getAllCookedSources());
-
-  auto &parseTree{*ci.getParsing().parseTree()};
-
-  // Measure the parse tree
-  MeasurementVisitor visitor;
-  Fortran::parser::Walk(parseTree, visitor);
-  llvm::outs() << "Parse tree comprises " << visitor.objects
-               << " objects and occupies " << visitor.bytes
-               << " total bytes.\n";
+  Fortran::frontend::debugMeasureParseTree(ci, getCurrentFileOrBufferName());
 }
 
 void DebugPreFIRTreeAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
   // Report and exit if fatal semantic errors are present
   if (reportFatalSemanticErrors()) {
     return;
   }
 
-  auto &parseTree{*ci.getParsing().parseTree()};
-
-  // Dump pre-FIR tree
-  if (auto ast{
-          Fortran::lower::createPFT(parseTree, ci.getSemanticsContext())}) {
-    Fortran::lower::dumpPFT(llvm::outs(), *ast);
-  } else {
-    unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-        clang::DiagnosticsEngine::Error, "Pre FIR Tree is NULL.");
-    ci.getDiagnostics().Report(diagID);
-  }
+  Fortran::frontend::dumpPreFIRTree(this->getInstance());
 }
 
 void DebugDumpParsingLogAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
-
-  ci.getParsing().Parse(llvm::errs());
-  ci.getParsing().DumpParsingLog(llvm::outs());
+  Fortran::frontend::debugDumpParsingLog(this->getInstance());
 }
 
 void GetDefinitionAction::executeAction() {
@@ -1473,17 +1368,7 @@ void InitOnlyAction::executeAction() {
 void PluginParseTreeAction::executeAction() {}
 
 void DebugDumpPFTAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
-
-  if (auto ast = Fortran::lower::createPFT(*ci.getParsing().parseTree(),
-                                           ci.getSemantics().context())) {
-    Fortran::lower::dumpPFT(llvm::outs(), *ast);
-    return;
-  }
-
-  unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-      clang::DiagnosticsEngine::Error, "Pre FIR Tree is NULL.");
-  ci.getDiagnostics().Report(diagID);
+  Fortran::frontend::dumpPreFIRTree(this->getInstance());
 }
 
 Fortran::parser::Parsing &PluginParseTreeAction::getParsing() {
diff --git a/flang/lib/Frontend/ParserActions.cpp b/flang/lib/Frontend/ParserActions.cpp
new file mode 100644
index 0000000000000..48cacaa5bed71
--- /dev/null
+++ b/flang/lib/Frontend/ParserActions.cpp
@@ -0,0 +1,152 @@
+//===--- ParserActions.cpp ------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/ParserActions.h"
+#include "flang/Frontend/CompilerInstance.h"
+#include "flang/Parser/dump-parse-tree.h"
+#include "flang/Parser/parsing.h"
+#include "flang/Parser/unparse.h"
+#include "flang/Parser/source.h"
+#include "flang/Parser/provenance.h"
+#include "flang/Semantics/unparse-with-symbols.h"
+#include "flang/Lower/Bridge.h"
+#include "flang/Lower/PFTBuilder.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace Fortran::frontend {
+
+Fortran::parser::AllCookedSources&
+getAllCooked(Fortran::frontend::CompilerInstance &ci) {
+  return ci.getParsing().allCooked();
+}
+
+void parseAndLowerTree(Fortran::frontend::CompilerInstance &ci,
+                       Fortran::lower::LoweringBridge &lb) {
+  Fortran::parser::Program &parseTree{*ci.getParsing().parseTree()};
+  lb.lower(parseTree, ci.getSem...
[truncated]

llvmbot · 2025-03-07T17:28:34Z

@llvm/pr-subscribers-flang-parser

Author: Kajetan Puchalski (mrkajetanp)

Changes

FrontendActions.cpp is currently one of the biggest compilation units in all of flang. Measuring its compilation gives the following metrics:

User time (seconds): 139.21
System time (seconds): 4.65
Maximum resident set size (kbytes): 5891440 (5.61 GB)

This commit separates out explicit invocations of the parser into a separate compilation unit - ParserActions.cpp - through helper functions in order to decrease the maximum compilation time and memory usage of a single unit.
After the split, the measurements of FrontendActions.cpp are as follows:

User time (seconds): 70.08
System time (seconds): 3.16
Maximum resident set size (kbytes): 3961492 (3.7 GB)

While the ones for the newly created ParserActions.cpp as follows:

User time (seconds): 104.33
System time (seconds): 3.37
Maximum resident set size (kbytes): 4185600 (3.99 GB)

Patch is 24.08 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/130309.diff

10 Files Affected:

(modified) flang/include/flang/Frontend/CompilerInstance.h (+1-2)
(modified) flang/include/flang/Frontend/CompilerInvocation.h (+2-2)
(modified) flang/include/flang/Frontend/FrontendActions.h (+1-18)
(added) flang/include/flang/Frontend/ParserActions.h (+65)
(added) flang/include/flang/Parser/options.h (+43)
(modified) flang/include/flang/Parser/parsing.h (+1-24)
(modified) flang/lib/Frontend/CMakeLists.txt (+1)
(modified) flang/lib/Frontend/FrontendAction.cpp (+1)
(modified) flang/lib/Frontend/FrontendActions.cpp (+16-131)
(added) flang/lib/Frontend/ParserActions.cpp (+152)

diff --git a/flang/include/flang/Frontend/CompilerInstance.h b/flang/include/flang/Frontend/CompilerInstance.h
index 509c9f4b9e91a..d34c128d01794 100644
--- a/flang/include/flang/Frontend/CompilerInstance.h
+++ b/flang/include/flang/Frontend/CompilerInstance.h
@@ -16,8 +16,7 @@
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/FrontendAction.h"
 #include "flang/Frontend/PreprocessorOptions.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Parser/provenance.h"
+#include "flang/Frontend/ParserActions.h"
 #include "flang/Semantics/runtime-type-info.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Support/StringOstream.h"
diff --git a/flang/include/flang/Frontend/CompilerInvocation.h b/flang/include/flang/Frontend/CompilerInvocation.h
index 9e6724be33033..d6ee1511cdb4b 100644
--- a/flang/include/flang/Frontend/CompilerInvocation.h
+++ b/flang/include/flang/Frontend/CompilerInvocation.h
@@ -18,7 +18,7 @@
 #include "flang/Frontend/PreprocessorOptions.h"
 #include "flang/Frontend/TargetOptions.h"
 #include "flang/Lower/LoweringOptions.h"
-#include "flang/Parser/parsing.h"
+#include "flang/Parser/options.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Support/LangOptions.h"
 #include "mlir/Support/Timing.h"
@@ -29,7 +29,7 @@
 
 namespace llvm {
 class TargetMachine;
-}
+} // namespace llvm
 
 namespace Fortran::frontend {
 
diff --git a/flang/include/flang/Frontend/FrontendActions.h b/flang/include/flang/Frontend/FrontendActions.h
index 4e3d3cb2657db..f9a45bd6c0a56 100644
--- a/flang/include/flang/Frontend/FrontendActions.h
+++ b/flang/include/flang/Frontend/FrontendActions.h
@@ -13,10 +13,8 @@
 #ifndef FORTRAN_FRONTEND_FRONTENDACTIONS_H
 #define FORTRAN_FRONTEND_FRONTENDACTIONS_H
 
-#include "flang/Frontend/CodeGenOptions.h"
 #include "flang/Frontend/FrontendAction.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Semantics/semantics.h"
+#include "flang/Frontend/ParserActions.h"
 
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/OwningOpRef.h"
@@ -26,21 +24,6 @@
 
 namespace Fortran::frontend {
 
-// TODO: This is a copy from f18.cpp. It doesn't really belong here and should
-// be moved to a more suitable place in future.
-struct MeasurementVisitor {
-  template <typename A>
-  bool Pre(const A &) {
-    return true;
-  }
-  template <typename A>
-  void Post(const A &) {
-    ++objects;
-    bytes += sizeof(A);
-  }
-  size_t objects{0}, bytes{0};
-};
-
 //===----------------------------------------------------------------------===//
 // Custom Consumer Actions
 //===----------------------------------------------------------------------===//
diff --git a/flang/include/flang/Frontend/ParserActions.h b/flang/include/flang/Frontend/ParserActions.h
new file mode 100644
index 0000000000000..9c76f2bc9b1ed
--- /dev/null
+++ b/flang/include/flang/Frontend/ParserActions.h
@@ -0,0 +1,65 @@
+//===- ParserActions.h -------------------------------------------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_PARSER_ACTIONS_H_
+#define FORTRAN_PARSER_ACTIONS_H_
+
+#include <string>
+
+namespace llvm {
+    class raw_string_ostream;
+    class raw_ostream;
+    class StringRef;
+} // namespace llvm
+
+namespace Fortran::lower {
+    class LoweringBridge;
+} // namespace Fortran::lower
+
+namespace Fortran::parser {
+    class Parsing;
+    class AllCookedSources;
+} // namespace Fortran::parser
+
+namespace lower::pft {
+    class Program;
+} // namespace lower::pft
+
+//=== Frontend Parser helpers ===
+
+namespace Fortran::frontend {
+class CompilerInstance;
+
+Fortran::parser::AllCookedSources& getAllCooked(CompilerInstance &ci);
+
+void parseAndLowerTree(CompilerInstance &ci, Fortran::lower::LoweringBridge &lb);
+
+void dumpTree(CompilerInstance &ci);
+
+void dumpProvenance(CompilerInstance &ci);
+
+void dumpPreFIRTree(CompilerInstance &ci);
+
+void formatOrDumpPrescanner(std::string &buf, llvm::raw_string_ostream &outForPP, CompilerInstance &ci);
+
+void debugMeasureParseTree(CompilerInstance &ci, llvm::StringRef filename);
+
+void debugUnparseNoSema(CompilerInstance &ci, llvm::raw_ostream &out);
+
+void debugUnparseWithSymbols(CompilerInstance &ci);
+
+void debugUnparseWithModules(CompilerInstance &ci);
+
+void debugDumpParsingLog(CompilerInstance &ci);
+} // namespace Fortran::frontend
+
+#endif // FORTRAN_PARSER_ACTIONS_H_
diff --git a/flang/include/flang/Parser/options.h b/flang/include/flang/Parser/options.h
new file mode 100644
index 0000000000000..a9f93221ffdf1
--- /dev/null
+++ b/flang/include/flang/Parser/options.h
@@ -0,0 +1,43 @@
+//===-- include/flang/Parser/options.h --------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_PARSER_OPTIONS_H_
+#define FORTRAN_PARSER_OPTIONS_H_
+
+#include <optional>
+#include <string>
+#include <vector>
+
+#include "characters.h"
+#include "flang/Support/Fortran-features.h"
+
+namespace Fortran::parser {
+
+struct Options {
+  Options() {}
+
+  using Predefinition = std::pair<std::string, std::optional<std::string>>;
+
+  bool isFixedForm{false};
+  int fixedFormColumns{72};
+  common::LanguageFeatureControl features;
+  std::vector<std::string> searchDirectories;
+  std::vector<std::string> intrinsicModuleDirectories;
+  std::vector<Predefinition> predefinitions;
+  bool instrumentedParse{false};
+  bool isModuleFile{false};
+  bool needProvenanceRangeToCharBlockMappings{false};
+  Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
+  bool prescanAndReformat{false}; // -E
+  bool expandIncludeLinesInPreprocessedOutput{true};
+  bool showColors{false};
+};
+
+} // namespace Fortran::parser
+
+#endif // FORTRAN_PARSER_OPTIONS_H_
diff --git a/flang/include/flang/Parser/parsing.h b/flang/include/flang/Parser/parsing.h
index 116b6bd6f191f..3b4f5f5145a58 100644
--- a/flang/include/flang/Parser/parsing.h
+++ b/flang/include/flang/Parser/parsing.h
@@ -9,41 +9,18 @@
 #ifndef FORTRAN_PARSER_PARSING_H_
 #define FORTRAN_PARSER_PARSING_H_
 
-#include "characters.h"
 #include "instrumented-parser.h"
 #include "message.h"
 #include "parse-tree.h"
 #include "provenance.h"
+#include "options.h"
 #include "flang/Parser/preprocessor.h"
-#include "flang/Support/Fortran-features.h"
 #include "llvm/Support/raw_ostream.h"
 #include <optional>
 #include <string>
-#include <utility>
-#include <vector>
 
 namespace Fortran::parser {
 
-struct Options {
-  Options() {}
-
-  using Predefinition = std::pair<std::string, std::optional<std::string>>;
-
-  bool isFixedForm{false};
-  int fixedFormColumns{72};
-  common::LanguageFeatureControl features;
-  std::vector<std::string> searchDirectories;
-  std::vector<std::string> intrinsicModuleDirectories;
-  std::vector<Predefinition> predefinitions;
-  bool instrumentedParse{false};
-  bool isModuleFile{false};
-  bool needProvenanceRangeToCharBlockMappings{false};
-  Fortran::parser::Encoding encoding{Fortran::parser::Encoding::UTF_8};
-  bool prescanAndReformat{false}; // -E
-  bool expandIncludeLinesInPreprocessedOutput{true};
-  bool showColors{false};
-};
-
 class Parsing {
 public:
   explicit Parsing(AllCookedSources &);
diff --git a/flang/lib/Frontend/CMakeLists.txt b/flang/lib/Frontend/CMakeLists.txt
index 80d63fca6fb76..c80373799b015 100644
--- a/flang/lib/Frontend/CMakeLists.txt
+++ b/flang/lib/Frontend/CMakeLists.txt
@@ -5,6 +5,7 @@ add_flang_library(flangFrontend
   CompilerInstance.cpp
   CompilerInvocation.cpp
   CodeGenOptions.cpp
+  ParserActions.cpp
   FrontendAction.cpp
   FrontendActions.cpp
   FrontendOptions.cpp
diff --git a/flang/lib/Frontend/FrontendAction.cpp b/flang/lib/Frontend/FrontendAction.cpp
index 9a555bc7cd23b..5813f776efad4 100644
--- a/flang/lib/Frontend/FrontendAction.cpp
+++ b/flang/lib/Frontend/FrontendAction.cpp
@@ -17,6 +17,7 @@
 #include "flang/Frontend/FrontendPluginRegistry.h"
 #include "clang/Basic/DiagnosticFrontend.h"
 #include "llvm/Support/Errc.h"
+#include "flang/Parser/parsing.h"
 #include "llvm/Support/VirtualFileSystem.h"
 
 using namespace Fortran::frontend;
diff --git a/flang/lib/Frontend/FrontendActions.cpp b/flang/lib/Frontend/FrontendActions.cpp
index 76d329d043731..69d7cf6cc9e7a 100644
--- a/flang/lib/Frontend/FrontendActions.cpp
+++ b/flang/lib/Frontend/FrontendActions.cpp
@@ -14,9 +14,8 @@
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Frontend/FrontendOptions.h"
-#include "flang/Frontend/PreprocessorOptions.h"
+#include "flang/Frontend/ParserActions.h"
 #include "flang/Lower/Bridge.h"
-#include "flang/Lower/PFTBuilder.h"
 #include "flang/Lower/Support/Verifier.h"
 #include "flang/Optimizer/Dialect/Support/FIRContext.h"
 #include "flang/Optimizer/Dialect/Support/KindMapping.h"
@@ -25,13 +24,7 @@
 #include "flang/Optimizer/Support/InitFIR.h"
 #include "flang/Optimizer/Support/Utils.h"
 #include "flang/Optimizer/Transforms/Passes.h"
-#include "flang/Parser/dump-parse-tree.h"
-#include "flang/Parser/parsing.h"
-#include "flang/Parser/provenance.h"
-#include "flang/Parser/source.h"
-#include "flang/Parser/unparse.h"
 #include "flang/Semantics/runtime-type-info.h"
-#include "flang/Semantics/semantics.h"
 #include "flang/Semantics/unparse-with-symbols.h"
 #include "flang/Support/default-kinds.h"
 #include "flang/Tools/CrossToolHelpers.h"
@@ -318,7 +311,7 @@ bool CodeGenAction::beginSourceFileAction() {
       *mlirCtx, ci.getSemanticsContext(), defKinds,
       ci.getSemanticsContext().intrinsics(),
       ci.getSemanticsContext().targetCharacteristics(),
-      ci.getParsing().allCooked(), ci.getInvocation().getTargetOpts().triple,
+      Fortran::frontend::getAllCooked(ci), ci.getInvocation().getTargetOpts().triple,
       kindMap, ci.getInvocation().getLoweringOpts(),
       ci.getInvocation().getFrontendOpts().envDefaults,
       ci.getInvocation().getFrontendOpts().features, targetMachine,
@@ -333,8 +326,7 @@ bool CodeGenAction::beginSourceFileAction() {
   }
 
   // Create a parse tree and lower it to FIR
-  Fortran::parser::Program &parseTree{*ci.getParsing().parseTree()};
-  lb.lower(parseTree, ci.getSemanticsContext());
+  Fortran::frontend::parseAndLowerTree(ci, lb);
 
   // Fetch module from lb, so we can set
   mlirModule = lb.getModuleAndRelease();
@@ -431,19 +423,8 @@ void PrintPreprocessedAction::executeAction() {
   std::string buf;
   llvm::raw_string_ostream outForPP{buf};
 
-  // Format or dump the prescanner's output
   CompilerInstance &ci = this->getInstance();
-  if (ci.getInvocation().getPreprocessorOpts().showMacros) {
-    ci.getParsing().EmitPreprocessorMacros(outForPP);
-  } else if (ci.getInvocation().getPreprocessorOpts().noReformat) {
-    ci.getParsing().DumpCookedChars(outForPP);
-  } else {
-    ci.getParsing().EmitPreprocessedSource(
-        outForPP, !ci.getInvocation().getPreprocessorOpts().noLineDirectives);
-  }
-
-  // Print getDiagnostics from the prescanner
-  ci.getParsing().messages().Emit(llvm::errs(), ci.getAllCookedSources());
+  Fortran::frontend::formatOrDumpPrescanner(buf, outForPP, ci);
 
   // If a pre-defined output stream exists, dump the preprocessed content there
   if (!ci.isOutputStreamNull()) {
@@ -463,60 +444,31 @@ void PrintPreprocessedAction::executeAction() {
 }
 
 void DebugDumpProvenanceAction::executeAction() {
-  this->getInstance().getParsing().DumpProvenance(llvm::outs());
+  Fortran::frontend::dumpProvenance(this->getInstance());
 }
 
 void ParseSyntaxOnlyAction::executeAction() {}
 
 void DebugUnparseNoSemaAction::executeAction() {
-  auto &invoc = this->getInstance().getInvocation();
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // TODO: Options should come from CompilerInvocation
-  Unparse(llvm::outs(), *parseTree,
-          /*encoding=*/Fortran::parser::Encoding::UTF_8,
-          /*capitalizeKeywords=*/true, /*backslashEscapes=*/false,
-          /*preStatement=*/nullptr,
-          invoc.getUseAnalyzedObjectsForUnparse() ? &invoc.getAsFortran()
-                                                  : nullptr);
+  Fortran::frontend::debugUnparseNoSema(this->getInstance(), llvm::outs());
 }
 
 void DebugUnparseAction::executeAction() {
-  auto &invoc = this->getInstance().getInvocation();
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
   CompilerInstance &ci = this->getInstance();
   auto os{ci.createDefaultOutputFile(
       /*Binary=*/false, /*InFile=*/getCurrentFileOrBufferName())};
 
-  // TODO: Options should come from CompilerInvocation
-  Unparse(*os, *parseTree,
-          /*encoding=*/Fortran::parser::Encoding::UTF_8,
-          /*capitalizeKeywords=*/true, /*backslashEscapes=*/false,
-          /*preStatement=*/nullptr,
-          invoc.getUseAnalyzedObjectsForUnparse() ? &invoc.getAsFortran()
-                                                  : nullptr);
-
-  // Report fatal semantic errors
+  Fortran::frontend::debugUnparseNoSema(ci, *os);
   reportFatalSemanticErrors();
 }
 
 void DebugUnparseWithSymbolsAction::executeAction() {
-  auto &parseTree{*getInstance().getParsing().parseTree()};
-
-  Fortran::semantics::UnparseWithSymbols(
-      llvm::outs(), parseTree, /*encoding=*/Fortran::parser::Encoding::UTF_8);
-
-  // Report fatal semantic errors
+  Fortran::frontend::debugUnparseWithSymbols(this->getInstance());
   reportFatalSemanticErrors();
 }
 
 void DebugUnparseWithModulesAction::executeAction() {
-  auto &parseTree{*getInstance().getParsing().parseTree()};
-  CompilerInstance &ci{getInstance()};
-  Fortran::semantics::UnparseWithModules(
-      llvm::outs(), ci.getSemantics().context(), parseTree,
-      /*encoding=*/Fortran::parser::Encoding::UTF_8);
+  Fortran::frontend::debugUnparseWithModules(this->getInstance());
   reportFatalSemanticErrors();
 }
 
@@ -540,12 +492,7 @@ void DebugDumpAllAction::executeAction() {
   CompilerInstance &ci = this->getInstance();
 
   // Dump parse tree
-  auto &parseTree{getInstance().getParsing().parseTree()};
-  llvm::outs() << "========================";
-  llvm::outs() << " Flang: parse tree dump ";
-  llvm::outs() << "========================\n";
-  Fortran::parser::DumpTree(llvm::outs(), parseTree,
-                            &ci.getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(ci);
 
   if (!ci.getRtTyTables().schemata) {
     unsigned diagID = ci.getDiagnostics().getCustomDiagID(
@@ -564,21 +511,11 @@ void DebugDumpAllAction::executeAction() {
 }
 
 void DebugDumpParseTreeNoSemaAction::executeAction() {
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // Dump parse tree
-  Fortran::parser::DumpTree(
-      llvm::outs(), parseTree,
-      &this->getInstance().getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(this->getInstance());
 }
 
 void DebugDumpParseTreeAction::executeAction() {
-  auto &parseTree{getInstance().getParsing().parseTree()};
-
-  // Dump parse tree
-  Fortran::parser::DumpTree(
-      llvm::outs(), parseTree,
-      &this->getInstance().getInvocation().getAsFortran());
+  Fortran::frontend::dumpTree(this->getInstance());
 
   // Report fatal semantic errors
   reportFatalSemanticErrors();
@@ -586,62 +523,20 @@ void DebugDumpParseTreeAction::executeAction() {
 
 void DebugMeasureParseTreeAction::executeAction() {
   CompilerInstance &ci = this->getInstance();
-
-  // Parse. In case of failure, report and return.
-  ci.getParsing().Parse(llvm::outs());
-
-  if ((ci.getParsing().parseTree().has_value() &&
-       !ci.getParsing().consumedWholeFile()) ||
-      (!ci.getParsing().messages().empty() &&
-       (ci.getInvocation().getWarnAsErr() ||
-        ci.getParsing().messages().AnyFatalError()))) {
-    unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-        clang::DiagnosticsEngine::Error, "Could not parse %0");
-    ci.getDiagnostics().Report(diagID) << getCurrentFileOrBufferName();
-
-    ci.getParsing().messages().Emit(llvm::errs(),
-                                    this->getInstance().getAllCookedSources());
-    return;
-  }
-
-  // Report the getDiagnostics from parsing
-  ci.getParsing().messages().Emit(llvm::errs(), ci.getAllCookedSources());
-
-  auto &parseTree{*ci.getParsing().parseTree()};
-
-  // Measure the parse tree
-  MeasurementVisitor visitor;
-  Fortran::parser::Walk(parseTree, visitor);
-  llvm::outs() << "Parse tree comprises " << visitor.objects
-               << " objects and occupies " << visitor.bytes
-               << " total bytes.\n";
+  Fortran::frontend::debugMeasureParseTree(ci, getCurrentFileOrBufferName());
 }
 
 void DebugPreFIRTreeAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
   // Report and exit if fatal semantic errors are present
   if (reportFatalSemanticErrors()) {
     return;
   }
 
-  auto &parseTree{*ci.getParsing().parseTree()};
-
-  // Dump pre-FIR tree
-  if (auto ast{
-          Fortran::lower::createPFT(parseTree, ci.getSemanticsContext())}) {
-    Fortran::lower::dumpPFT(llvm::outs(), *ast);
-  } else {
-    unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-        clang::DiagnosticsEngine::Error, "Pre FIR Tree is NULL.");
-    ci.getDiagnostics().Report(diagID);
-  }
+  Fortran::frontend::dumpPreFIRTree(this->getInstance());
 }
 
 void DebugDumpParsingLogAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
-
-  ci.getParsing().Parse(llvm::errs());
-  ci.getParsing().DumpParsingLog(llvm::outs());
+  Fortran::frontend::debugDumpParsingLog(this->getInstance());
 }
 
 void GetDefinitionAction::executeAction() {
@@ -1473,17 +1368,7 @@ void InitOnlyAction::executeAction() {
 void PluginParseTreeAction::executeAction() {}
 
 void DebugDumpPFTAction::executeAction() {
-  CompilerInstance &ci = this->getInstance();
-
-  if (auto ast = Fortran::lower::createPFT(*ci.getParsing().parseTree(),
-                                           ci.getSemantics().context())) {
-    Fortran::lower::dumpPFT(llvm::outs(), *ast);
-    return;
-  }
-
-  unsigned diagID = ci.getDiagnostics().getCustomDiagID(
-      clang::DiagnosticsEngine::Error, "Pre FIR Tree is NULL.");
-  ci.getDiagnostics().Report(diagID);
+  Fortran::frontend::dumpPreFIRTree(this->getInstance());
 }
 
 Fortran::parser::Parsing &PluginParseTreeAction::getParsing() {
diff --git a/flang/lib/Frontend/ParserActions.cpp b/flang/lib/Frontend/ParserActions.cpp
new file mode 100644
index 0000000000000..48cacaa5bed71
--- /dev/null
+++ b/flang/lib/Frontend/ParserActions.cpp
@@ -0,0 +1,152 @@
+//===--- ParserActions.cpp ------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Frontend/ParserActions.h"
+#include "flang/Frontend/CompilerInstance.h"
+#include "flang/Parser/dump-parse-tree.h"
+#include "flang/Parser/parsing.h"
+#include "flang/Parser/unparse.h"
+#include "flang/Parser/source.h"
+#include "flang/Parser/provenance.h"
+#include "flang/Semantics/unparse-with-symbols.h"
+#include "flang/Lower/Bridge.h"
+#include "flang/Lower/PFTBuilder.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace Fortran::frontend {
+
+Fortran::parser::AllCookedSources&
+getAllCooked(Fortran::frontend::CompilerInstance &ci) {
+  return ci.getParsing().allCooked();
+}
+
+void parseAndLowerTree(Fortran::frontend::CompilerInstance &ci,
+                       Fortran::lower::LoweringBridge &lb) {
+  Fortran::parser::Program &parseTree{*ci.getParsing().parseTree()};
+  lb.lower(parseTree, ci.getSem...
[truncated]

mrkajetanp · 2025-03-07T17:30:39Z

Review requests:
@tblah @DavidTruby @tarunprabhu @kparzysz @klausler @kiranchandramohan

tblah

Thank you for this @mrkajetanp and welcome to flang!

Our intention here is to make flang a little eaiser to build on machines with less memory per CPU core. This change partitions one of the most difficult to compile translation units into more manageable chunks. Furthermore, this should speed up incremental re-builds: for example, if flang is built fully, then a change is made to flang's parse tree, now only ParserActions will have to be rebuilt (which is much faster than the older FrontendActions.cpp).

LGTM but please wait for approval from other reviewers.

github-actions · 2025-03-07T17:43:01Z

✅ With the latest revision this PR passed the C/C++ code formatter.

kparzysz

Very nice!

flang/include/flang/Frontend/ParserActions.h

flang/include/flang/Parser/options.h

flang/lib/Frontend/ParserActions.cpp

github-actions · 2025-03-10T11:34:06Z

@mrkajetanp Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

llvmbot added flang:driver flang Flang issues not falling into any other category flang:parser labels Mar 7, 2025

tblah requested review from klausler, kiranchandramohan, DavidTruby, tblah, tarunprabhu and kparzysz March 7, 2025 17:31

klausler removed their request for review March 7, 2025 17:37

tblah reviewed Mar 7, 2025

View reviewed changes

kparzysz approved these changes Mar 7, 2025

View reviewed changes

flang/include/flang/Frontend/ParserActions.h Outdated Show resolved Hide resolved

flang/include/flang/Parser/options.h Show resolved Hide resolved

flang/lib/Frontend/ParserActions.cpp Outdated Show resolved Hide resolved

[flang] Fix ParserActions style & formatting

bbfbc0e

tblah merged commit 0c7e895 into llvm:main Mar 10, 2025
11 checks passed

shiltian mentioned this pull request Mar 10, 2025

[AMDGPU] Fix test failures when expensive checks are enabled #130644

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[flang] Move parser invocations into ParserActions #130309

[flang] Move parser invocations into ParserActions #130309

Uh oh!

mrkajetanp commented Mar 7, 2025

Uh oh!

github-actions bot commented Mar 7, 2025

Uh oh!

llvmbot commented Mar 7, 2025

Uh oh!

llvmbot commented Mar 7, 2025

Uh oh!

mrkajetanp commented Mar 7, 2025

Uh oh!

tblah left a comment

Uh oh!

github-actions bot commented Mar 7, 2025 •

edited

Loading

Uh oh!

kparzysz left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

Uh oh!

[flang] Move parser invocations into ParserActions #130309

[flang] Move parser invocations into ParserActions #130309

Uh oh!

Conversation

mrkajetanp commented Mar 7, 2025

Uh oh!

github-actions bot commented Mar 7, 2025

Uh oh!

llvmbot commented Mar 7, 2025

Uh oh!

llvmbot commented Mar 7, 2025

Uh oh!

mrkajetanp commented Mar 7, 2025

Uh oh!

tblah left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kparzysz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

Uh oh!

github-actions bot commented Mar 7, 2025 •

edited

Loading