[clang][nvlink-wrapper] Add support for opt-remarks command line options

[miguelcsx]

Problem

When using -fsave-optimization-record with offloading, the Clang driver passes optimization record options like -plugin-opt=opt-remarks-format=yaml to clang-nvlink-wrapper. However, the wrapper doesn't recognize these options, causing compilation to fail with:

Solution

This patch adds support for the standard optimization record command line options to clang-nvlink-wrapper, matching the interface provided by LLD and gold-plugin as documented in the LLVM Remarks documentation.

Changes

• NVLinkOpts.td: Added definitions for opt-remarks-filename, opt-remarks-format, opt-remarks-filter, and opt-remarks-with-hotness options
• NVLinkOpts.td: Added plugin-opt= aliases for these options to match what the Clang driver sends
• ClangNVLinkWrapper.cpp: Updated createLTO() to use command line arguments when available, falling back to existing global variables

Testing

The fix allows -fsave-optimization-record to work correctly with offloading, generating optimization records during the LTO phase without throwing unknown argument errors.

This change maintains backward compatibility and follows the existing pattern used by other LLVM linkers.

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[llvmbot]

@llvm/pr-subscribers-offload

@llvm/pr-subscribers-clang

Author: Miguel Cárdenas (miguelcsx)

Changes

Problem

When using -fsave-optimization-record with offloading, the Clang driver passes optimization record options like -plugin-opt=opt-remarks-format=yaml to clang-nvlink-wrapper. However, the wrapper doesn't recognize these options, causing compilation to fail with:

Solution

This patch adds support for the standard optimization record command line options to clang-nvlink-wrapper, matching the interface provided by LLD and gold-plugin as documented in the LLVM Remarks documentation.

Changes

• NVLinkOpts.td: Added definitions for opt-remarks-filename, opt-remarks-format, opt-remarks-filter, and opt-remarks-with-hotness options
• NVLinkOpts.td: Added plugin-opt= aliases for these options to match what the Clang driver sends
• ClangNVLinkWrapper.cpp: Updated createLTO() to use command line arguments when available, falling back to existing global variables

Testing

The fix allows -fsave-optimization-record to work correctly with offloading, generating optimization records during the LTO phase without throwing unknown argument errors.

This change maintains backward compatibility and follows the existing pattern used by other LLVM linkers.

---

Full diff: https://github.com/llvm/llvm-project/pull/145200.diff

2 Files Affected:

• (modified) clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp (+16-4)
• (modified) clang/tools/clang-nvlink-wrapper/NVLinkOpts.td (+19)

diff --git a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
index faf73a7c2f193..4e36a04e0286b 100644
--- a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
+++ b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
@@ -341,11 +341,23 @@ Expected<std::unique_ptr<lto::LTO>> createLTO(const ArgList &Args) {
   Conf.CPU = Args.getLastArgValue(OPT_arch);
   Conf.Options = codegen::InitTargetOptionsFromCodeGenFlags(Triple);
 
-  Conf.RemarksFilename = RemarksFilename;
-  Conf.RemarksPasses = RemarksPasses;
-  Conf.RemarksWithHotness = RemarksWithHotness;
+  if (auto *Arg = Args.getLastArg(OPT_opt_remarks_filename))
+    Conf.RemarksFilename = Arg->getValue();
+  else
+    Conf.RemarksFilename = RemarksFilename;
+  
+  if (auto *Arg = Args.getLastArg(OPT_opt_remarks_filter))
+    Conf.RemarksPasses = Arg->getValue();
+  else
+    Conf.RemarksPasses = RemarksPasses;
+  
+  if (auto *Arg = Args.getLastArg(OPT_opt_remarks_format))
+    Conf.RemarksFormat = Arg->getValue();
+  else
+    Conf.RemarksFormat = RemarksFormat;
+  
+  Conf.RemarksWithHotness = Args.hasArg(OPT_opt_remarks_with_hotness) || RemarksWithHotness;
   Conf.RemarksHotnessThreshold = RemarksHotnessThreshold;
-  Conf.RemarksFormat = RemarksFormat;
 
   Conf.MAttrs = llvm::codegen::getMAttrs();
   std::optional<CodeGenOptLevel> CGOptLevelOrNone =
diff --git a/clang/tools/clang-nvlink-wrapper/NVLinkOpts.td b/clang/tools/clang-nvlink-wrapper/NVLinkOpts.td
index 6de1a25c14f8b..a0f7dce5bf6b3 100644
--- a/clang/tools/clang-nvlink-wrapper/NVLinkOpts.td
+++ b/clang/tools/clang-nvlink-wrapper/NVLinkOpts.td
@@ -72,6 +72,25 @@ def : Joined<["--", "-"], "plugin-opt=emit-llvm">,
   Flags<[WrapperOnlyOption]>, Alias<lto_emit_llvm>;
 def : Joined<["--", "-"], "plugin-opt=emit-asm">,
   Flags<[WrapperOnlyOption]>, Alias<lto_emit_asm>;
+
+def opt_remarks_filename : Separate<["--"], "opt-remarks-filename">,
+  Flags<[WrapperOnlyOption]>, HelpText<"YAML output file for optimization remarks">;
+def opt_remarks_format : Separate<["--"], "opt-remarks-format">,
+  Flags<[WrapperOnlyOption]>, HelpText<"The format used for serializing remarks (default: YAML)">;
+def opt_remarks_filter : Separate<["--"], "opt-remarks-filter">, 
+  Flags<[WrapperOnlyOption]>, HelpText<"Regex for the passes that need to be serialized to the output file">;
+def opt_remarks_with_hotness : Flag<["--"], "opt-remarks-with-hotness">,
+  Flags<[WrapperOnlyOption]>, HelpText<"Include hotness information in the optimization remarks file">;
+
+def : Joined<["--", "-"], "plugin-opt=opt-remarks-filename=">,
+  Flags<[WrapperOnlyOption]>, Alias<opt_remarks_filename>;
+def : Joined<["--", "-"], "plugin-opt=opt-remarks-format=">,
+  Flags<[WrapperOnlyOption]>, Alias<opt_remarks_format>;
+def : Joined<["--", "-"], "plugin-opt=opt-remarks-filter=">,
+  Flags<[WrapperOnlyOption]>, Alias<opt_remarks_filter>;
+def : Flag<["--", "-"], "plugin-opt=opt-remarks-with-hotness">,
+  Flags<[WrapperOnlyOption]>, Alias<opt_remarks_with_hotness>;
+
 def plugin_opt : Joined<["--", "-"], "plugin-opt=">, Flags<[WrapperOnlyOption]>,
   HelpText<"Options passed to LLVM, not including the Clang invocation. Use "
            "'--plugin-opt=--help' for a list of options.">;

[jhuber6]

So, this is probably a little weird since we now have two ways this is parsed. One through the LLVM commandline interface and another here as the command options interface. The reason we had the previous one was I think related to plugins expecting these to be defined somewhere? I don't exactly remember, but this is fine because we now accept the plugin opt version which is more consistent since we are pretending this is a functional linker after all.

[github-actions]

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:

git-clang-format --diff HEAD~1 HEAD --extensions cpp -- clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp

View the diff from clang-format here.

diff --git a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
index ee7cca7bb..4b6397121 100644
--- a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
+++ b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp
@@ -348,7 +348,7 @@ Expected<std::unique_ptr<lto::LTO>> createLTO(const ArgList &Args) {
   Conf.RemarksFormat =
       Args.getLastArgValue(OPT_opt_remarks_format, RemarksFormat);
 
-  Conf.RemarksWithHotness = 
+  Conf.RemarksWithHotness =
       Args.hasArg(OPT_opt_remarks_with_hotness) || RemarksWithHotness;
   Conf.RemarksHotnessThreshold = RemarksHotnessThreshold;
 

[jhuber6]

Can you see why the CI is unhappy?