[Multilib] Custom flags processing for library selection #110659

vhscampos · 2024-10-01T12:53:41Z

This patch is the third step to extend the current multilib system to support the selection of library variants which do not correspond to existing command-line options.

Proposal can be found in https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

The multilib mechanism supports libraries that target code generation or language options such as --target, -mcpu, -mfpu, -mbranch-protection. However, some library variants are particular to features that do not correspond to any command-line options. Examples include variants for multithreading and semihosting.

This work introduces a way to instruct the multilib system to consider these features in library selection. This particular patch is comprised of the core processing of these flags.

Custom flags in the command-line are read and forwarded to the multilib system. If multiple flag values are present for the same flag declaration, the last one wins. Default flag values are inserted for flag declarations for which no value was given.
Feed MacroDefines back into the driver. Each item <string> in the MacroDefines list is formatted as -D<string>.

Library variants should list their requirement on one or more custom flags like they do for any other flag. The new command-line option is passed as-is to the multilib system, therefore it should be listed in the format -fmultilib-flag=<str>.

Moreover, a variant that does not specify a requirement on any particular flag can be matched against any value of that flag.

If the user specifies -fmultilib-flag=<name> with a name that is invalid, but close enough to any valid flag value name in terms of edit distance, a suggesting error is shown:

error: unsupported option '-fmultilib-flag=invalidname'; did you mean '-fmultilib-flag=validname'?

The candidate with the smallest edit distance is chosen for the suggestion, up to a certain maximum value (implementation detail), after which a non-suggesting error is shown instead:

error: unsupported option '-fmultilib-flag=invalidname'

llvmbot · 2024-10-24T09:18:05Z

@llvm/pr-subscribers-clang-driver

Author: Victor Campos (vhscampos)

Changes

Select library variants in the multilib system using the flags passed in the '-fmultilib-flag=' format.

Multilib flags that were not passed in the command-line have their default value fed into the library selection mechanism.

A warning is shown if the flag's value is invalid. The closest valid value is suggested (string edit distance).

Details about this change can be found in this thread: https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

Full diff: https://github.com/llvm/llvm-project/pull/110659.diff

5 Files Affected:

(modified) clang/include/clang/Basic/DiagnosticDriverKinds.td (+6)
(modified) clang/include/clang/Driver/Multilib.h (+3)
(modified) clang/lib/Driver/Driver.cpp (+1-3)
(modified) clang/lib/Driver/Multilib.cpp (+134-3)
(added) clang/test/Driver/baremetal-multilib-custom-flags.yaml (+57)

diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index 65551bd7761a9d..6874614557f837 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -14,6 +14,12 @@ def err_drv_no_such_file_with_suggestion : Error<
 def err_drv_unsupported_opt : Error<"unsupported option '%0'">;
 def err_drv_unsupported_opt_with_suggestion : Error<
   "unsupported option '%0'; did you mean '%1'?">;
+def warn_drv_unsupported_opt : Warning<
+  "unsupported option '%0'">,
+  InGroup<UnusedCommandLineArgument>;
+def warn_drv_unsupported_opt_with_suggestion : Warning<
+  "unsupported option '%0'; did you mean '%1'?">,
+  InGroup<UnusedCommandLineArgument>;
 def err_drv_unsupported_opt_for_target : Error<
   "unsupported option '%0' for target '%1'">;
 def err_drv_unsupported_opt_for_language_mode : Error<
diff --git a/clang/include/clang/Driver/Multilib.h b/clang/include/clang/Driver/Multilib.h
index 333b1d2b555bd9..f99d4e6313ff26 100644
--- a/clang/include/clang/Driver/Multilib.h
+++ b/clang/include/clang/Driver/Multilib.h
@@ -163,6 +163,9 @@ class MultilibSet {
   const_iterator begin() const { return Multilibs.begin(); }
   const_iterator end() const { return Multilibs.end(); }
 
+  Multilib::flags_list
+  processCustomFlags(const Driver &D, const Multilib::flags_list &Flags) const;
+
   /// Select compatible variants, \returns false if none are compatible
   bool select(const Driver &D, const Multilib::flags_list &Flags,
               llvm::SmallVectorImpl<Multilib> &) const;
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index 9878a9dad78d40..cee10d36070616 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -2324,9 +2324,7 @@ bool Driver::HandleImmediateArgs(Compilation &C) {
   }
 
   if (C.getArgs().hasArg(options::OPT_print_multi_lib)) {
-    for (const Multilib &Multilib : TC.getMultilibs())
-      if (!Multilib.isError())
-        llvm::outs() << Multilib << "\n";
+    llvm::outs() << TC.getMultilibs();
     return false;
   }
 
diff --git a/clang/lib/Driver/Multilib.cpp b/clang/lib/Driver/Multilib.cpp
index 81fe97517b0f91..a4a4a74a9c7ad4 100644
--- a/clang/lib/Driver/Multilib.cpp
+++ b/clang/lib/Driver/Multilib.cpp
@@ -12,6 +12,7 @@
 #include "clang/Driver/Driver.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/SmallSet.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Error.h"
@@ -95,9 +96,113 @@ MultilibSet &MultilibSet::FilterOut(FilterCallback F) {
 
 void MultilibSet::push_back(const Multilib &M) { Multilibs.push_back(M); }
 
+static void WarnUnclaimedMultilibCustomFlags(
+    const Driver &D, const SmallVector<StringRef> &UnclaimedCustomFlagValues,
+    const SmallVector<custom_flag::CustomFlagDeclarationPtr> &CustomFlagDecls) {
+  struct EditDistanceInfo {
+    StringRef FlagValue;
+    unsigned EditDistance;
+  };
+  const unsigned MaxEditDistance = 5;
+
+  for (StringRef Unclaimed : UnclaimedCustomFlagValues) {
+    std::optional<EditDistanceInfo> BestCandidate;
+    for (const auto &Decl : CustomFlagDecls) {
+      for (const auto &Value : Decl->ValueList) {
+        const std::string &FlagValueName = Value.Name;
+        unsigned EditDistance =
+            Unclaimed.edit_distance(FlagValueName, /*AllowReplacements=*/true,
+                                    /*MaxEditDistance=*/MaxEditDistance);
+        if (!BestCandidate || (EditDistance <= MaxEditDistance &&
+                               EditDistance < BestCandidate->EditDistance)) {
+          BestCandidate = {FlagValueName, EditDistance};
+        }
+      }
+    }
+    if (!BestCandidate)
+      D.Diag(clang::diag::warn_drv_unsupported_opt)
+          << (custom_flag::Prefix + Unclaimed).str();
+    else
+      D.Diag(clang::diag::warn_drv_unsupported_opt_with_suggestion)
+          << (custom_flag::Prefix + Unclaimed).str()
+          << (custom_flag::Prefix + BestCandidate->FlagValue).str();
+  }
+}
+
+namespace clang::driver::custom_flag {
+class ValueNameToDetailMap {
+  SmallVector<std::pair<StringRef, const CustomFlagValueDetail *>> Mapping;
+
+public:
+  template <typename It>
+  ValueNameToDetailMap(It FlagDeclsBegin, It FlagDeclsEnd) {
+    for (auto DeclIt = FlagDeclsBegin; DeclIt != FlagDeclsEnd; ++DeclIt) {
+      const CustomFlagDeclarationPtr &Decl = *DeclIt;
+      for (const auto &Value : Decl->ValueList)
+        Mapping.emplace_back(Value.Name, &Value);
+    }
+  }
+
+  const CustomFlagValueDetail *get(StringRef Key) const {
+    auto Iter = llvm::find_if(
+        Mapping, [&](const auto &Pair) { return Pair.first == Key; });
+    return Iter != Mapping.end() ? Iter->second : nullptr;
+  }
+};
+} // namespace clang::driver::custom_flag
+
+Multilib::flags_list
+MultilibSet::processCustomFlags(const Driver &D,
+                                const Multilib::flags_list &Flags) const {
+  Multilib::flags_list Result;
+  SmallVector<const custom_flag::CustomFlagValueDetail *>
+      ClaimedCustomFlagValues;
+  SmallVector<StringRef> UnclaimedCustomFlagValueStrs;
+
+  const auto ValueNameToValueDetail = custom_flag::ValueNameToDetailMap(
+      CustomFlagDecls.begin(), CustomFlagDecls.end());
+
+  for (StringRef Flag : Flags) {
+    if (!Flag.starts_with(custom_flag::Prefix)) {
+      Result.push_back(Flag.str());
+      continue;
+    }
+
+    StringRef CustomFlagValueStr = Flag.substr(custom_flag::Prefix.size());
+    const custom_flag::CustomFlagValueDetail *Detail =
+        ValueNameToValueDetail.get(CustomFlagValueStr);
+    if (Detail)
+      ClaimedCustomFlagValues.push_back(Detail);
+    else
+      UnclaimedCustomFlagValueStrs.push_back(CustomFlagValueStr);
+  }
+
+  llvm::SmallSet<custom_flag::CustomFlagDeclarationPtr, 32>
+      TriggeredCustomFlagDecls;
+
+  for (auto *CustomFlagValue : llvm::reverse(ClaimedCustomFlagValues)) {
+    if (!TriggeredCustomFlagDecls.insert(CustomFlagValue->Decl).second)
+      continue;
+    Result.push_back(std::string(custom_flag::Prefix) + CustomFlagValue->Name);
+  }
+
+  for (const auto &Decl : CustomFlagDecls) {
+    if (TriggeredCustomFlagDecls.contains(Decl))
+      continue;
+    Result.push_back(std::string(custom_flag::Prefix) +
+                     Decl->ValueList[Decl->DefaultValueIdx].Name);
+  }
+
+  WarnUnclaimedMultilibCustomFlags(D, UnclaimedCustomFlagValueStrs,
+                                   CustomFlagDecls);
+
+  return Result;
+}
+
 bool MultilibSet::select(const Driver &D, const Multilib::flags_list &Flags,
                          llvm::SmallVectorImpl<Multilib> &Selected) const {
-  llvm::StringSet<> FlagSet(expandFlags(Flags));
+  Multilib::flags_list FlagsWithCustom = processCustomFlags(D, Flags);
+  llvm::StringSet<> FlagSet(expandFlags(FlagsWithCustom));
   Selected.clear();
   bool AnyErrors = false;
 
@@ -393,8 +498,34 @@ LLVM_DUMP_METHOD void MultilibSet::dump() const {
 }
 
 void MultilibSet::print(raw_ostream &OS) const {
-  for (const auto &M : *this)
-    OS << M << "\n";
+  const auto ValueNameToValueDetail = custom_flag::ValueNameToDetailMap(
+      CustomFlagDecls.begin(), CustomFlagDecls.end());
+
+  for (const auto &M : *this) {
+    if (M.isError())
+      continue;
+    llvm::SmallString<128> Buf;
+    llvm::raw_svector_ostream StrOS(Buf);
+
+    M.print(StrOS);
+
+    for (StringRef Flag : M.flags()) {
+      if (!Flag.starts_with(custom_flag::Prefix))
+        continue;
+
+      StringRef CustomFlagValueStr = Flag.substr(custom_flag::Prefix.size());
+      const custom_flag::CustomFlagValueDetail *Detail =
+          ValueNameToValueDetail.get(CustomFlagValueStr);
+
+      if (!Detail || !Detail->ExtraBuildArgs)
+        continue;
+
+      for (StringRef Arg : *Detail->ExtraBuildArgs)
+        StrOS << '@' << (Arg[0] == '-' ? Arg.substr(1) : Arg);
+    }
+
+    OS << StrOS.str() << '\n';
+  }
 }
 
 raw_ostream &clang::driver::operator<<(raw_ostream &OS, const MultilibSet &MS) {
diff --git a/clang/test/Driver/baremetal-multilib-custom-flags.yaml b/clang/test/Driver/baremetal-multilib-custom-flags.yaml
new file mode 100644
index 00000000000000..3a19951c741247
--- /dev/null
+++ b/clang/test/Driver/baremetal-multilib-custom-flags.yaml
@@ -0,0 +1,57 @@
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-DEFAULT %s
+
+# CHECK-DEFAULT:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-DEFAULT-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/include"
+# CHECK-DEFAULT-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=no-multithreaded --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-NOMULTI %s
+
+# CHECK-NOMULTI:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-NOMULTI-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/include"
+# CHECK-NOMULTI-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=multithreaded --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-MULTI %s
+
+# CHECK-MULTI:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-MULTI-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/multithreaded/thumb/v8-m.main/nofp/include"
+# CHECK-MULTI-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/multithreaded/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=singlethreaded -fmultilib-flag=no-io --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-WARNING %s
+# CHECK-WARNING-DAG: warning: unsupported option '-fmultilib-flag=singlethreaded'
+# CHECK-WARNING-DAG: warning: unsupported option '-fmultilib-flag=no-io'; did you mean '-fmultilib-flag=io-none'?
+
+---
+MultilibVersion: 1.0
+
+Groups:
+- Name: stdlib
+  Type: Exclusive
+
+Variants:
+- Dir: arm-none-eabi/thumb/v8-m.main/nofp
+  Flags: [--target=thumbv8m.main-unknown-none-eabi, -mfpu=none, -fmultilib-flag=no-multithreaded]
+  Group: stdlib
+- Dir: arm-none-eabi/multithreaded/thumb/v8-m.main/nofp
+  Flags: [--target=thumbv8m.main-unknown-none-eabi, -mfpu=none, -fmultilib-flag=multithreaded]
+  Group: stdlib
+
+Flags:
+  - Name: multithreading
+    Values:
+    - Name: no-multithreaded
+    - Name: multithreaded
+    Default: no-multithreaded
+  - Name: io
+    Values:
+    - Name: io-none
+    - Name: io-semihosting
+    - Name: io-linux-syscalls
+    Default: io-none
\ No newline at end of file

llvmbot · 2024-10-24T09:18:06Z

@llvm/pr-subscribers-clang

Author: Victor Campos (vhscampos)

Changes

Select library variants in the multilib system using the flags passed in the '-fmultilib-flag=' format.

Multilib flags that were not passed in the command-line have their default value fed into the library selection mechanism.

A warning is shown if the flag's value is invalid. The closest valid value is suggested (string edit distance).

Details about this change can be found in this thread: https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

Full diff: https://github.com/llvm/llvm-project/pull/110659.diff

5 Files Affected:

(modified) clang/include/clang/Basic/DiagnosticDriverKinds.td (+6)
(modified) clang/include/clang/Driver/Multilib.h (+3)
(modified) clang/lib/Driver/Driver.cpp (+1-3)
(modified) clang/lib/Driver/Multilib.cpp (+134-3)
(added) clang/test/Driver/baremetal-multilib-custom-flags.yaml (+57)

diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index 65551bd7761a9d..6874614557f837 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -14,6 +14,12 @@ def err_drv_no_such_file_with_suggestion : Error<
 def err_drv_unsupported_opt : Error<"unsupported option '%0'">;
 def err_drv_unsupported_opt_with_suggestion : Error<
   "unsupported option '%0'; did you mean '%1'?">;
+def warn_drv_unsupported_opt : Warning<
+  "unsupported option '%0'">,
+  InGroup<UnusedCommandLineArgument>;
+def warn_drv_unsupported_opt_with_suggestion : Warning<
+  "unsupported option '%0'; did you mean '%1'?">,
+  InGroup<UnusedCommandLineArgument>;
 def err_drv_unsupported_opt_for_target : Error<
   "unsupported option '%0' for target '%1'">;
 def err_drv_unsupported_opt_for_language_mode : Error<
diff --git a/clang/include/clang/Driver/Multilib.h b/clang/include/clang/Driver/Multilib.h
index 333b1d2b555bd9..f99d4e6313ff26 100644
--- a/clang/include/clang/Driver/Multilib.h
+++ b/clang/include/clang/Driver/Multilib.h
@@ -163,6 +163,9 @@ class MultilibSet {
   const_iterator begin() const { return Multilibs.begin(); }
   const_iterator end() const { return Multilibs.end(); }
 
+  Multilib::flags_list
+  processCustomFlags(const Driver &D, const Multilib::flags_list &Flags) const;
+
   /// Select compatible variants, \returns false if none are compatible
   bool select(const Driver &D, const Multilib::flags_list &Flags,
               llvm::SmallVectorImpl<Multilib> &) const;
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index 9878a9dad78d40..cee10d36070616 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -2324,9 +2324,7 @@ bool Driver::HandleImmediateArgs(Compilation &C) {
   }
 
   if (C.getArgs().hasArg(options::OPT_print_multi_lib)) {
-    for (const Multilib &Multilib : TC.getMultilibs())
-      if (!Multilib.isError())
-        llvm::outs() << Multilib << "\n";
+    llvm::outs() << TC.getMultilibs();
     return false;
   }
 
diff --git a/clang/lib/Driver/Multilib.cpp b/clang/lib/Driver/Multilib.cpp
index 81fe97517b0f91..a4a4a74a9c7ad4 100644
--- a/clang/lib/Driver/Multilib.cpp
+++ b/clang/lib/Driver/Multilib.cpp
@@ -12,6 +12,7 @@
 #include "clang/Driver/Driver.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/SmallSet.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Error.h"
@@ -95,9 +96,113 @@ MultilibSet &MultilibSet::FilterOut(FilterCallback F) {
 
 void MultilibSet::push_back(const Multilib &M) { Multilibs.push_back(M); }
 
+static void WarnUnclaimedMultilibCustomFlags(
+    const Driver &D, const SmallVector<StringRef> &UnclaimedCustomFlagValues,
+    const SmallVector<custom_flag::CustomFlagDeclarationPtr> &CustomFlagDecls) {
+  struct EditDistanceInfo {
+    StringRef FlagValue;
+    unsigned EditDistance;
+  };
+  const unsigned MaxEditDistance = 5;
+
+  for (StringRef Unclaimed : UnclaimedCustomFlagValues) {
+    std::optional<EditDistanceInfo> BestCandidate;
+    for (const auto &Decl : CustomFlagDecls) {
+      for (const auto &Value : Decl->ValueList) {
+        const std::string &FlagValueName = Value.Name;
+        unsigned EditDistance =
+            Unclaimed.edit_distance(FlagValueName, /*AllowReplacements=*/true,
+                                    /*MaxEditDistance=*/MaxEditDistance);
+        if (!BestCandidate || (EditDistance <= MaxEditDistance &&
+                               EditDistance < BestCandidate->EditDistance)) {
+          BestCandidate = {FlagValueName, EditDistance};
+        }
+      }
+    }
+    if (!BestCandidate)
+      D.Diag(clang::diag::warn_drv_unsupported_opt)
+          << (custom_flag::Prefix + Unclaimed).str();
+    else
+      D.Diag(clang::diag::warn_drv_unsupported_opt_with_suggestion)
+          << (custom_flag::Prefix + Unclaimed).str()
+          << (custom_flag::Prefix + BestCandidate->FlagValue).str();
+  }
+}
+
+namespace clang::driver::custom_flag {
+class ValueNameToDetailMap {
+  SmallVector<std::pair<StringRef, const CustomFlagValueDetail *>> Mapping;
+
+public:
+  template <typename It>
+  ValueNameToDetailMap(It FlagDeclsBegin, It FlagDeclsEnd) {
+    for (auto DeclIt = FlagDeclsBegin; DeclIt != FlagDeclsEnd; ++DeclIt) {
+      const CustomFlagDeclarationPtr &Decl = *DeclIt;
+      for (const auto &Value : Decl->ValueList)
+        Mapping.emplace_back(Value.Name, &Value);
+    }
+  }
+
+  const CustomFlagValueDetail *get(StringRef Key) const {
+    auto Iter = llvm::find_if(
+        Mapping, [&](const auto &Pair) { return Pair.first == Key; });
+    return Iter != Mapping.end() ? Iter->second : nullptr;
+  }
+};
+} // namespace clang::driver::custom_flag
+
+Multilib::flags_list
+MultilibSet::processCustomFlags(const Driver &D,
+                                const Multilib::flags_list &Flags) const {
+  Multilib::flags_list Result;
+  SmallVector<const custom_flag::CustomFlagValueDetail *>
+      ClaimedCustomFlagValues;
+  SmallVector<StringRef> UnclaimedCustomFlagValueStrs;
+
+  const auto ValueNameToValueDetail = custom_flag::ValueNameToDetailMap(
+      CustomFlagDecls.begin(), CustomFlagDecls.end());
+
+  for (StringRef Flag : Flags) {
+    if (!Flag.starts_with(custom_flag::Prefix)) {
+      Result.push_back(Flag.str());
+      continue;
+    }
+
+    StringRef CustomFlagValueStr = Flag.substr(custom_flag::Prefix.size());
+    const custom_flag::CustomFlagValueDetail *Detail =
+        ValueNameToValueDetail.get(CustomFlagValueStr);
+    if (Detail)
+      ClaimedCustomFlagValues.push_back(Detail);
+    else
+      UnclaimedCustomFlagValueStrs.push_back(CustomFlagValueStr);
+  }
+
+  llvm::SmallSet<custom_flag::CustomFlagDeclarationPtr, 32>
+      TriggeredCustomFlagDecls;
+
+  for (auto *CustomFlagValue : llvm::reverse(ClaimedCustomFlagValues)) {
+    if (!TriggeredCustomFlagDecls.insert(CustomFlagValue->Decl).second)
+      continue;
+    Result.push_back(std::string(custom_flag::Prefix) + CustomFlagValue->Name);
+  }
+
+  for (const auto &Decl : CustomFlagDecls) {
+    if (TriggeredCustomFlagDecls.contains(Decl))
+      continue;
+    Result.push_back(std::string(custom_flag::Prefix) +
+                     Decl->ValueList[Decl->DefaultValueIdx].Name);
+  }
+
+  WarnUnclaimedMultilibCustomFlags(D, UnclaimedCustomFlagValueStrs,
+                                   CustomFlagDecls);
+
+  return Result;
+}
+
 bool MultilibSet::select(const Driver &D, const Multilib::flags_list &Flags,
                          llvm::SmallVectorImpl<Multilib> &Selected) const {
-  llvm::StringSet<> FlagSet(expandFlags(Flags));
+  Multilib::flags_list FlagsWithCustom = processCustomFlags(D, Flags);
+  llvm::StringSet<> FlagSet(expandFlags(FlagsWithCustom));
   Selected.clear();
   bool AnyErrors = false;
 
@@ -393,8 +498,34 @@ LLVM_DUMP_METHOD void MultilibSet::dump() const {
 }
 
 void MultilibSet::print(raw_ostream &OS) const {
-  for (const auto &M : *this)
-    OS << M << "\n";
+  const auto ValueNameToValueDetail = custom_flag::ValueNameToDetailMap(
+      CustomFlagDecls.begin(), CustomFlagDecls.end());
+
+  for (const auto &M : *this) {
+    if (M.isError())
+      continue;
+    llvm::SmallString<128> Buf;
+    llvm::raw_svector_ostream StrOS(Buf);
+
+    M.print(StrOS);
+
+    for (StringRef Flag : M.flags()) {
+      if (!Flag.starts_with(custom_flag::Prefix))
+        continue;
+
+      StringRef CustomFlagValueStr = Flag.substr(custom_flag::Prefix.size());
+      const custom_flag::CustomFlagValueDetail *Detail =
+          ValueNameToValueDetail.get(CustomFlagValueStr);
+
+      if (!Detail || !Detail->ExtraBuildArgs)
+        continue;
+
+      for (StringRef Arg : *Detail->ExtraBuildArgs)
+        StrOS << '@' << (Arg[0] == '-' ? Arg.substr(1) : Arg);
+    }
+
+    OS << StrOS.str() << '\n';
+  }
 }
 
 raw_ostream &clang::driver::operator<<(raw_ostream &OS, const MultilibSet &MS) {
diff --git a/clang/test/Driver/baremetal-multilib-custom-flags.yaml b/clang/test/Driver/baremetal-multilib-custom-flags.yaml
new file mode 100644
index 00000000000000..3a19951c741247
--- /dev/null
+++ b/clang/test/Driver/baremetal-multilib-custom-flags.yaml
@@ -0,0 +1,57 @@
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-DEFAULT %s
+
+# CHECK-DEFAULT:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-DEFAULT-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/include"
+# CHECK-DEFAULT-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=no-multithreaded --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-NOMULTI %s
+
+# CHECK-NOMULTI:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-NOMULTI-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/include"
+# CHECK-NOMULTI-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=multithreaded --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-MULTI %s
+
+# CHECK-MULTI:      "-cc1" "-triple" "thumbv8m.main-unknown-none-eabi"
+# CHECK-MULTI-SAME: "-internal-isystem" "[[SYSROOT:[^"]*]]/bin/../lib/clang-runtimes/arm-none-eabi/multithreaded/thumb/v8-m.main/nofp/include"
+# CHECK-MULTI-NEXT: "-L[[SYSROOT]]/bin/../lib/clang-runtimes/arm-none-eabi/multithreaded/thumb/v8-m.main/nofp/lib"
+
+# RUN: %clang --multi-lib-config=%s -no-canonical-prefixes -x c %s -### -o /dev/null 2>&1 \
+# RUN:     --target=thumbv8m.main-none-eabi -mfpu=none -fmultilib-flag=singlethreaded -fmultilib-flag=no-io --sysroot= \
+# RUN:   | FileCheck --check-prefix=CHECK-WARNING %s
+# CHECK-WARNING-DAG: warning: unsupported option '-fmultilib-flag=singlethreaded'
+# CHECK-WARNING-DAG: warning: unsupported option '-fmultilib-flag=no-io'; did you mean '-fmultilib-flag=io-none'?
+
+---
+MultilibVersion: 1.0
+
+Groups:
+- Name: stdlib
+  Type: Exclusive
+
+Variants:
+- Dir: arm-none-eabi/thumb/v8-m.main/nofp
+  Flags: [--target=thumbv8m.main-unknown-none-eabi, -mfpu=none, -fmultilib-flag=no-multithreaded]
+  Group: stdlib
+- Dir: arm-none-eabi/multithreaded/thumb/v8-m.main/nofp
+  Flags: [--target=thumbv8m.main-unknown-none-eabi, -mfpu=none, -fmultilib-flag=multithreaded]
+  Group: stdlib
+
+Flags:
+  - Name: multithreading
+    Values:
+    - Name: no-multithreaded
+    - Name: multithreaded
+    Default: no-multithreaded
+  - Name: io
+    Values:
+    - Name: io-none
+    - Name: io-semihosting
+    - Name: io-linux-syscalls
+    Default: io-none
\ No newline at end of file

clang/include/clang/Basic/DiagnosticDriverKinds.td

clang/lib/Driver/Multilib.cpp

vhscampos · 2024-11-05T15:13:19Z

FYI I added a test for -print-multi-lib and its interaction with ExtraBuildArgs.

github-actions · 2024-11-20T14:51:10Z

✅ With the latest revision this PR passed the C/C++ code formatter.

lenary

LGTM, with one tiny nit.

clang/lib/Driver/Multilib.cpp

This patch is the first step to extend the current multilib system to support the selection of library variants which do not correspond to existing command-line options. Proposal can be found in https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058 The multilib mechanism supports libraries that target code generation or language options such as `--target`, `-mcpu`, `-mfpu`, `-mbranch-protection`. However, some library variants are particular to features that do not correspond to any command-line options. Examples include variants for multithreading and semihosting. This work introduces a way to instruct the multilib system to consider these features in library selection. This particular patch comprises a new section in `multilib.yaml` to declare flags for which no option exists. Henceforth this sort of flag will be called `custom flag` for clarity. The `multilib.yaml` file will have a new section called Flags which contains the declarations of the target’s custom flags: ```yaml Flags: - Name: multithreaded Values: - Name: no-multithreaded MacroDefines: [__SINGLE_THREAD__] - Name: multithreaded Default: no-multithreaded - Name: io Values: - Name: io-none - Name: io-semihosting MacroDefines: [SEMIHOSTING] - Name: io-linux-syscalls MacroDefines: [LINUX_SYSCALLS, HOSTED=1] Default: io-none ``` - Name: the name to categorize a flag. - Values: a list of possible values. - Default: it specifies which value this flag should take if not specified in the command-line invocation. It must be one value from the Values field. Each flag Value follows this description: - Name (required): the name of the custom flag value (string). This is the string to be used in `-fmultilib-flag=<string>`. - MacroDefines (optional): a list of strings to be used as macro definitions. Each string is fed into the driver as ``-D<string>``. A Default value is useful to save users from specifying custom flags that have a most commonly used value. The namespace of flag values is common across all flags. This means that flag values must be unique.

Select library variants in the multilib system using the flags passed following the '-fmultilib-flag=' format. Multilib flags that were not passed in the command-line have their default value fed into the library selection mechanism. A warning is shown if the flag's value name is invalid. If the wrong name is close enough to any valid one, according to edit distance, the closest valid value name is suggested. Details about this change can be found in this thread: https://discourse.llvm.org/t/rfc-multilib-custom-flags/81058

The expected output is reliant on the syntax of file hierarchy. This varies significantly between Linux and Windows. Following what other multilib tests do, I am disabling the test on Windows.

vhscampos force-pushed the users/vhscampos/multilib-flags-cmd-line-option branch from 3e06d0b to 0020798 Compare October 1, 2024 13:30

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 87b7902 to 1d394d0 Compare October 1, 2024 13:32

vhscampos force-pushed the users/vhscampos/multilib-flags-cmd-line-option branch from 0020798 to 829fad5 Compare October 24, 2024 09:15

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 1d394d0 to 838993d Compare October 24, 2024 09:16

vhscampos marked this pull request as ready for review October 24, 2024 09:17

llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Oct 24, 2024

statham-arm reviewed Oct 29, 2024

View reviewed changes

clang/include/clang/Basic/DiagnosticDriverKinds.td Outdated Show resolved Hide resolved

clang/lib/Driver/Multilib.cpp Show resolved Hide resolved

vhscampos force-pushed the users/vhscampos/multilib-flags-cmd-line-option branch from 829fad5 to ee2d268 Compare November 4, 2024 13:52

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 838993d to b5af73a Compare November 4, 2024 15:41

statham-arm approved these changes Nov 4, 2024

View reviewed changes

clang/lib/Driver/Multilib.cpp Show resolved Hide resolved

vhscampos force-pushed the users/vhscampos/multilib-flags-cmd-line-option branch from ee2d268 to 7b8f0d9 Compare November 20, 2024 14:47

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 0b76e71 to b2de258 Compare November 20, 2024 14:47

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from f6ff48d to 3bc43d3 Compare December 10, 2024 12:51

vhscampos force-pushed the users/vhscampos/multilib-flags-cmd-line-option branch 2 times, most recently from c0bb75d to 17265b1 Compare December 11, 2024 13:30

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 3bc43d3 to 414c55e Compare December 11, 2024 13:30

lenary approved these changes Jan 6, 2025

View reviewed changes

clang/lib/Driver/Multilib.cpp Show resolved Hide resolved

Base automatically changed from users/vhscampos/multilib-flags-cmd-line-option to main January 13, 2025 13:53

vhscampos added 6 commits January 14, 2025 11:28

Fix clang-format errors

131a8ca

Disable test on Windows

403ee8c

The expected output is reliant on the syntax of file hierarchy. This varies significantly between Linux and Windows. Following what other multilib tests do, I am disabling the test on Windows.

Repurpose DriverArgs to MacroDefines

aa46b7b

Add clarifying comment

46b830b

vhscampos added 2 commits January 14, 2025 11:29

Fix clang-format error

a921766

Rewrite code for the removal of shared_ptr use

71dd4c2

vhscampos force-pushed the users/vhscampos/multilib-flags-processing branch from 06a5187 to 71dd4c2 Compare January 14, 2025 14:09

vhscampos changed the base branch from main to users/vhscampos/multilib-flags-yaml-parsing January 14, 2025 14:12

Base automatically changed from users/vhscampos/multilib-flags-yaml-parsing to main January 15, 2025 10:11

Merge branch 'main' into users/vhscampos/multilib-flags-processing

70ffd62

vhscampos merged commit 3a9380f into main Jan 16, 2025
8 checks passed

vhscampos deleted the users/vhscampos/multilib-flags-processing branch January 16, 2025 09:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Multilib] Custom flags processing for library selection #110659

[Multilib] Custom flags processing for library selection #110659

Uh oh!

vhscampos commented Oct 1, 2024 •

edited

Loading

Uh oh!

llvmbot commented Oct 24, 2024

Uh oh!

llvmbot commented Oct 24, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vhscampos commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 20, 2024 •

edited

Loading

Uh oh!

lenary left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[Multilib] Custom flags processing for library selection #110659

[Multilib] Custom flags processing for library selection #110659

Uh oh!

Conversation

vhscampos commented Oct 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Oct 24, 2024

Uh oh!

llvmbot commented Oct 24, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vhscampos commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lenary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vhscampos commented Oct 1, 2024 •

edited

Loading

github-actions bot commented Nov 20, 2024 •

edited

Loading