Skip to content

[llvm-debuginfod-find] Enable multicall driver #108082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 11, 2024

Conversation

Prabhuk
Copy link
Contributor

@Prabhuk Prabhuk commented Sep 10, 2024

Migrate llvm-debuginfod-find tool to use GenericOptTable.
Enable multicall driver.

@llvmbot
Copy link
Member

llvmbot commented Sep 10, 2024

@llvm/pr-subscribers-debuginfo

Author: Prabhuk (Prabhuk)

Changes

Migrate llvm-debuginfod-find tool to use GenericOptTable.
Enable multicall driver.


Full diff: https://github.com/llvm/llvm-project/pull/108082.diff

3 Files Affected:

  • (modified) llvm/tools/llvm-debuginfod-find/CMakeLists.txt (+11-1)
  • (added) llvm/tools/llvm-debuginfod-find/Opts.td (+16)
  • (modified) llvm/tools/llvm-debuginfod-find/llvm-debuginfod-find.cpp (+87-22)
diff --git a/llvm/tools/llvm-debuginfod-find/CMakeLists.txt b/llvm/tools/llvm-debuginfod-find/CMakeLists.txt
index b98c431c1839bb..39da11fcd95990 100644
--- a/llvm/tools/llvm-debuginfod-find/CMakeLists.txt
+++ b/llvm/tools/llvm-debuginfod-find/CMakeLists.txt
@@ -1,11 +1,21 @@
 set(LLVM_LINK_COMPONENTS
+  Option
   Object
   Support
   )
+set(LLVM_TARGET_DEFINITIONS Opts.td)
+tablegen(LLVM Opts.inc -gen-opt-parser-defs)
+add_public_tablegen_target(DebugInfodFindOptsTableGen)
+
 add_llvm_tool(llvm-debuginfod-find
   llvm-debuginfod-find.cpp
+  DEPENDS
+  DebugInfodFindOptsTableGen
+  GENERATE_DRIVER
   )
-target_link_libraries(llvm-debuginfod-find PRIVATE LLVMDebuginfod)
+if(NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
+  target_link_libraries(llvm-debuginfod-find PRIVATE LLVMDebuginfod)
+endif()
 if(LLVM_INSTALL_BINUTILS_SYMLINKS)
   add_llvm_tool_symlink(debuginfod-find llvm-debuginfod-find)
 endif()
diff --git a/llvm/tools/llvm-debuginfod-find/Opts.td b/llvm/tools/llvm-debuginfod-find/Opts.td
new file mode 100644
index 00000000000000..8341fd313930dc
--- /dev/null
+++ b/llvm/tools/llvm-debuginfod-find/Opts.td
@@ -0,0 +1,16 @@
+include "llvm/Option/OptParser.td"
+
+class F<string name, string help> : Flag<["-"], name>, HelpText<help>;
+class FF<string name, string help>: Flag<["--"], name>, HelpText<help>;
+class S<string name, string meta, string help>: Separate<["--"], name>, HelpText<help>, MetaVarName<meta>;
+
+def help : FF<"help", "Display available options">;
+def : F<"h", "Alias for --help">, Alias<help>;
+
+def fetch_executable : FF<"executable", "If set, fetch a binary file associated with this build id, containing the executable sections.">;
+def fetch_debuginfo : FF<"debuginfo", "If set, fetch a binary file associated with this build id, containing the debuginfo sections.">;
+def fetch_source : S<"source", "<string>", "Fetch a source file associated with this build id, which is at this relative path relative to the compilation directory.">;
+def dump_to_stdout : FF<"dump", "If set, dumps the contents of the fetched artifact "
+                          "to standard output. Otherwise, dumps the absolute "
+                          "path to the cached artifact on disk.">;
+def debug_file_directory : S<"debug-file-directory", "<string>", "Path to directory where to look for debug files.">;
\ No newline at end of file
diff --git a/llvm/tools/llvm-debuginfod-find/llvm-debuginfod-find.cpp b/llvm/tools/llvm-debuginfod-find/llvm-debuginfod-find.cpp
index 425ee8d986a829..1f4404aaa391fc 100644
--- a/llvm/tools/llvm-debuginfod-find/llvm-debuginfod-find.cpp
+++ b/llvm/tools/llvm-debuginfod-find/llvm-debuginfod-find.cpp
@@ -16,14 +16,89 @@
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringRef.h"
 #include "llvm/Debuginfod/BuildIDFetcher.h"
 #include "llvm/Debuginfod/Debuginfod.h"
 #include "llvm/Debuginfod/HTTPClient.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/Option.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/LLVMDriver.h"
 
 using namespace llvm;
 
+// Command-line option boilerplate.
+namespace {
+enum ID {
+  OPT_INVALID = 0, // This is not an option ID.
+#define OPTION(...) LLVM_MAKE_OPT_ID(__VA_ARGS__),
+#include "Opts.inc"
+#undef OPTION
+};
+
+#define PREFIX(NAME, VALUE)                                                    \
+  static constexpr StringLiteral NAME##_init[] = VALUE;                        \
+  static constexpr ArrayRef<StringLiteral> NAME(NAME##_init,                   \
+                                                std::size(NAME##_init) - 1);
+#include "Opts.inc"
+#undef PREFIX
+
+using namespace llvm::opt;
+static constexpr opt::OptTable::Info InfoTable[] = {
+#define OPTION(...) LLVM_CONSTRUCT_OPT_INFO(__VA_ARGS__),
+#include "Opts.inc"
+#undef OPTION
+};
+
+class DebuginfodFindOptTable : public opt::GenericOptTable {
+public:
+  DebuginfodFindOptTable() : GenericOptTable(InfoTable) {}
+};
+
+} // end anonymous namespace
+
+static std::string InputBuildID;
+static bool FetchExecutable;
+static bool FetchDebuginfo;
+static std::string FetchSource;
+static bool DumpToStdout;
+static std::vector<std::string> DebugFileDirectory;
+
+static void parseArgs(int argc, char **argv) {
+  DebuginfodFindOptTable Tbl;
+  llvm::StringRef ToolName = argv[0];
+  llvm::BumpPtrAllocator A;
+  llvm::StringSaver Saver{A};
+  opt::InputArgList Args =
+      Tbl.parseArgs(argc, argv, OPT_UNKNOWN, Saver, [&](StringRef Msg) {
+        llvm::errs() << Msg << '\n';
+        std::exit(1);
+      });
+
+  if (Args.hasArg(OPT_help)) {
+    Tbl.printHelp(llvm::outs(),
+                  "llvm-debuginfod-find [options] <input build_id>",
+                  ToolName.str().c_str());
+    std::exit(0);
+  }
+
+  InputBuildID = Args.getLastArgValue(OPT_INPUT);
+
+  FetchExecutable = Args.hasArg(OPT_fetch_executable);
+  FetchDebuginfo = Args.hasArg(OPT_fetch_debuginfo);
+  DumpToStdout = Args.hasArg(OPT_dump_to_stdout);
+  FetchSource = Args.getLastArgValue(OPT_fetch_source, "");
+  DebugFileDirectory = Args.getAllArgValues(OPT_debug_file_directory);
+}
+
+[[noreturn]] static void helpExit() {
+  errs() << "Must specify exactly one of --executable, "
+            "--source=/path/to/file, or --debuginfo.\n";
+  exit(1);
+}
+
+/*
 cl::OptionCategory DebuginfodFindCategory("llvm-debuginfod-find Options");
 
 cl::opt<std::string> InputBuildID(cl::Positional, cl::Required,
@@ -60,30 +135,17 @@ static cl::list<std::string> DebugFileDirectory(
     cl::desc("Path to directory where to look for debug files."),
     cl::cat(DebuginfodFindCategory));
 
-[[noreturn]] static void helpExit() {
-  errs() << "Must specify exactly one of --executable, "
-            "--source=/path/to/file, or --debuginfo.";
-  exit(1);
-}
+*/
 
-ExitOnError ExitOnErr;
+ExitOnError ExitOnDebuginfodFindError;
 
 static std::string fetchDebugInfo(object::BuildIDRef BuildID);
 
-int main(int argc, char **argv) {
-  InitLLVM X(argc, argv);
+int llvm_debuginfod_find_main(int argc, char **argv,
+                              const llvm::ToolContext &) {
+  // InitLLVM X(argc, argv);
   HTTPClient::initialize();
-
-  cl::HideUnrelatedOptions({&DebuginfodFindCategory});
-  cl::ParseCommandLineOptions(
-      argc, argv,
-      "llvm-debuginfod-find: Fetch debuginfod artifacts\n\n"
-      "This program is a frontend to the debuginfod client library. The cache "
-      "directory, request timeout (in seconds), and debuginfod server urls are "
-      "set by these environment variables:\n"
-      "DEBUGINFOD_CACHE_PATH (default set by sys::path::cache_directory)\n"
-      "DEBUGINFOD_TIMEOUT (defaults to 90s)\n"
-      "DEBUGINFOD_URLS=[comma separated URLs] (defaults to empty)\n");
+  parseArgs(argc, argv);
 
   if (FetchExecutable + FetchDebuginfo + (FetchSource != "") != 1)
     helpExit();
@@ -97,9 +159,10 @@ int main(int argc, char **argv) {
 
   std::string Path;
   if (FetchSource != "")
-    Path = ExitOnErr(getCachedOrDownloadSource(ID, FetchSource));
+    Path =
+        ExitOnDebuginfodFindError(getCachedOrDownloadSource(ID, FetchSource));
   else if (FetchExecutable)
-    Path = ExitOnErr(getCachedOrDownloadExecutable(ID));
+    Path = ExitOnDebuginfodFindError(getCachedOrDownloadExecutable(ID));
   else if (FetchDebuginfo)
     Path = fetchDebugInfo(ID);
   else
@@ -110,11 +173,13 @@ int main(int argc, char **argv) {
     // Print the contents of the artifact.
     ErrorOr<std::unique_ptr<MemoryBuffer>> Buf = MemoryBuffer::getFile(
         Path, /*IsText=*/false, /*RequiresNullTerminator=*/false);
-    ExitOnErr(errorCodeToError(Buf.getError()));
+    ExitOnDebuginfodFindError(errorCodeToError(Buf.getError()));
     outs() << Buf.get()->getBuffer();
   } else
     // Print the path to the cached artifact file.
     outs() << Path << "\n";
+
+  return 0;
 }
 
 // Find a debug file in local build ID directories and via debuginfod.

@Prabhuk Prabhuk force-pushed the llvm_debuginfod_find_multicall branch from 553ff90 to 1baa7d5 Compare September 11, 2024 01:41
Migrate llvm-debuginfod-find tool to use GenericOptTable. Enable
multicall driver.
@Prabhuk Prabhuk force-pushed the llvm_debuginfod_find_multicall branch from 1baa7d5 to 8e175eb Compare September 11, 2024 01:43
@Prabhuk Prabhuk merged commit bc152fb into llvm:main Sep 11, 2024
8 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 11, 2024

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux running on sanitizer-buildbot1 while building llvm at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/66/builds/3780

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure) (timed out)
...
[377/384] Generating FuzzerUtils-x86_64-Test
[378/384] Generating FuzzerTestObjects.gtest-all.cc.x86_64.o
[379/384] Generating Fuzzer-x86_64-Test
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10234 tests, 88 workers --
command timed out: 1200 seconds without output running [b'python', b'../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=3694.064471
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Step 14 (test compiler-rt default) failure: test compiler-rt default (failure)
...
  CMakeLists.txt:6 (include)


CMake Deprecation Warning at /home/b/sanitizer-x86_64-linux/build/llvm-project/cmake/Modules/CMakePolicy.cmake:11 (cmake_policy):
  The OLD behavior for policy CMP0116 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)
-- Could NOT find LLVM (missing: LLVM_DIR)
-- Could NOT find Clang (missing: Clang_DIR)
-- LLVM host triple: x86_64-unknown-linux-gnu
-- LLVM default target triple: x86_64-unknown-linux-gnu
-- Using libc++abi testing configuration: /home/b/sanitizer-x86_64-linux/build/llvm-project/libcxxabi/test/configs/llvm-libc++abi-static.cfg.in
-- Using libc++ testing configuration: /home/b/sanitizer-x86_64-linux/build/llvm-project/libcxx/test/configs/llvm-libc++-static.cfg.in
-- check-runtimes does nothing.
-- Configuring done (0.7s)
-- Generating done (0.1s)
-- Build files have been written to: /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64
[366/384] Linking CXX static library /home/b/sanitizer-x86_64-linux/build/build_default/lib/clang/20/lib/i386-unknown-linux-gnu/libclang_rt.fuzzer_no_main.a
[367/384] Performing build step for 'libcxx_fuzzer_x86_64'
ninja: no work to do.
[369/384] Linking CXX static library /home/b/sanitizer-x86_64-linux/build/build_default/lib/clang/20/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer_interceptors.a
[370/384] Linking CXX static library /home/b/sanitizer-x86_64-linux/build/build_default/lib/clang/20/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer_no_main.a
[371/384] Linking CXX static library /home/b/sanitizer-x86_64-linux/build/build_default/lib/clang/20/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer.a
[372/384] Generating FuzzedDataProviderTestObjects.FuzzedDataProviderUnittest.cpp.x86_64.o
[373/384] Generating MSAN_INST_GTEST.gtest-all.cc.x86_64-with-call.o
[374/384] Generating FuzzerTestObjects.FuzzerUnittest.cpp.x86_64.o
[375/384] Generating MSAN_INST_GTEST.gtest-all.cc.x86_64.o
[376/384] Generating FuzzedDataProviderTestObjects.gtest-all.cc.x86_64.o
[377/384] Generating FuzzerUtils-x86_64-Test
[378/384] Generating FuzzerTestObjects.gtest-all.cc.x86_64.o
[379/384] Generating Fuzzer-x86_64-Test
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10234 tests, 88 workers --

command timed out: 1200 seconds without output running [b'python', b'../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=3694.064471
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..

@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 11, 2024

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/1125

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/43/86' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-4680-43-86.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=86 GTEST_SHARD_INDEX=43 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


@Prabhuk Prabhuk deleted the llvm_debuginfod_find_multicall branch September 19, 2024 18:56
boomanaiden154 added a commit that referenced this pull request Sep 23, 2024
This patch removes a comment in llvm-debuginfod-find containing all the
cl::opt entries, which are redundant after the conversion to using
optTable. These seem to have been introduced in #108082 along with a
conversion to optTable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants