Skip to content

[BOLT] Enable standalone build #87196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Conversation

pca006132
Copy link
Contributor

Upstream changes from NixOS/nixpkgs#289587, this enables standalone build for BOLT, which makes packaging easier. Most of the changes in bolt/CMakeLists.txt are copied from clang/CMakeLists.txt. bolt/lib/Target/*/CMakeLists.txt is also modified to compile tablegen instead of relying on files in LLVM_BINARY_DIR. bolt/lib/Utils/CMakeLists.txt is changed to generate the VCS information by itself instead of relying on llvm_vcsrevision_h, which is not available for the standalone build. These changes should not affect existing builds, but will enable standalone build.

llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h is changed to remove an (apparently) unused config.h include which prevents the standalone build from working, so far I don't encounter any compilation issue after removing that config header.

Copy link

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be
notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write
permissions for the repository. In which case you can instead tag reviewers by
name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review
by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate
is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot
Copy link
Member

llvmbot commented Mar 31, 2024

@llvm/pr-subscribers-backend-risc-v

Author: pca006132 (pca006132)

Changes

Upstream changes from NixOS/nixpkgs#289587, this enables standalone build for BOLT, which makes packaging easier. Most of the changes in bolt/CMakeLists.txt are copied from clang/CMakeLists.txt. bolt/lib/Target/*/CMakeLists.txt is also modified to compile tablegen instead of relying on files in LLVM_BINARY_DIR. bolt/lib/Utils/CMakeLists.txt is changed to generate the VCS information by itself instead of relying on llvm_vcsrevision_h, which is not available for the standalone build. These changes should not affect existing builds, but will enable standalone build.

llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h is changed to remove an (apparently) unused config.h include which prevents the standalone build from working, so far I don't encounter any compilation issue after removing that config header.


Full diff: https://github.com/llvm/llvm-project/pull/87196.diff

7 Files Affected:

  • (modified) bolt/CMakeLists.txt (+145-1)
  • (modified) bolt/lib/Target/AArch64/CMakeLists.txt (+33)
  • (modified) bolt/lib/Target/RISCV/CMakeLists.txt (+31)
  • (modified) bolt/lib/Target/X86/CMakeLists.txt (+26)
  • (modified) bolt/lib/Utils/CMakeLists.txt (+32-4)
  • (modified) bolt/lib/Utils/CommandLineOpts.cpp (+1-1)
  • (modified) llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h (-1)
diff --git a/bolt/CMakeLists.txt b/bolt/CMakeLists.txt
index cc3a70fa35e0ab..9635ec454d7756 100644
--- a/bolt/CMakeLists.txt
+++ b/bolt/CMakeLists.txt
@@ -1,4 +1,15 @@
-include(ExternalProject)
+cmake_minimum_required(VERSION 3.20.0)
+
+if(NOT DEFINED LLVM_COMMON_CMAKE_UTILS)
+  set(LLVM_COMMON_CMAKE_UTILS ${CMAKE_CURRENT_SOURCE_DIR}/../cmake)
+endif()
+include(${LLVM_COMMON_CMAKE_UTILS}/Modules/CMakePolicy.cmake
+  NO_POLICY_SCOPE)
+
+if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  project(bolt)
+  set(BOLT_BUILT_STANDALONE TRUE)
+endif()
 
 set(BOLT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 set(BOLT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
@@ -7,6 +18,137 @@ set(CMAKE_CXX_STANDARD 17)
 # Add path for custom modules.
 list(INSERT CMAKE_MODULE_PATH 0 "${BOLT_SOURCE_DIR}/cmake/modules")
 
+# standalone build, copied from clang
+if(BOLT_BUILT_STANDALONE)
+  set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
+  set(CMAKE_CXX_STANDARD_REQUIRED YES)
+  set(CMAKE_CXX_EXTENSIONS NO)
+
+  if(NOT MSVC_IDE)
+    set(LLVM_ENABLE_ASSERTIONS ${ENABLE_ASSERTIONS}
+      CACHE BOOL "Enable assertions")
+    # Assertions should follow llvm-config's.
+    mark_as_advanced(LLVM_ENABLE_ASSERTIONS)
+  endif()
+
+  find_package(LLVM REQUIRED HINTS "${LLVM_CMAKE_DIR}")
+  list(APPEND CMAKE_MODULE_PATH "${LLVM_DIR}")
+
+  # Turn into CACHE PATHs for overwritting
+
+  # We can't check LLVM_CONFIG here, because find_package(LLVM ...) also sets
+  # LLVM_CONFIG.
+  if (NOT LLVM_CONFIG_FOUND)
+    # Pull values from LLVMConfig.cmake.  We can drop this once the llvm-config
+    # path is removed.
+    set(INCLUDE_DIRS ${LLVM_INCLUDE_DIRS})
+    set(LLVM_OBJ_DIR "${LLVM_BINARY_DIR}")
+    # N.B. this is just a default value, the CACHE PATHs below can be overridden.
+    set(MAIN_SRC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../llvm")
+  else()
+    set(INCLUDE_DIRS "${LLVM_BINARY_DIR}/include" "${MAIN_INCLUDE_DIR}")
+  endif()
+
+  set(LLVM_INCLUDE_DIRS ${INCLUDE_DIRS} CACHE PATH "Path to llvm/include and any other header dirs needed")
+  set(LLVM_BINARY_DIR "${LLVM_OBJ_ROOT}" CACHE PATH "Path to LLVM build tree")
+  set(LLVM_MAIN_SRC_DIR "${MAIN_SRC_DIR}" CACHE PATH "Path to LLVM source tree")
+
+  set(LLVM_TOOLS_BINARY_DIR "${LLVM_TOOLS_BINARY_DIR}" CACHE PATH "Path to llvm/bin")
+  set(LLVM_LIBRARY_DIR "${LLVM_LIBRARY_DIR}" CACHE PATH "Path to llvm/lib")
+
+  find_program(LLVM_TABLEGEN_EXE "llvm-tblgen" ${LLVM_TOOLS_BINARY_DIR}
+    NO_DEFAULT_PATH)
+
+  # They are used as destination of target generators.
+  set(LLVM_RUNTIME_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/bin)
+  set(LLVM_LIBRARY_OUTPUT_INTDIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib${LLVM_LIBDIR_SUFFIX})
+  if(WIN32 OR CYGWIN)
+    # DLL platform -- put DLLs into bin.
+    set(LLVM_SHLIB_OUTPUT_INTDIR ${LLVM_RUNTIME_OUTPUT_INTDIR})
+  else()
+    set(LLVM_SHLIB_OUTPUT_INTDIR ${LLVM_LIBRARY_OUTPUT_INTDIR})
+  endif()
+
+  option(LLVM_INSTALL_TOOLCHAIN_ONLY
+    "Only include toolchain files in the 'install' target." OFF)
+
+  option(LLVM_FORCE_USE_OLD_TOOLCHAIN
+    "Set to ON to force using an old, unsupported host toolchain." OFF)
+  option(CLANG_ENABLE_BOOTSTRAP "Generate the clang bootstrap target" OFF)
+  option(LLVM_ENABLE_LIBXML2 "Use libxml2 if available." ON)
+
+  include(AddLLVM)
+  include(TableGen)
+  include(HandleLLVMOptions)
+  include(CheckAtomic)
+  include(GetErrcMessages)
+  include(LLVMDistributionSupport)
+
+  if (NOT DEFINED LLVM_INCLUDE_TESTS)
+    set(LLVM_INCLUDE_TESTS ON)
+  endif()
+
+  include_directories(${LLVM_INCLUDE_DIRS})
+  link_directories("${LLVM_LIBRARY_DIR}")
+
+  set( CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin )
+  set( CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX} )
+  set( CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX} )
+
+  find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION} REQUIRED
+    COMPONENTS Interpreter)
+
+  if(LLVM_INCLUDE_TESTS)
+    # Check prebuilt llvm/utils.
+    if(EXISTS ${LLVM_TOOLS_BINARY_DIR}/FileCheck${CMAKE_EXECUTABLE_SUFFIX}
+        AND EXISTS ${LLVM_TOOLS_BINARY_DIR}/count${CMAKE_EXECUTABLE_SUFFIX}
+        AND EXISTS ${LLVM_TOOLS_BINARY_DIR}/not${CMAKE_EXECUTABLE_SUFFIX})
+      set(LLVM_UTILS_PROVIDED ON)
+    endif()
+
+    # Seek installed Lit.
+    find_program(LLVM_LIT
+                 NAMES llvm-lit lit.py lit
+                 PATHS "${LLVM_MAIN_SRC_DIR}/utils/lit"
+                 DOC "Path to lit.py")
+
+    if(EXISTS ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
+      # Note: path not really used, except for checking if lit was found
+      if(EXISTS ${LLVM_MAIN_SRC_DIR}/utils/llvm-lit)
+        add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit utils/llvm-lit)
+      endif()
+      if(NOT LLVM_UTILS_PROVIDED)
+        add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/FileCheck utils/FileCheck)
+        add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/count utils/count)
+        add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/not utils/not)
+        set(LLVM_UTILS_PROVIDED ON)
+        set(CLANG_TEST_DEPS FileCheck count not)
+      endif()
+    endif()
+
+    if(LLVM_LIT)
+      # Define the default arguments to use with 'lit', and an option for the user
+      # to override.
+      set(LIT_ARGS_DEFAULT "-sv")
+      if (MSVC OR XCODE)
+        set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")
+      endif()
+      set(LLVM_LIT_ARGS "${LIT_ARGS_DEFAULT}" CACHE STRING "Default options for lit")
+
+      get_errc_messages(LLVM_LIT_ERRC_MESSAGES)
+
+      # On Win32 hosts, provide an option to specify the path to the GnuWin32 tools.
+      if( WIN32 AND NOT CYGWIN )
+        set(LLVM_LIT_TOOLS_DIR "" CACHE PATH "Path to GnuWin32 tools")
+      endif()
+    else()
+      set(LLVM_INCLUDE_TESTS OFF)
+    endif()
+
+    umbrella_lit_testsuite_begin(check-all)
+  endif() # LLVM_INCLUDE_TESTS
+endif() # standalone
+
 # Determine default set of targets to build -- the intersection of
 # those BOLT supports and those LLVM is targeting.
 set(BOLT_TARGETS_TO_BUILD_all "AArch64;X86;RISCV")
@@ -92,6 +234,8 @@ if (BOLT_ENABLE_RUNTIME)
   if(CMAKE_SYSROOT)
     list(APPEND extra_args -DCMAKE_SYSROOT=${CMAKE_SYSROOT})
   endif()
+
+  include(ExternalProject)
   ExternalProject_Add(bolt_rt
     SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/runtime"
     STAMP_DIR ${CMAKE_CURRENT_BINARY_DIR}/bolt_rt-stamps
diff --git a/bolt/lib/Target/AArch64/CMakeLists.txt b/bolt/lib/Target/AArch64/CMakeLists.txt
index be03e247aa96b1..ba7be426203bdd 100644
--- a/bolt/lib/Target/AArch64/CMakeLists.txt
+++ b/bolt/lib/Target/AArch64/CMakeLists.txt
@@ -4,6 +4,39 @@ set(LLVM_LINK_COMPONENTS
   AArch64Desc
   )
 
+if(BOLT_BUILT_STANDALONE)
+  # tablegen, copied from llvm/lib/Target/AAarch64/CMakeLists.txt
+  set(LLVM_TARGET_DEFINITIONS ${LLVM_MAIN_SRC_DIR}/lib/Target/AArch64/AArch64.td)
+  list(APPEND LLVM_TABLEGEN_FLAGS -I ${LLVM_MAIN_SRC_DIR}/lib/Target/AArch64)
+  tablegen(LLVM AArch64GenAsmMatcher.inc -gen-asm-matcher)
+  tablegen(LLVM AArch64GenAsmWriter.inc -gen-asm-writer)
+  tablegen(LLVM AArch64GenAsmWriter1.inc -gen-asm-writer -asmwriternum=1)
+  tablegen(LLVM AArch64GenCallingConv.inc -gen-callingconv)
+  tablegen(LLVM AArch64GenDAGISel.inc -gen-dag-isel)
+  tablegen(LLVM AArch64GenDisassemblerTables.inc -gen-disassembler)
+  tablegen(LLVM AArch64GenFastISel.inc -gen-fast-isel)
+  tablegen(LLVM AArch64GenGlobalISel.inc -gen-global-isel)
+  tablegen(LLVM AArch64GenO0PreLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="AArch64O0PreLegalizerCombiner")
+  tablegen(LLVM AArch64GenPreLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="AArch64PreLegalizerCombiner")
+  tablegen(LLVM AArch64GenPostLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="AArch64PostLegalizerCombiner")
+  tablegen(LLVM AArch64GenPostLegalizeGILowering.inc -gen-global-isel-combiner
+                -combiners="AArch64PostLegalizerLowering")
+  tablegen(LLVM AArch64GenInstrInfo.inc -gen-instr-info)
+  tablegen(LLVM AArch64GenMCCodeEmitter.inc -gen-emitter)
+  tablegen(LLVM AArch64GenMCPseudoLowering.inc -gen-pseudo-lowering)
+  tablegen(LLVM AArch64GenRegisterBank.inc -gen-register-bank)
+  tablegen(LLVM AArch64GenRegisterInfo.inc -gen-register-info)
+  tablegen(LLVM AArch64GenSubtargetInfo.inc -gen-subtarget)
+  tablegen(LLVM AArch64GenSystemOperands.inc -gen-searchable-tables)
+  tablegen(LLVM AArch64GenExegesis.inc -gen-exegesis)
+
+  add_public_tablegen_target(AArch64CommonTableGen)
+  include_directories(${CMAKE_CURRENT_BINARY_DIR})
+endif()
+
 add_llvm_library(LLVMBOLTTargetAArch64
   AArch64MCPlusBuilder.cpp
 
diff --git a/bolt/lib/Target/RISCV/CMakeLists.txt b/bolt/lib/Target/RISCV/CMakeLists.txt
index 7f955760632008..6016a13a1b0829 100644
--- a/bolt/lib/Target/RISCV/CMakeLists.txt
+++ b/bolt/lib/Target/RISCV/CMakeLists.txt
@@ -4,6 +4,37 @@ set(LLVM_LINK_COMPONENTS
   RISCVDesc
   )
 
+if(BOLT_BUILT_STANDALONE)
+  # tablegen, copied from llvm/lib/Target/RISCV/CMakeLists.txt
+  set(LLVM_TARGET_DEFINITIONS ${LLVM_MAIN_SRC_DIR}/lib/Target/RISCV/RISCV.td)
+  list(APPEND LLVM_TABLEGEN_FLAGS -I ${LLVM_MAIN_SRC_DIR}/lib/Target/RISCV)
+  tablegen(LLVM RISCVGenAsmMatcher.inc -gen-asm-matcher)
+  tablegen(LLVM RISCVGenAsmWriter.inc -gen-asm-writer)
+  tablegen(LLVM RISCVGenCompressInstEmitter.inc -gen-compress-inst-emitter)
+  tablegen(LLVM RISCVGenMacroFusion.inc -gen-macro-fusion-pred)
+  tablegen(LLVM RISCVGenDAGISel.inc -gen-dag-isel)
+  tablegen(LLVM RISCVGenDisassemblerTables.inc -gen-disassembler)
+  tablegen(LLVM RISCVGenInstrInfo.inc -gen-instr-info)
+  tablegen(LLVM RISCVGenMCCodeEmitter.inc -gen-emitter)
+  tablegen(LLVM RISCVGenMCPseudoLowering.inc -gen-pseudo-lowering)
+  tablegen(LLVM RISCVGenRegisterBank.inc -gen-register-bank)
+  tablegen(LLVM RISCVGenRegisterInfo.inc -gen-register-info)
+  tablegen(LLVM RISCVGenSearchableTables.inc -gen-searchable-tables)
+  tablegen(LLVM RISCVGenSubtargetInfo.inc -gen-subtarget)
+
+  set(LLVM_TARGET_DEFINITIONS ${LLVM_MAIN_SRC_DIR}/lib/Target/RISCV/RISCVGISel.td)
+  tablegen(LLVM RISCVGenGlobalISel.inc -gen-global-isel)
+  tablegen(LLVM RISCVGenO0PreLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="RISCVO0PreLegalizerCombiner")
+  tablegen(LLVM RISCVGenPreLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="RISCVPreLegalizerCombiner")
+  tablegen(LLVM RISCVGenPostLegalizeGICombiner.inc -gen-global-isel-combiner
+                -combiners="RISCVPostLegalizerCombiner")
+
+  add_public_tablegen_target(RISCVCommonTableGen)
+  include_directories(${CMAKE_CURRENT_BINARY_DIR})
+endif()
+
 add_llvm_library(LLVMBOLTTargetRISCV
   RISCVMCPlusBuilder.cpp
 
diff --git a/bolt/lib/Target/X86/CMakeLists.txt b/bolt/lib/Target/X86/CMakeLists.txt
index 2b769bc7e7f5ce..4527c454012379 100644
--- a/bolt/lib/Target/X86/CMakeLists.txt
+++ b/bolt/lib/Target/X86/CMakeLists.txt
@@ -5,6 +5,32 @@ set(LLVM_LINK_COMPONENTS
   X86Desc
   )
 
+if(BOLT_BUILT_STANDALONE)
+  # tablegen, copied from llvm/lib/Target/X86/CMakeLists.txt
+  set(LLVM_TARGET_DEFINITIONS ${LLVM_MAIN_SRC_DIR}/lib/Target/X86/X86.td)
+  list(APPEND LLVM_TABLEGEN_FLAGS -I ${LLVM_MAIN_SRC_DIR}/lib/Target/X86)
+  tablegen(LLVM X86GenAsmMatcher.inc -gen-asm-matcher)
+  tablegen(LLVM X86GenAsmWriter.inc -gen-asm-writer)
+  tablegen(LLVM X86GenAsmWriter1.inc -gen-asm-writer -asmwriternum=1)
+  tablegen(LLVM X86GenCallingConv.inc -gen-callingconv)
+  tablegen(LLVM X86GenDAGISel.inc -gen-dag-isel)
+  tablegen(LLVM X86GenDisassemblerTables.inc -gen-disassembler)
+  tablegen(LLVM X86GenCompressEVEXTables.inc -gen-x86-compress-evex-tables)
+  tablegen(LLVM X86GenExegesis.inc -gen-exegesis)
+  tablegen(LLVM X86GenFastISel.inc -gen-fast-isel)
+  tablegen(LLVM X86GenGlobalISel.inc -gen-global-isel)
+  tablegen(LLVM X86GenInstrInfo.inc -gen-instr-info
+                                    -instr-info-expand-mi-operand-info=0)
+  tablegen(LLVM X86GenMnemonicTables.inc -gen-x86-mnemonic-tables -asmwriternum=1)
+  tablegen(LLVM X86GenRegisterBank.inc -gen-register-bank)
+  tablegen(LLVM X86GenRegisterInfo.inc -gen-register-info)
+  tablegen(LLVM X86GenSubtargetInfo.inc -gen-subtarget)
+  tablegen(LLVM X86GenFoldTables.inc -gen-x86-fold-tables -asmwriternum=1)
+
+  add_public_tablegen_target(X86CommonTableGen)
+  include_directories(${CMAKE_CURRENT_BINARY_DIR})
+endif()
+
 add_llvm_library(LLVMBOLTTargetX86
   X86MCPlusBuilder.cpp
   X86MCSymbolizer.cpp
diff --git a/bolt/lib/Utils/CMakeLists.txt b/bolt/lib/Utils/CMakeLists.txt
index d1403314274bd6..79623b33f4e816 100644
--- a/bolt/lib/Utils/CMakeLists.txt
+++ b/bolt/lib/Utils/CMakeLists.txt
@@ -1,15 +1,43 @@
+find_first_existing_vc_file("${LLVM_MAIN_SRC_DIR}" llvm_vc)
+find_first_existing_vc_file("${BOLT_SOURCE_DIR}" bolt_vc)
+
+# The VC revision include that we want to generate.
+set(version_inc "${CMAKE_CURRENT_BINARY_DIR}/VCSVersion.inc")
+
+set(generate_vcs_version_script "${LLVM_CMAKE_DIR}/GenerateVersionFromVCS.cmake")
+
+if(llvm_vc AND LLVM_APPEND_VC_REV)
+  set(llvm_source_dir ${LLVM_MAIN_SRC_DIR})
+endif()
+
+# Create custom target to generate the VC revision include.
+add_custom_command(OUTPUT "${version_inc}"
+  DEPENDS "${llvm_vc}" "${bolt_vc}" "${generate_vcs_version_script}"
+  COMMAND ${CMAKE_COMMAND} "-DNAMES=BOLT"
+                           "-DHEADER_FILE=${version_inc}"
+                           "-DBOLT_SOURCE_DIR=${BOLT_SOURCE_DIR}"
+                           "-DLLVM_VC_REPOSITORY=${llvm_vc_repository}"
+                           "-DLLVM_VC_REVISION=${llvm_vc_revision}"
+                           "-DLLVM_FORCE_VC_REVISION=${LLVM_FORCE_VC_REVISION}"
+                           "-DLLVM_FORCE_VC_REPOSITORY=${LLVM_FORCE_VC_REPOSITORY}"
+                           -P "${generate_vcs_version_script}")
+
+# Mark the generated header as being generated.
+set_source_files_properties("${version_inc}"
+  PROPERTIES GENERATED TRUE
+             HEADER_FILE_ONLY TRUE)
+
+include_directories(${CMAKE_CURRENT_BINARY_DIR})
+
 add_llvm_library(LLVMBOLTUtils
   CommandLineOpts.cpp
   Utils.cpp
-
+  ${version_inc}
   DISABLE_LLVM_LINK_LLVM_DYLIB
 
   LINK_LIBS
   ${LLVM_PTHREAD_LIB}
 
-  DEPENDS
-  llvm_vcsrevision_h
-
   LINK_COMPONENTS
   Support
   )
diff --git a/bolt/lib/Utils/CommandLineOpts.cpp b/bolt/lib/Utils/CommandLineOpts.cpp
index ba296c10c00ae1..fe819821a115a1 100644
--- a/bolt/lib/Utils/CommandLineOpts.cpp
+++ b/bolt/lib/Utils/CommandLineOpts.cpp
@@ -11,7 +11,7 @@
 //===----------------------------------------------------------------------===//
 
 #include "bolt/Utils/CommandLineOpts.h"
-#include "llvm/Support/VCSRevision.h"
+#include "VCSVersion.inc"
 
 using namespace llvm;
 
diff --git a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h
index 3cfddb530cdf63..3fca9ef7b80dc1 100644
--- a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h
+++ b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVMCTargetDesc.h
@@ -13,7 +13,6 @@
 #ifndef LLVM_LIB_TARGET_RISCV_MCTARGETDESC_RISCVMCTARGETDESC_H
 #define LLVM_LIB_TARGET_RISCV_MCTARGETDESC_RISCVMCTARGETDESC_H
 
-#include "llvm/Config/config.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Support/DataTypes.h"
 #include <memory>

@kloczek
Copy link

kloczek commented Mar 31, 2024

I've been trying to test this PR.
Looks like second part of this PR is for llvm so I've extracted only first patch.

  • looks like with this PR cmake is able to pass
  • actual build fails because it expects that libbolt_rt_instr.a and libbolt_rt_hugify.a will be installed
[tkloczko@pers-jacek x86_64-redhat-linux-gnu]$ make -k
/usr/bin/cmake -S/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src -B/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/CMakeFiles /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu//CMakeFiles/progress.marks
make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make  -f CMakeFiles/bolt_rt.dir/build.make CMakeFiles/bolt_rt.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/CMakeFiles/bolt_rt.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make  -f CMakeFiles/bolt_rt.dir/build.make CMakeFiles/bolt_rt.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
[  0%] Performing build step for 'bolt_rt'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/bolt_rt-bins && make
make[3]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/bolt_rt-bins'
[ 25%] Linking CXX static library /usr/lib64/libbolt_rt_instr.a
/usr/bin/ar: /usr/lib64/libbolt_rt_instr.a: Permission denied
make[5]: *** [CMakeFiles/bolt_rt_instr.dir/build.make:98: /usr/lib64/libbolt_rt_instr.a] Error 1
make[5]: Target 'CMakeFiles/bolt_rt_instr.dir/build' not remade because of errors.
make[4]: *** [CMakeFiles/Makefile2:85: CMakeFiles/bolt_rt_instr.dir/all] Error 2
[ 50%] Linking CXX static library /usr/lib64/libbolt_rt_hugify.a
/usr/bin/ar: /usr/lib64/libbolt_rt_hugify.a: Permission denied
make[5]: *** [CMakeFiles/bolt_rt_hugify.dir/build.make:98: /usr/lib64/libbolt_rt_hugify.a] Error 1

@kloczek
Copy link

kloczek commented Mar 31, 2024

I've been testing this PR on top of 18.1,2.

@pca006132
Copy link
Contributor Author

@kloczek it is weird that the system tries to link directly in /usr/lib64. I think typically it should build in /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu, linked, and then copied to /usr/lib64 only when you attempt to install it. What cmake arguments are you using?

Copy link
Contributor

@aaupov aaupov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this patch, really appreciate the effort. Looks good overall with several nits/questions. Please also include a test plan.

@kloczek
Copy link

kloczek commented Apr 1, 2024

Just FTR: tested updater PR and result is the same.

@kloczek
Copy link

kloczek commented Apr 1, 2024

@kloczek it is weird that the system tries to link directly in /usr/lib64. I think typically it should build in /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu, linked, and then copied to /usr/lib64 only when you attempt to install it. What cmake arguments are you using?

Below is after run make -k in build tree

[tkloczko@pers-jacek x86_64-redhat-linux-gnu]$ find . -name lib\*.a
./lib64/libLLVMBOLTUtils.a
./lib64/libLLVMBOLTCore.a
./lib64/libLLVMBOLTPasses.a
./lib64/libLLVMBOLTRuntimeLibs.a
./lib64/libLLVMBOLTProfile.a

As you may see there is no here on that list libbolt_rt_instr.a and libbolt_rt_hugify.a

@kloczek
Copy link

kloczek commented Apr 1, 2024

@pca006132 pca006132

What cmake arguments are you using?

here is cmake -L output:

-- Cache values
BOLT_BUILD_TOOLS:BOOL=ON
BOLT_CLANG_EXE:FILEPATH=
BOLT_ENABLE_RUNTIME:BOOL=ON
BOLT_INCLUDE_DOCS:BOOL=ON
BOLT_LLD_EXE:FILEPATH=
BOLT_TARGETS_TO_BUILD:STRING=AArch64;X86;RISCV
CLANG_ENABLE_BOOTSTRAP:BOOL=OFF
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_CXX_ARCHIVE_CREATE:STRING=<CMAKE_AR> Dqc <TARGET> <LINK_FLAGS> <OBJECTS>
CMAKE_CXX_ARCHIVE_FINISH:STRING=<CMAKE_RANLIB> -D <TARGET>
CMAKE_CXX_STANDARD:STRING=17
CMAKE_C_ARCHIVE_CREATE:STRING=<CMAKE_AR> Dqc <TARGET> <LINK_FLAGS> <OBJECTS>
CMAKE_C_ARCHIVE_FINISH:STRING=<CMAKE_RANLIB> -D <TARGET>
CMAKE_INSTALL_PREFIX:PATH=/usr
GNU_LD_EXECUTABLE:FILEPATH=/usr/bin/ld.bfd
LLVM_ALLOW_PROBLEMATIC_CONFIGURATIONS:BOOL=OFF
LLVM_BINARY_DIR:PATH=/usr
LLVM_ENABLE_IDE:BOOL=OFF
LLVM_ENABLE_LTO:STRING=OFF
LLVM_EXPORT_SYMBOLS_FOR_PLUGINS:BOOL=OFF
LLVM_FORCE_ENABLE_STATS:BOOL=OFF
LLVM_FORCE_USE_OLD_TOOLCHAIN:BOOL=OFF
LLVM_INCLUDE_DIRS:PATH=/usr/include
LLVM_INDIVIDUAL_TEST_COVERAGE:BOOL=OFF
LLVM_INSTALL_TOOLCHAIN_ONLY:BOOL=OFF
LLVM_LIBRARY_DIR:PATH=/usr/lib64
LLVM_MAIN_SRC_DIR:PATH=/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/../llvm
LLVM_PARALLEL_COMPILE_JOBS:STRING=
LLVM_PARALLEL_LINK_JOBS:STRING=
LLVM_SOURCE_PREFIX:STRING=
LLVM_TABLEGEN_EXE:FILEPATH=/usr/bin/llvm-tblgen
LLVM_THINLTO_CACHE_PATH:STRING=/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/lto.cache
LLVM_THIRD_PARTY_DIR:STRING=/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/../third-party
LLVM_TOOLS_BINARY_DIR:PATH=/usr/bin
LLVM_UNITTEST_LINK_FLAGS:STRING=
LLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO:BOOL=OFF
LLVM_USE_RELATIVE_PATHS_IN_FILES:BOOL=OFF

@pca006132
Copy link
Contributor Author

@aaupov I am a bit busy recently, so will need some time to come up with tests. So far what I have tested is that it builds and seems to work, but things like doxygen, LIT tests are not running in our nixpkgs build.

@pca006132
Copy link
Contributor Author

@kloczek I think the problem is LLVM_LIBRARY_DIR. I think you should probably set it to somewhere in the build directory. It should not write to /usr/lib64 when you are not installing it.

@Ericson2314
Copy link
Member

Ericson2314 commented Apr 1, 2024

@pca006132 I see

set(LLVM_LIBRARY_DIR "/nix/store/vsand2v4wisiwzkynm8li5f6af63ac6q-llvm-16.0.6-lib/lib${LLVM_LIBDIR_SUFFIX}")

in my installed LLVMConfig.cmake, which makes me think @kloczek's LLVM_LIBRARY_DIR may in fact be correct. Rather I think it is that Bolt's runtime is using LLVM_LIBRARY_DIR wrong in bolt/runtime/CMakeLists.txt.

@Ericson2314
Copy link
Member

@kloczek Can you tried a build with BOLT_ENABLE_RUNTIME=OFF? That should confirm my theory well enough.

@Ericson2314
Copy link
Member

@pca006132 What I think that boils down to is that the remove-only parts (not the CMAKE_INSTALL_LIBDIR parts) of https://github.com/NixOS/nixpkgs/blob/bca374aa31d4a9d0446216478a4242d629e72258/pkgs/development/compilers/llvm/17/bolt/gnu-install-dirs.patch should in fact go upstream.

@pca006132
Copy link
Contributor Author

Interesting, so how others build bolt with runtime enabled? maybe they point LLVM_LIBRARY_DIR to their local LLVM build?

@kloczek
Copy link

kloczek commented Apr 2, 2024

@kloczek Can you tried a build with BOLT_ENABLE_RUNTIME=OFF? That should confirm my theory well enough.

Tested and result is negative but outcome is diffreent

[tkloczko@pers-jacek x86_64-redhat-linux-gnu]$ make
/usr/bin/cmake -S/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src -B/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/CMakeFiles /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu//CMakeFiles/progress.marks
make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make  -f lib/Utils/CMakeFiles/LLVMBOLTUtils.dir/build.make lib/Utils/CMakeFiles/LLVMBOLTUtils.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/lib/Utils /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/lib/Utils /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-

[..]

[ 52%] Built target LLVMBOLTRuntimeLibs
make  -f lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build.make lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/lib/Target/AArch64 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/lib/Target/AArch64 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu/lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make  -f lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build.make lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make[2]: *** No rule to make target '/home/tkloczko/rpmbuild/BUILD/llvm/lib/Target/AArch64/AArch64.td', needed by 'lib/Target/AArch64/AArch64GenAsmMatcher.inc'.  Stop.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:1018: lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/all] Error 2
make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.2.src/x86_64-redhat-linux-gnu'

@Ericson2314
Copy link
Member

@pca006132 Yes, one of the gotchas of standalone builds in general is that <FOO>_LIBRARY_DIR is a build directory when it is part of the build, but a (previously installed) install directory if it prebuilt / system package.


@kloczek

[ 52%] Built target LLVMBOLTRuntimeLibs

Hmm, I wonder if there is an existing bug that BOLT_ENABLE_RUNTIME=OFF doesn't work?

@kloczek
Copy link

kloczek commented Apr 4, 2024

Hmm, I wonder if there is an existing bug that BOLT_ENABLE_RUNTIME=OFF doesn't work?

SQ (Stupid Question): why in this case is necessary to disable runtime? 😋

@pca006132
Copy link
Contributor Author

@kloczek that is to skip the part that leads to linker error. And do you have the source to the llvm directory (https://github.com/llvm/llvm-project/tree/main/llvm)? It requires some files there to build the runtime.

@pca006132
Copy link
Contributor Author

@aaupov what is the requirement for a test plan? Should I give a docker script to run the standalone build or is something like a nix file sufficient?

@aaupov
Copy link
Contributor

aaupov commented Apr 6, 2024

@aaupov what is the requirement for a test plan? Should I give a docker script to run the standalone build or is something like a nix file sufficient?

A reproducer invocation would be enough. Docker script is also welcome, you can use the official Dockerfile (https://github.com/llvm/llvm-project/blob/main/bolt/utils/docker/Dockerfile) as reference.

@kloczek
Copy link

kloczek commented Apr 7, 2024

@kloczek that is to skip the part that leads to linker error. And do you have the source to the llvm directory (https://github.com/llvm/llvm-project/tree/main/llvm)?

As this PR is about building bolt from per subproject dist tar ball of course I haven't.

It requires some files there to build the runtime.

I'm not 100% sure but this sounds like not proper separation of the code between llvm and bolt 🤔

@pca006132
Copy link
Contributor Author

Yes, ideally libllvm should provide those. Not sure how other projects handle this.

@kloczek
Copy link

kloczek commented Apr 7, 2024

Yes, ideally libllvm should provide those. Not sure how other projects handle this.

Maybe it would be good open for this ticket to have full visibility of this problem for other developers? 🤔
(that llvm should provide something for other subprojects and other subproject may benefit on provide such interface)


Just FTR: I've retested updated PR .. still fails

Extracted part of the `make -k` output with errors
[ 52%] Built target LLVMBOLTRuntimeLibs
make  -f lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build.make lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/lib/Target/AArch64 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/AArch64 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make  -f lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build.make lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[2]: *** No rule to make target '/home/tkloczko/rpmbuild/BUILD/llvm/lib/Target/AArch64/AArch64.td', needed by 'lib/Target/AArch64/AArch64GenAsmMatcher.inc'.
make[2]: Target 'lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/build' not remade because of errors.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:1018: lib/Target/AArch64/CMakeFiles/AArch64CommonTableGen.dir/all] Error 2
make  -f lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/build.make lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/lib/Target/X86 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/X86 /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make  -f lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/build.make lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[2]: *** No rule to make target '/home/tkloczko/rpmbuild/BUILD/llvm/lib/Target/X86/X86.td', needed by 'lib/Target/X86/X86GenAsmMatcher.inc'.
make[2]: Target 'lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/build' not remade because of errors.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:1124: lib/Target/X86/CMakeFiles/X86CommonTableGen.dir/all] Error 2
make  -f lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/build.make lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
cd /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/lib/Target/RISCV /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/RISCV /home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu/lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make  -f lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/build.make lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/build
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[2]: *** No rule to make target '/home/tkloczko/rpmbuild/BUILD/llvm/lib/Target/RISCV/RISCV.td', needed by 'lib/Target/RISCV/RISCVGenAsmMatcher.inc'.
make[2]: *** No rule to make target '/home/tkloczko/rpmbuild/BUILD/llvm/lib/Target/RISCV/RISCVGISel.td', needed by 'lib/Target/RISCV/RISCVGenGlobalISel.inc'.
make[2]: Target 'lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/build' not remade because of errors.
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'
make[1]: *** [CMakeFiles/Makefile2:1230: lib/Target/RISCV/CMakeFiles/RISCVCommonTableGen.dir/all] Error 2
make  -f tools/bat-dump/CMakeFiles/llvm-bat-dump.dir/build.make tools/bat-dump/CMakeFiles/llvm-bat-dump.dir/depend
make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/bolt-18.1.3.src/x86_64-redhat-linux-gnu'

@pca006132
Copy link
Contributor Author

The update will not fix the issue with tablegen, it is still not in the bolt subdirectory.

@kloczek
Copy link

kloczek commented Apr 30, 2024

Sorry for asking .. any update? 🤔

@pca006132
Copy link
Contributor Author

Sorry for asking .. any update? 🤔

Sorry I was a bit busy lately. I tried to use nix (and avoid building libllvm locally) for running the test cases but there is some problem with lit. Will try docker.

@pca006132
Copy link
Contributor Author

pca006132 commented May 1, 2024

I tried the following:

FROM ubuntu:22.04 AS builder

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC

RUN apt-get update && \
    apt-get install -y --no-install-recommends ca-certificates git \
      build-essential cmake ninja-build python3 libjemalloc-dev \
      python3-psutil && \
    rm -rf /var/lib/apt/lists

WORKDIR /home/bolt

COPY . llvm-project

RUN mkdir build && \
    cd build && \
    cmake -G Ninja ../llvm-project/llvm \
      -DLLVM_ENABLE_PROJECTS="clang;lld" \
      -DLLVM_TARGETS_TO_BUILD="X86;AArch64" \
      -DCMAKE_BUILD_TYPE=Release \
      -DLLVM_ENABLE_ASSERTIONS=ON \
      -DCMAKE_EXE_LINKER_FLAGS="-Wl,--push-state -Wl,-whole-archive -ljemalloc_pic -Wl,--pop-state -lpthread -lstdc++ -lm -ldl" \
      -DCMAKE_INSTALL_PREFIX=/home/bolt/install && \
    ninja install

RUN mkdir buildBolt && \
    cd buildBolt && \
    cmake -G Ninja ../llvm-project/bolt \
      -DLLVM_INCLUDE_TESTS=ON \
      -DLLVM_TARGETS_TO_BUILD="X86;AArch64" \
      -DCMAKE_BUILD_TYPE=Release \
      -DLLVM_ENABLE_ASSERTIONS=ON \
      -DCMAKE_EXE_LINKER_FLAGS="-Wl,--push-state -Wl,-whole-archive -ljemalloc_pic -Wl,--pop-state -lpthread -lstdc++ -lm -ldl" \
      -DBOLT_CLANG_EXE="/home/bolt/install/bin/clang" \
      -DBOLT_LLD_EXE="/home/bolt/install/bin/lld" \
      -DCMAKE_INSTALL_PREFIX=/home/bolt/install && \
    ninja check-bolt && \
    ninja install-llvm-bolt install-perf2bolt install-merge-fdata \
      install-llvm-boltdiff install-bolt_rt

FROM ubuntu:20.04

COPY --from=builder /home/bolt/install /usr/local

but it failed with

CMake Error at /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:743 (add_dependencies):
  The dependency target "AArch64CommonTableGen" of target
  "LLVMBOLTTargetAArch64" does not exist.
Call Stack (most recent call first):
  /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:874 (llvm_add_library)
  lib/Target/AArch64/CMakeLists.txt:7 (add_llvm_library)


CMake Error at /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:743 (add_dependencies):
  The dependency target "X86CommonTableGen" of target "LLVMBOLTTargetX86"
  does not exist.
Call Stack (most recent call first):
  /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:874 (llvm_add_library)
  lib/Target/X86/CMakeLists.txt:8 (add_llvm_library)


CMake Error at /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:1984 (add_dependencies):
  The dependency target "FileCheck" of target "check-bolt" does not exist.
Call Stack (most recent call first):
  /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:2050 (add_lit_target)
  test/CMakeLists.txt:69 (add_lit_testsuite)


CMake Error at /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:1984 (add_dependencies):
  The dependency target "count" of target "check-bolt" does not exist.
Call Stack (most recent call first):
  /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:2050 (add_lit_target)
  test/CMakeLists.txt:69 (add_lit_testsuite)


CMake Error at /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:1984 (add_dependencies):
  The dependency target "llvm-readelf" of target "check-bolt" does not exist.
Call Stack (most recent call first):
  /home/bolt/install/lib/cmake/llvm/AddLLVM.cmake:2050 (add_lit_target)
  test/CMakeLists.txt:69 (add_lit_testsuite)
...

Not sure why. When I compile using nix, it works fine (the test does not work because of how nixos places some of the binaries...). Maybe some cmake flag is not set correctly?

@kloczek
Copy link

kloczek commented May 25, 2024

Gentle ping .. any update? 🤔

@pca006132
Copy link
Contributor Author

No, not much idea about how to get the docker script working. I think maybe I will fix the downstream nix script and see if that can be regarded as a reproducer or something.

@kloczek
Copy link

kloczek commented Jun 24, 2024

Tomorrow will be branched LLVM 19.x.
Any update? 🤔

@pca006132
Copy link
Contributor Author

sorry I found that I don't really have time to work on this. please feel free to take over the patch. closing for now

@pca006132 pca006132 closed this Jun 25, 2024
rafaelauler pushed a commit that referenced this pull request Jul 25, 2024
Continue from #87196 as author did not have much time, I have taken over
working on this PR. We would like to have this so it'll be easier to
package for Nix.

Can be tested by copying cmake, bolt, third-party, and llvm directories
out into their own directory with this PR applied and then build bolt.

---------

Co-authored-by: pca006132 <[email protected]>
yuxuanchen1997 pushed a commit that referenced this pull request Jul 25, 2024
Summary:
Continue from #87196 as author did not have much time, I have taken over
working on this PR. We would like to have this so it'll be easier to
package for Nix.

Can be tested by copying cmake, bolt, third-party, and llvm directories
out into their own directory with this PR applied and then build bolt.

---------

Co-authored-by: pca006132 <[email protected]>

Test Plan: 

Reviewers: 

Subscribers: 

Tasks: 

Tags: 


Differential Revision: https://phabricator.intern.facebook.com/D60250600
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants