Skip to content

[CMake] Fix some breakages when using ninja multi config #65451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 22, 2025
Merged

[CMake] Fix some breakages when using ninja multi config #65451

merged 2 commits into from
Feb 22, 2025

Conversation

ur4t
Copy link
Contributor

@ur4t ur4t commented Sep 6, 2023

When using multi-config generator to build libLLVM.so like cmake -G 'Ninja Multi-Config' -Sllvm -B/tmp/out/ninja-multi -DCMAKE_CONFIGURATION_TYPES='Debug;Release' -DLLVM_LINK_LLVM_DYLIB=on -DLLVM_TARGETS_TO_BUILD=host && cmake --build /tmp/out/ninja-multi --config Debug, lld complains error: cannot find version script /tmp/out/ninja-multi/Debug/lib/tools/llvm-shlib/simple_version_script.map.

This patch adds multi-config compatibility when configuring simple_version_script.map.

Fixes #63800.

When using multi-config generator, clang's headers is not copied to proper directories, which is fixed as well.

@ur4t
Copy link
Contributor Author

ur4t commented Sep 13, 2023

@llvmbot Is there any mechanism to classify this PR?

@ur4t
Copy link
Contributor Author

ur4t commented Sep 13, 2023

@EugeneZelenko Is there any mechanism to classify and deal with this PR?

@EugeneZelenko EugeneZelenko added cmake Build system in general and CMake in particular llvm-tools All llvm tools that do not have corresponding tag labels Sep 13, 2023
@EugeneZelenko
Copy link
Contributor

@ur4t: You could also look into file history and add some of developers who changed file in past as reviewers.

@ur4t
Copy link
Contributor Author

ur4t commented Sep 13, 2023

@mgorny It seems that the most recently modification on llvm/tools/llvm-shlib/CMakeLists.txt is performed by you. Would you like to participate in reviewing this PR?

@mgorny
Copy link
Member

mgorny commented Sep 13, 2023

I'm not really an expert on this. @MaskRay perhaps?

@ur4t
Copy link
Contributor Author

ur4t commented Sep 14, 2023

@MaskRay Would you like to participate in reviewing this PR?

FYI, building libLLVM.so requires simple_version_script.map, but corresponding cmake configure_file command does not take multiple configurations into account. Maybe because there is no multiple configuration generator before Ninja Multi-Config on platforms supporting libLLVM.so.

This PR fixes this issue #63800 by adding multiple configurations compatibility like other CMakeLists.txt found doing so.

@MaskRay
Copy link
Member

MaskRay commented Sep 24, 2023

I'm not really an expert on this. @MaskRay perhaps?

I am not familiar with CMake... but this patch does address a problem.

I suggest that you add to the description the following instruction and also some description from #63800.

% cmake -G 'Ninja Multi-Config' -Sllvm -B/tmp/out/ninja-multi -DCMAKE_CONFIGURATION_TYPES='Debug;Release' -DLLVM_LINK_LLVM_DYLIB=on -DLLVM_TARGETS_TO_BUILD=host
% ninja -C /tmp/out/ninja-multi -f build-Debug.ninja llvm-mc
...
ld.lld: error: cannot find version script /tmp/out/ninja-multi/Debug/lib/tools/llvm-shlib/simple_version_script.map

"llvm shared library"

Use the preciser term -DLLVM_LINK_LLVM_DYLIB=on

${CMAKE_CURRENT_SOURCE_DIR}/simple_version_script.map.in
${LLVM_LIBRARY_DIR}/tools/llvm-shlib/simple_version_script.map)

if(NOT "${CMAKE_CFG_INTDIR}" STREQUAL ".")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally we prefer

if (...) {
} else {
}

over

if (!...) {
} else {
}

CMake is similar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "if" style is adjusted in 72e1dc0.

@MaskRay MaskRay changed the title [llvm][CMake] Fix llvm shared library when using ninja multi config [CMake] Fix -DLLVM_LINK_LLVM_DYLIB=on builds when using ninja multi config Sep 24, 2023
@ur4t
Copy link
Contributor Author

ur4t commented Feb 17, 2025

@MaskRay This tiny PR has been staled for months even years. Could we please merge it?

@llvmbot llvmbot added clang Clang issues not falling into any other category backend:X86 clang:headers Headers provided by Clang, e.g. for intrinsics labels Feb 17, 2025
@llvmbot
Copy link
Member

llvmbot commented Feb 17, 2025

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-backend-x86

Author: None (ur4t)

Changes

When using multi-config generator to build libLLVM.so like cmake -G 'Ninja Multi-Config' -Sllvm -B/tmp/out/ninja-multi -DCMAKE_CONFIGURATION_TYPES='Debug;Release' -DLLVM_LINK_LLVM_DYLIB=on -DLLVM_TARGETS_TO_BUILD=host && cmake --build /tmp/out/ninja-multi --config Debug, lld complains error: cannot find version script /tmp/out/ninja-multi/Debug/lib/tools/llvm-shlib/simple_version_script.map.

This patch adds multi-config compatibility when configuring simple_version_script.map.

Fixes #63800.


Full diff: https://github.com/llvm/llvm-project/pull/65451.diff

2 Files Affected:

  • (modified) clang/lib/Headers/CMakeLists.txt (+19-6)
  • (modified) llvm/tools/llvm-shlib/CMakeLists.txt (+17-8)
diff --git a/clang/lib/Headers/CMakeLists.txt b/clang/lib/Headers/CMakeLists.txt
index 43124111b7ba5..a3a505bcb7f88 100644
--- a/clang/lib/Headers/CMakeLists.txt
+++ b/clang/lib/Headers/CMakeLists.txt
@@ -395,12 +395,25 @@ set(riscv_generated_files)
 
 function(copy_header_to_output_dir src_dir file)
   set(src ${src_dir}/${file})
-  set(dst ${output_dir}/${file})
-  add_custom_command(OUTPUT ${dst}
-    DEPENDS ${src}
-    COMMAND ${CMAKE_COMMAND} -E copy_if_different ${src} ${dst}
-    COMMENT "Copying clang's ${file}...")
-  list(APPEND out_files ${dst})
+  if("${CMAKE_CFG_INTDIR}" STREQUAL ".")
+    set(dst ${output_dir}/${file})
+    add_custom_command(OUTPUT ${dst}
+      DEPENDS ${src}
+      COMMAND ${CMAKE_COMMAND} -E copy_if_different ${src} ${dst}
+      COMMENT "Copying clang's ${file}...")
+    list(APPEND out_files ${dst})
+  else()
+    foreach(BUILD_MODE ${CMAKE_CONFIGURATION_TYPES})
+      # Replace the special string with a per config directory.
+      string(REPLACE ${CMAKE_CFG_INTDIR} ${BUILD_MODE} per_conf_output_dir ${output_dir})
+      set(dst ${per_conf_output_dir}/${file})
+      add_custom_command(OUTPUT ${dst}
+        DEPENDS ${src}
+        COMMAND ${CMAKE_COMMAND} -E copy_if_different ${src} ${dst}
+        COMMENT "Copying clang's ${file}...")
+      list(APPEND out_files ${dst})
+    endforeach()
+  endif()
   set(out_files ${out_files} PARENT_SCOPE)
 endfunction(copy_header_to_output_dir)
 
diff --git a/llvm/tools/llvm-shlib/CMakeLists.txt b/llvm/tools/llvm-shlib/CMakeLists.txt
index ede3c5034e045..a5b0cab0f1ce5 100644
--- a/llvm/tools/llvm-shlib/CMakeLists.txt
+++ b/llvm/tools/llvm-shlib/CMakeLists.txt
@@ -45,10 +45,19 @@ if(LLVM_BUILD_LLVM_DYLIB)
   if("${CMAKE_SYSTEM_NAME}" STREQUAL "Darwin")
     set(LIB_NAMES -Wl,-all_load ${LIB_NAMES})
   else()
-    configure_file(
-    ${CMAKE_CURRENT_SOURCE_DIR}/simple_version_script.map.in
-    ${LLVM_LIBRARY_DIR}/tools/llvm-shlib/simple_version_script.map)
-
+    if("${CMAKE_CFG_INTDIR}" STREQUAL ".")
+      configure_file(
+        ${CMAKE_CURRENT_SOURCE_DIR}/simple_version_script.map.in
+        ${LLVM_LIBRARY_DIR}/tools/llvm-shlib/simple_version_script.map)
+    else()
+      foreach(BUILD_MODE ${CMAKE_CONFIGURATION_TYPES})
+        # Replace the special string with a per config directory.
+        string(REPLACE ${CMAKE_CFG_INTDIR} ${BUILD_MODE} PER_CONF_LIBRARY_DIR ${LLVM_LIBRARY_DIR})
+        configure_file(
+          ${CMAKE_CURRENT_SOURCE_DIR}/simple_version_script.map.in
+          ${PER_CONF_LIBRARY_DIR}/tools/llvm-shlib/simple_version_script.map)
+      endforeach()
+    endif()
     if(MSVC)
       target_link_directories(LLVM PRIVATE ${LLVM_LIBRARY_DIR})
       foreach(library ${LIB_NAMES})
@@ -156,7 +165,10 @@ if(LLVM_BUILD_LLVM_C_DYLIB AND MSVC)
   # Need to separate lib names with newlines.
   string(REPLACE ";" "\n" FILE_CONTENT "${FULL_LIB_NAMES}")
 
-  if(NOT "${CMAKE_CFG_INTDIR}" STREQUAL ".")
+  if("${CMAKE_CFG_INTDIR}" STREQUAL ".")
+    # Write out the full lib names into file to be read by the python script.
+    file(WRITE ${LIBSFILE} "${FILE_CONTENT}")
+  else()
     foreach(BUILD_MODE ${CMAKE_CONFIGURATION_TYPES})
       # Replace the special string with a per config directory.
       string(REPLACE ${CMAKE_CFG_INTDIR} ${BUILD_MODE} PER_CONF_CONTENT "${FILE_CONTENT}")
@@ -166,9 +178,6 @@ if(LLVM_BUILD_LLVM_C_DYLIB AND MSVC)
       # ${CMAKE_CFG_INTDIR} correctly and select the right one.
       file(WRITE ${LLVM_BINARY_DIR}/${BUILD_MODE}/libllvm-c.args "${PER_CONF_CONTENT}")
     endforeach()
-  else()
-    # Write out the full lib names into file to be read by the python script.
-    file(WRITE ${LIBSFILE} "${FILE_CONTENT}")
   endif()
 
   # Generate the exports file dynamically.

@ur4t ur4t changed the title [CMake] Fix -DLLVM_LINK_LLVM_DYLIB=on builds when using ninja multi config [CMake] Fix some breakages when using ninja multi config Feb 17, 2025
@MaskRay
Copy link
Member

MaskRay commented Feb 21, 2025

Could you rebase this patch?

In addition, please unselect the hidden email feature per suggestion on https://discourse.llvm.org/t/hidden-emails-on-github-should-we-do-something-about-it/74223

@ur4t
Copy link
Contributor Author

ur4t commented Feb 22, 2025

@MaskRay I have unselected the hidden email feature and updated the branch. Is it necessary to reset email of my old commits?

@MaskRay MaskRay merged commit 62c7891 into llvm:main Feb 22, 2025
11 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Feb 22, 2025

LLVM Buildbot has detected a new failure on builder lldb-x86_64-debian running on lldb-x86_64-debian while building clang,llvm at step 6 "test".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/162/builds/16701

Here is the relevant piece of the build log for the reference
Step 6 (test) failure: build (failure)
...
PASS: lldb-unit :: Core/./LLDBCoreTests/50/67 (2742 of 2787)
PASS: lldb-unit :: Utility/./UtilityTests/78/127 (2743 of 2787)
PASS: lldb-shell :: Subprocess/vfork-follow-parent-wp.test (2744 of 2787)
PASS: lldb-unit :: Utility/./UtilityTests/79/127 (2745 of 2787)
PASS: lldb-shell :: Unwind/basic-block-sections.test (2746 of 2787)
PASS: lldb-api :: tools/lldb-server/TestGdbRemoteLaunch.py (2747 of 2787)
PASS: lldb-api :: python_api/watchpoint/watchlocation/TestSetWatchlocation.py (2748 of 2787)
PASS: lldb-api :: tools/lldb-server/attach-wait/TestGdbRemoteAttachWait.py (2749 of 2787)
PASS: lldb-unit :: Host/./HostTests/2/98 (2750 of 2787)
PASS: lldb-unit :: Host/./HostTests/30/98 (2751 of 2787)
FAIL: lldb-api :: tools/lldb-dap/launch/TestDAP_launch.py (2752 of 2787)
******************** TEST 'lldb-api :: tools/lldb-dap/launch/TestDAP_launch.py' FAILED ********************
Script:
--
/usr/bin/python3 /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./lib --env LLVM_INCLUDE_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/include --env LLVM_TOOLS_DIR=/home/worker/2.0.1/lldb-x86_64-debian/build/./bin --arch x86_64 --build-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex --lldb-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/lldb --compiler /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/clang --dsymutil /home/worker/2.0.1/lldb-x86_64-debian/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./bin --lldb-obj-root /home/worker/2.0.1/lldb-x86_64-debian/build/tools/lldb --lldb-libs-dir /home/worker/2.0.1/lldb-x86_64-debian/build/./lib -t /home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/tools/lldb-dap/launch -p TestDAP_launch.py
--
Exit Code: 1

Command Output (stdout):
--
lldb version 21.0.0git (https://github.com/llvm/llvm-project.git revision 62c78919c678915936fcc08212b02db23738dd4d)
  clang revision 62c78919c678915936fcc08212b02db23738dd4d
  llvm revision 62c78919c678915936fcc08212b02db23738dd4d
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']
========= DEBUG ADAPTER PROTOCOL LOGS =========
1740247649.804245472 stdin/stdout --> 
Content-Length: 344

{
  "arguments": {
    "adapterID": "lldb-native",
    "clientID": "vscode",
    "columnsStartAt1": true,
    "linesStartAt1": true,
    "locale": "en-us",
    "pathFormat": "path",
    "sourceInitFile": false,
    "supportsRunInTerminalRequest": true,
    "supportsStartDebuggingRequest": true,
    "supportsVariablePaging": true,
    "supportsVariableType": true
  },
  "command": "initialize",
  "seq": 1,
  "type": "request"
}
1740247649.806897640 stdin/stdout <-- 
Content-Length: 1631


@llvm-ci
Copy link
Collaborator

llvm-ci commented Feb 23, 2025

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building clang,llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/2342

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/38/87' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-6416-38-87.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=87 GTEST_SHARD_INDEX=38 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


@ur4t ur4t deleted the fix-llvm-shlib-ninja-multi-config-github-issue-63800 branch February 23, 2025 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:X86 clang:headers Headers provided by Clang, e.g. for intrinsics clang Clang issues not falling into any other category cmake Build system in general and CMake in particular llvm-tools All llvm tools that do not have corresponding tag
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect position of generated simple_version_script.map when using Ninja Multi-Config
6 participants