Skip to content

[libc] support fully OOT build #101287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 20, 2024
Merged

Conversation

SchrodingerZhu
Copy link
Contributor

@SchrodingerZhu SchrodingerZhu commented Jul 31, 2024

Fully OOT build along with SCUDO:

mkdir oot
cp -r cmake libc compiler-rt oot
cd oot
mkdir build
cd build
cmake ../libc -DLIBC_USE_NEW_HEADER_GEN=On -DLLVM_LIBC_FULL_BUILD=On -DLLVM_LIBC_FULL_BUILD=On -DLLVM_LIBC_INCLUDE_SCUDO=On -DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=On -DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=Off -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DLLVM_LIBC_COMPILER_RT_PATH=../compiler-rt/ -DCOMPILER_RT_STANDALONE_BUILD=On -GNinja -DLLVM_COMPILER_IS_GCC_COMPATIBLE=On

@llvmbot llvmbot added the libc label Jul 31, 2024
@llvmbot
Copy link
Member

llvmbot commented Jul 31, 2024

@llvm/pr-subscribers-libc

Author: Schrodinger ZHU Yifan (SchrodingerZhu)

Changes

Fully OOT build along with SCUDO:

cmake ../libc -DLIBC_USE_NEW_HEADER_GEN=On -DLLVM_LIBC_FULL_BUILD=On -DLLVM_LIBC_FULL_BUILD=On -DLLVM_LIBC_INCLUDE_SCUDO=On -DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=On -DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=Off -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DLLVM_LIBC_COMPILER_RT_PATH=../compiler-rt/ -DCOMPILER_RT_STANDALONE_BUILD=On -GNinja

Current facing the following problems:

[10/455] Building CXX object startup/linux/CMakeFiles/libc.startup.linux.do_start.dir/do_start.cpp.o
FAILED: startup/linux/CMakeFiles/libc.startup.linux.do_start.dir/do_start.cpp.o 
/usr/bin/clang++ -DLIBC_NAMESPACE=__llvm_libc -I/home/schrodinger/development/llvm-project/libc -isystem /home/schrodinger/development/llvm-project/build-2/include -std=gnu++17 -DLIBC_QSORT_IMPL=LIBC_QSORT_QUICK_SORT -ffreestanding -fno-builtin -fno-stack-protector -DLIBC_COPT_RAW_MUTEX_DEFAULT_SPIN_COUNT=100 -DLIBC_COPT_TIMEOUT_ENSURE_MONOTONICITY=1 -MD -MT startup/linux/CMakeFiles/libc.startup.linux.do_start.dir/do_start.cpp.o -MF startup/linux/CMakeFiles/libc.startup.linux.do_start.dir/do_start.cpp.o.d -o startup/linux/CMakeFiles/libc.startup.linux.do_start.dir/do_start.cpp.o -c /home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp
In file included from /home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp:13:
In file included from /home/schrodinger/development/llvm-project/libc/src/stdlib/atexit.h:12:
/home/schrodinger/development/llvm-project/libc/hdr/types/atexithandler_t.h:18:2: error: // type not available in overlay mode
   18 | #error // type not available in overlay mode
      |  ^
In file included from /home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp:13:
/home/schrodinger/development/llvm-project/libc/src/stdlib/atexit.h:17:12: error: unknown type name '__atexithandler_t'
   17 | int atexit(__atexithandler_t);
      |            ^
/home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp:136:3: error: no matching function for call to 'atexit'
  136 |   atexit([]() { cleanup_tls(tls.tp, tls.size); });
      |   ^~~~~~
/home/schrodinger/development/llvm-project/libc/src/stdlib/atexit.h:17:5: note: candidate function not viable: no known conversion from '(lambda at /home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp:136:10)' to 'int' for 1st argument
   17 | int atexit(__atexithandler_t);
      |     ^
/home/schrodinger/development/llvm-project/libc/startup/linux/do_start.cpp:141:3: error: no matching function for call to 'atexit'
  141 |   atexit(&call_fini_array_callbacks);
      |   ^~~~~~
/home/schrodinger/development/llvm-project/libc/src/stdlib/atexit.h:17:5: note: candidate function not viable: no known conversion from 'void (*)()' to 'int' for 1st argument
   17 | int atexit(__atexithandler_t);
      |     ^
4 errors generated.
[12/455] Building CXX object src/__support/OSUtil/linux/CMakeFiles/libc.src.__support.OSUtil.linux.linux_util.dir/fcntl.cpp.o
FAILED: src/__support/OSUtil/linux/CMakeFiles/libc.src.__support.OSUtil.linux.linux_util.dir/fcntl.cpp.o 
/usr/bin/clang++ -DLIBC_NAMESPACE=__llvm_libc -I/home/schrodinger/development/llvm-project/libc -isystem /home/schrodinger/development/llvm-project/build-2/include -std=gnu++17 -DLIBC_QSORT_IMPL=LIBC_QSORT_QUICK_SORT -MD -MT src/__support/OSUtil/linux/CMakeFiles/libc.src.__support.OSUtil.linux.linux_util.dir/fcntl.cpp.o -MF src/__support/OSUtil/linux/CMakeFiles/libc.src.__support.OSUtil.linux.linux_util.dir/fcntl.cpp.o.d -o src/__support/OSUtil/linux/CMakeFiles/libc.src.__support.OSUtil.linux.linux_util.dir/fcntl.cpp.o -c /home/schrodinger/development/llvm-project/libc/src/__support/OSUtil/linux/fcntl.cpp
/home/schrodinger/development/llvm-project/libc/src/__support/OSUtil/linux/fcntl.cpp:41:20: error: variable has incomplete type 'struct flock64'
   41 |     struct fl

Full diff: https://github.com/llvm/llvm-project/pull/101287.diff

2 Files Affected:

  • (modified) libc/CMakeLists.txt (+18-13)
  • (modified) libc/newhdrgen/yaml_to_classes.py (+1-1)
diff --git a/libc/CMakeLists.txt b/libc/CMakeLists.txt
index 1ce3bdf53b525..2b15d124abb79 100644
--- a/libc/CMakeLists.txt
+++ b/libc/CMakeLists.txt
@@ -1,6 +1,8 @@
 cmake_minimum_required(VERSION 3.20.0)
 set(LLVM_SUBPROJECT_TITLE "libc")
 
+include(CheckCXXCompilerFlag)
+
 # Include LLVM's cmake policies.
 if(NOT DEFINED LLVM_COMMON_CMAKE_UTILS)
   set(LLVM_COMMON_CMAKE_UTILS ${CMAKE_CURRENT_SOURCE_DIR}/../cmake)
@@ -53,18 +55,19 @@ set(LIBC_NAMESPACE ${default_namespace}
 
 add_subdirectory(newhdrgen)
 
-
-if(LLVM_LIBC_FULL_BUILD OR LLVM_LIBC_GPU_BUILD)
-  if(NOT LIBC_HDRGEN_EXE)
-    # We need to set up hdrgen first since other targets depend on it.
-    add_subdirectory(utils/LibcTableGenUtil)
-    add_subdirectory(utils/HdrGen)
-    # Calling add_tablegen sets variables like LIBC_TABLEGEN_EXE in
-    # PARENT_SCOPE which get lost until saved in the cache.
-    set(LIBC_TABLEGEN_EXE "${LIBC_TABLEGEN_EXE}" CACHE INTERNAL "")
-    set(LIBC_TABLEGEN_TARGET "${LIBC_TABLEGEN_TARGET}" CACHE INTERNAL "")
-  else()
-    message(STATUS "Will use ${LIBC_HDRGEN_EXE} for libc header generation.")
+if(NOT LIBC_USE_NEW_HEADER_GEN)
+  if(LLVM_LIBC_FULL_BUILD OR LLVM_LIBC_GPU_BUILD)
+    if(NOT LIBC_HDRGEN_EXE)
+      # We need to set up hdrgen first since other targets depend on it.
+      add_subdirectory(utils/LibcTableGenUtil)
+      add_subdirectory(utils/HdrGen)
+      # Calling add_tablegen sets variables like LIBC_TABLEGEN_EXE in
+      # PARENT_SCOPE which get lost until saved in the cache.
+      set(LIBC_TABLEGEN_EXE "${LIBC_TABLEGEN_EXE}" CACHE INTERNAL "")
+      set(LIBC_TABLEGEN_TARGET "${LIBC_TABLEGEN_TARGET}" CACHE INTERNAL "")
+    else()
+      message(STATUS "Will use ${LIBC_HDRGEN_EXE} for libc header generation.")
+    endif()
   endif()
 endif()
 # We will build the GPU utilities if we are not doing a runtimes build.
@@ -327,7 +330,9 @@ endif()
 
 option(LLVM_LIBC_INCLUDE_SCUDO "Include the SCUDO standalone as the allocator for LLVM libc" OFF)
 if(LLVM_LIBC_INCLUDE_SCUDO)
-  if (NOT ("compiler-rt" IN_LIST LLVM_ENABLE_PROJECTS OR "compiler-rt" IN_LIST LLVM_ENABLE_RUNTIMES))
+  if (LLVM_LIBC_COMPILER_RT_PATH)
+    add_subdirectory(${LLVM_LIBC_COMPILER_RT_PATH} ${CMAKE_CURRENT_BINARY_DIR}/compiler-rt)
+  elseif(NOT ("compiler-rt" IN_LIST LLVM_ENABLE_PROJECTS OR "compiler-rt" IN_LIST LLVM_ENABLE_RUNTIMES))
     message(FATAL_ERROR "SCUDO cannot be included without adding compiler-rt to LLVM_ENABLE_PROJECTS or LLVM_ENABLE_RUNTIMES")
   endif()
 endif()
diff --git a/libc/newhdrgen/yaml_to_classes.py b/libc/newhdrgen/yaml_to_classes.py
old mode 100644
new mode 100755
index d3bdfde2e2a22..9d3593a1bafc1
--- a/libc/newhdrgen/yaml_to_classes.py
+++ b/libc/newhdrgen/yaml_to_classes.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 #
 # ===- Generate headers for libc functions  -------------------*- python -*--==#
 #

@SchrodingerZhu SchrodingerZhu changed the title [libc][WIP] support fully oot build [libc][WIP] support fully OOT build Jul 31, 2024
@SchrodingerZhu SchrodingerZhu changed the title [libc][WIP] support fully OOT build [libc] support fully OOT build Jul 31, 2024
@SchrodingerZhu SchrodingerZhu marked this pull request as draft July 31, 2024 05:34
Comment on lines +333 to +361
if (LLVM_LIBC_COMPILER_RT_PATH)
add_subdirectory(${LLVM_LIBC_COMPILER_RT_PATH} ${CMAKE_CURRENT_BINARY_DIR}/compiler-rt)
elseif(NOT ("compiler-rt" IN_LIST LLVM_ENABLE_PROJECTS OR "compiler-rt" IN_LIST LLVM_ENABLE_RUNTIMES))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this part changing exactly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

user can provide compiler-rt source directory on their own.

@SchrodingerZhu SchrodingerZhu requested a review from jhuber6 July 31, 2024 15:31
@SchrodingerZhu SchrodingerZhu marked this pull request as ready for review July 31, 2024 17:59
@nickdesaulniers
Copy link
Member

ok2merge?

@SchrodingerZhu
Copy link
Contributor Author

yes. I think it is still applicable

@SchrodingerZhu SchrodingerZhu merged commit c0efcc0 into llvm:main Nov 20, 2024
5 of 6 checks passed
@SchrodingerZhu SchrodingerZhu deleted the libc/fully-oot branch November 20, 2024 19:12
jhuber6 added a commit that referenced this pull request Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants