Skip to content

[libc] add multi-platform pre-commit github actions #119104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e6e227a
[libc] add multi-platform pre-commit github actions
SchrodingerZhu Dec 8, 2024
49ca7bc
[libc] add multi-platform pre-commit github actions
SchrodingerZhu Dec 8, 2024
a16c16e
[libc] add multi-platform pre-commit github actions
SchrodingerZhu Dec 8, 2024
feb3428
[libc] add multi-platform pre-commit github actions
SchrodingerZhu Dec 8, 2024
d1af585
[libc] add multi-platform pre-commit github actions
SchrodingerZhu Dec 8, 2024
0cf345f
prepare to fix windows build
SchrodingerZhu Dec 8, 2024
008ffef
add ubuntu preparation step
SchrodingerZhu Dec 8, 2024
1263dbc
setup ninja
SchrodingerZhu Dec 8, 2024
a7738ed
remove build type
SchrodingerZhu Dec 8, 2024
8f3c0ee
fix winget installation
SchrodingerZhu Dec 8, 2024
df70886
fix winget installation
SchrodingerZhu Dec 8, 2024
08d00cc
more fixes
SchrodingerZhu Dec 8, 2024
fcb2500
more fixes
SchrodingerZhu Dec 8, 2024
0be378f
more fixes
SchrodingerZhu Dec 8, 2024
779c806
enable fullbuild
SchrodingerZhu Dec 8, 2024
009b76c
fix
SchrodingerZhu Dec 8, 2024
6776e8d
fix
SchrodingerZhu Dec 8, 2024
a9eba5e
fix
SchrodingerZhu Dec 8, 2024
8482b9d
fix
SchrodingerZhu Dec 8, 2024
5c8fba1
skip gcc and speed up clang testing
SchrodingerZhu Dec 8, 2024
39e83dc
use windows-2022 instead of windows-latest
SchrodingerZhu Dec 8, 2024
6f3b9eb
improve windows sccache hit
SchrodingerZhu Dec 8, 2024
0b4323b
try use ninja on all platforms
SchrodingerZhu Dec 8, 2024
7cee92f
Merge branch 'main' into libc/github-action
SchrodingerZhu Dec 8, 2024
19c4456
switch caching
SchrodingerZhu Dec 8, 2024
1b0d03b
switch caching (fix syntax)
SchrodingerZhu Dec 8, 2024
2356ffd
switch caching (changing key)
SchrodingerZhu Dec 8, 2024
2e014bd
switch caching (fix)
SchrodingerZhu Dec 8, 2024
e2f1982
Merge branch 'main' into libc/github-action
SchrodingerZhu Dec 8, 2024
ad8b7d8
lift uncompressed local storage size to 1G
SchrodingerZhu Dec 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .github/workflows/libc-fullbuild-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# This workflow is for pre-commit testing of the LLVM-libc project.
name: LLVM-libc Pre-commit Fullbuild Tests

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add:

permissions:
  contents: read

To each new file you've added. This is standard best practices for GitHub actions definitions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same level of name? Sure. Adding now

on:
pull_request:
branches: [ "main" ]
paths:
- 'libc/**'
- '.github/workflows/libc-fullbuild-tests.yml'

jobs:
build:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
include:
- c_compiler: clang
cpp_compiler: clang++
# TODO: add back gcc build when it is fixed
# - c_compiler: gcc
# cpp_compiler: g++
Comment on lines +20 to +22
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have more context about what's going wrong with the gcc builds?

Copy link
Contributor Author

@SchrodingerZhu SchrodingerZhu Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

steps:
- uses: actions/checkout@v4

- name: Setup ccache
uses: hendrikmuhs/[email protected]
with:
max-size: 1G
key: libc_fullbuild_${{ matrix.c_compiler }}
variant: sccache

- name: Prepare dependencies (Ubuntu)
run: |
sudo apt-get update
sudo apt-get install -y libmpfr-dev libgmp-dev libmpc-dev ninja-build linux-headers-generic linux-libc-dev
sudo ln -sf /usr/include/$(uname -p)-linux-gnu/asm /usr/include/asm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need the symlink?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debian and its variants has a multilib sysroot, so arch-dependent headers are not installed into /usr/include/.

Ideally, libc should also add /usr/include/$(uname -p)-linux-gnu/ in its include directories but we do not do that.


- name: Set reusable strings
id: strings
shell: bash
run: |
echo "build-output-dir=${{ github.workspace }}/build" >> "$GITHUB_OUTPUT"
echo "build-install-dir=${{ github.workspace }}/install" >> "$GITHUB_OUTPUT"

- name: Configure CMake
run: >
cmake -B ${{ steps.strings.outputs.build-output-dir }}
-DCMAKE_CXX_COMPILER=${{ matrix.cpp_compiler }}
-DCMAKE_C_COMPILER=${{ matrix.c_compiler }}
-DCMAKE_BUILD_TYPE=MinSizeRel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MinSizeRel (vs. Release)?

Copy link
Contributor Author

@SchrodingerZhu SchrodingerZhu Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GHA has limited cache capability (10G in total). And we need to share it with other projects such as libcxx. Therefore, I decide to optimize for sizes.

-DCMAKE_C_COMPILER_LAUNCHER=sccache
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache
-DCMAKE_INSTALL_PREFIX=${{ steps.strings.outputs.build-install-dir }}
-DLLVM_ENABLE_RUNTIMES="libc;compiler-rt"
-DLLVM_LIBC_FULL_BUILD=ON
-DLLVM_LIBC_INCLUDE_SCUDO=ON
-DCOMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC=ON
-DCOMPILER_RT_BUILD_GWP_ASAN=OFF
-DCOMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED=OFF
-G Ninja
-S ${{ github.workspace }}/runtimes

- name: Build
run: >
cmake
--build ${{ steps.strings.outputs.build-output-dir }}
--parallel
--target install

- name: Test
run: >
cmake
--build ${{ steps.strings.outputs.build-output-dir }}
--parallel
--target check-libc
92 changes: 92 additions & 0 deletions .github/workflows/libc-overlay-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# This workflow is for pre-commit testing of the LLVM-libc project.
name: LLVM-libc Pre-commit Overlay Tests

on:
pull_request:
branches: [ "main" ]
paths:
- 'libc/**'
- '.github/workflows/libc-overlay-tests.yml'

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
# Set fail-fast to false to ensure that feedback is delivered for all matrix combinations.
fail-fast: false
matrix:
include:
# TODO: add linux gcc when it is fixed
- os: ubuntu-24.04
compiler:
c_compiler: clang
cpp_compiler: clang++
- os: windows-2022
compiler:
c_compiler: clang-cl
cpp_compiler: clang-cl
- os: macos-14
compiler:
c_compiler: clang
cpp_compiler: clang++

steps:
- uses: actions/checkout@v4

- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1
with:
max-size: 1G
key: libc_overlay_build_${{ matrix.os }}_${{ matrix.compiler.c_compiler }}
variant: sccache

- name: Prepare dependencies (Ubuntu)
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libmpfr-dev libgmp-dev libmpc-dev ninja-build

- name: Prepare dependencies (Windows)
if: runner.os == 'Windows'
run: |
choco install ninja
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is choco?

https://github.com/chocolatey/choco

does that itself have to be installed, or does it ship on the github buildbots? I guess same question for brew on OSX; usually those don't come pre-installed IIRC.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


- name: Prepare dependencies (macOS)
if: runner.os == 'macOS'
run: |
brew install ninja

- name: Set reusable strings
id: strings
shell: bash
run: |
echo "build-output-dir=${{ github.workspace }}/build" >> "$GITHUB_OUTPUT"

- name: Configure CMake
run: >
cmake -B ${{ steps.strings.outputs.build-output-dir }}
-DCMAKE_CXX_COMPILER=${{ matrix.compiler.cpp_compiler }}
-DCMAKE_C_COMPILER=${{ matrix.compiler.c_compiler }}
-DCMAKE_BUILD_TYPE=MinSizeRel
-DCMAKE_C_COMPILER_LAUNCHER=sccache
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache
-DCMAKE_POLICY_DEFAULT_CMP0141=NEW
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to set 141? Surely we can omit that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, following the instructions from https://github.com/mozilla/sccache/tree/main?tab=readme-ov-file#usage

-DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary for MinSizeRel or Release?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-DLLVM_ENABLE_RUNTIMES=libc
-G Ninja
-S ${{ github.workspace }}/runtimes

- name: Build
run: >
cmake
--build ${{ steps.strings.outputs.build-output-dir }}
--parallel
--config MinSizeRel
--target libc

- name: Test
run: >
cmake
--build ${{ steps.strings.outputs.build-output-dir }}
--parallel
--target check-libc
Loading