Skip to content

Commit a30095a

Browse files
rprichardrprichard
authored
[libc++][Android] Add libcxx-builder-android Docker image (#69273)
Add a Dockerfile for a new Docker image, libcxx-builder-android, that extends libcxx-builder with support for testing Android. The image includes these things: * An Android Clang compiler and sysroot. * The Android platform-tools (e.g. adb), so that an Android buildbot can run programs on an Android device. At container startup, copy these platform tools to an "android-platform-tools" Docker volume to share them with an emulator container. This copying ensures that the emulator and libcxx-builder containers avoid mismatched adb versions. * Docker, so that an Android buildbot can manage a sibling Docker container that runs the Android emulator. Add an Android-specific run-buildbot-container script for local development. Currently using this script requires building libcxx-build-android and an emulator image locally. Fixes: #69270 Differential Revision: https://reviews.llvm.org/D155271
1 parent ea9e116 commit a30095a

File tree

4 files changed

+135
-0
lines changed

4 files changed

+135
-0
lines changed

‎libcxx/utils/ci/vendor/android/Dockerfile

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
#===----------------------------------------------------------------------===##
2+
#
3+
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
# See https://llvm.org/LICENSE.txt for license information.
5+
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
#
7+
#===----------------------------------------------------------------------===##
8+
9+
#
10+
# This Dockerfile extends ldionne/libcxx-builder with Android support, including
11+
# Android Clang and sysroot, Android platform-tools, and the Docker client.
12+
#
13+
# $ docker build -t libcxx-builder-android libcxx/utils/ci/vendor/android
14+
#
15+
16+
FROM ldionne/libcxx-builder
17+
18+
# Switch back to the root user to install things into /opt and /usr.
19+
USER root
20+
WORKDIR /
21+
22+
# Install the Android platform tools (e.g. adb) into /opt/android/sdk.
23+
RUN apt-get update && apt-get install -y unzip
24+
RUN mkdir -p /opt/android/sdk && cd /opt/android/sdk && \
25+
curl -LO https://dl.google.com/android/repository/platform-tools-latest-linux.zip && \
26+
unzip platform-tools-latest-linux.zip && \
27+
rm platform-tools-latest-linux.zip
28+
29+
# Install the current Android compiler. Specify the prebuilts commit to retrieve
30+
# this compiler version even after it's removed from HEAD.
31+
ENV ANDROID_CLANG_VERSION=r498229b
32+
ENV ANDROID_CLANG_PREBUILTS_COMMIT=5186d132c99aa75dc25207c392e3ea5b93d0107e
33+
RUN git clone --filter=blob:none --sparse \
34+
https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86 \
35+
/opt/android/clang && \
36+
git -C /opt/android/clang checkout ${ANDROID_CLANG_PREBUILTS_COMMIT} && \
37+
git -C /opt/android/clang sparse-checkout add clang-${ANDROID_CLANG_VERSION} && \
38+
rm -fr /opt/android/clang/.git && \
39+
ln -sf /opt/android/clang/clang-${ANDROID_CLANG_VERSION} /opt/android/clang/clang-current && \
40+
# The "git sparse-checkout" and "ln" commands succeed even if nothing was
41+
# checked out, so use this "ls" command to fix that.
42+
ls /opt/android/clang/clang-current/bin/clang
43+
44+
# Install an Android sysroot. New AOSP sysroots are available at
45+
# https://ci.android.com/builds/branches/aosp-main/grid, the "ndk" target. The
46+
# NDK also makes its sysroot prebuilt available at
47+
# https://android.googlesource.com/platform/prebuilts/ndk/+/refs/heads/dev/platform/sysroot.
48+
ENV ANDROID_SYSROOT_BID=10957860
49+
RUN cd /opt/android && \
50+
curl -L -o ndk_platform.tar.bz2 \
51+
https://androidbuildinternal.googleapis.com/android/internal/build/v3/builds/${ANDROID_SYSROOT_BID}/ndk/attempts/latest/artifacts/ndk_platform.tar.bz2/url && \
52+
tar xf ndk_platform.tar.bz2 && \
53+
rm ndk_platform.tar.bz2
54+
55+
# Install Docker. Mark the binary setuid so it can be run without prefixing it
56+
# with sudo. Adding the container user to the docker group doesn't work because
57+
# /var/run/docker.sock is owned by the host's docker GID, not the container's
58+
# docker GID.
59+
RUN apt-get update && apt-get install -y gpg && \
60+
install -m 0755 -d /etc/apt/keyrings && \
61+
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg && \
62+
chmod a+r /etc/apt/keyrings/docker.gpg && \
63+
echo "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
64+
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
65+
tee /etc/apt/sources.list.d/docker.list > /dev/null && \
66+
apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin && \
67+
chmod u+s /usr/bin/docker
68+
69+
COPY ./container-setup.sh /opt/android/container-setup.sh
70+
71+
USER libcxx-builder
72+
WORKDIR /home/libcxx-builder
73+
74+
# Add Android platform-tools to the PATH.
75+
ENV PATH="/opt/android/sdk/platform-tools:${PATH}"
76+
77+
# Reset the buildkite-agent.cfg file. The tags are provided by an environment
78+
# variable instead.
79+
RUN cp /home/libcxx-builder/.buildkite-agent/buildkite-agent.dist.cfg \
80+
/home/libcxx-builder/.buildkite-agent/buildkite-agent.cfg
81+
82+
# Modify the Buildkite agent cmdline to do Android setup stuff first.
83+
CMD /opt/android/container-setup.sh && buildkite-agent start

‎libcxx/utils/ci/vendor/android/container-setup.sh

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/usr/bin/env bash
2+
#===----------------------------------------------------------------------===##
3+
#
4+
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
5+
# See https://llvm.org/LICENSE.txt for license information.
6+
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
7+
#
8+
#===----------------------------------------------------------------------===##
9+
10+
set -e
11+
12+
# Different versions of adb can sometimes be incompatible (i.e. "adb server
13+
# version (nn) doesn't match this client (mm); killing..."). Ensure that the adb
14+
# in the main builder image matches that in the emulator by sharing the
15+
# platform-tools from the main image.
16+
if [ -d /mnt/android-platform-tools ]; then
17+
sudo rm -fr /mnt/android-platform-tools/platform-tools
18+
sudo cp -r /opt/android/sdk/platform-tools /mnt/android-platform-tools
19+
fi

‎libcxx/utils/ci/vendor/android/run-buildbot-container

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#!/usr/bin/env bash
2+
3+
# Similar to libcxx/utils/ci/run-buildbot-container, but adds additional options
4+
# needed for running Android tests.
5+
6+
set -e
7+
8+
MONOREPO_ROOT="$(git rev-parse --show-toplevel)"
9+
if [[ ! -d "${MONOREPO_ROOT}/libcxx/utils/ci/vendor/android" ]]; then
10+
echo "Was unable to find the root of the LLVM monorepo; are you running from within the monorepo?"
11+
exit 1
12+
fi
13+
14+
DOCKER_OPTIONS=(-it)
15+
DOCKER_OPTIONS+=(--volume "${MONOREPO_ROOT}:/llvm")
16+
DOCKER_OPTIONS+=(--workdir "/llvm")
17+
DOCKER_OPTIONS+=(--cap-add=SYS_PTRACE)
18+
19+
# Mount this volume to allow the main image to share its copy of the Android
20+
# platform tools with the emulator image, ensuring that the adb versions match.
21+
# This argument will create a new volume if it doesn't already exist.
22+
DOCKER_OPTIONS+=(--volume android-platform-tools:/mnt/android-platform-tools)
23+
24+
# Pass through the Docker socket so that the buildbot can start a sibling
25+
# container running an Android emulator.
26+
if [ -S /var/run/docker.sock ]; then
27+
DOCKER_OPTIONS+=(--volume /var/run/docker.sock:/var/run/docker.sock)
28+
fi
29+
30+
docker run "${DOCKER_OPTIONS[@]}" libcxx-builder-android \
31+
bash -c 'git config --global --add safe.directory /llvm; (/opt/android/container-setup.sh && exec bash)'

‎libcxx/utils/data/ignore_format.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7298,4 +7298,6 @@ libcxx/utils/ci/Dockerfile
72987298
libcxx/utils/ci/macos-ci-setup
72997299
libcxx/utils/ci/run-buildbot
73007300
libcxx/utils/ci/run-buildbot-container
7301+
libcxx/utils/ci/vendor/android/Dockerfile
7302+
libcxx/utils/ci/vendor/android/run-buildbot-container
73017303
libcxx/utils/libcxx-lit

0 commit comments

Comments
 (0)