Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libc++][Android] Fix Android bugs in the CI Dockerfile #99623

Merged
merged 1 commit into from
Jul 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 27 additions & 15 deletions libcxx/utils/ci/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ RUN sudo apt-get update \
#RUN apt-get update && apt-get install -y ninja-build python3 python3-distutils python3-psutil git gdb ccache
# TODO add ninja-build once 1.11 is available in Ubuntu, also remove the manual installation.
RUN <<EOF
set -e
wget -qO /tmp/ninja.gz https://github.com/ninja-build/ninja/releases/latest/download/ninja-linux.zip
gunzip /tmp/ninja.gz
chmod a+x /tmp/ninja
Expand All @@ -115,6 +116,7 @@ EOF

# These two locales are not enabled by default so generate them
RUN <<EOF
set -e
printf "fr_CA ISO-8859-1\ncs_CZ ISO-8859-2" | sudo tee -a /etc/locale.gen
sudo mkdir /usr/local/share/i1en/
printf "fr_CA ISO-8859-1\ncs_CZ ISO-8859-2" | sudo tee -a /usr/local/share/i1en/SUPPORTED
Expand All @@ -129,6 +131,7 @@ EOF
# 14 release branch CI uses it. The tip-of-trunk CI will never use Clang 12,
# though.
RUN <<EOF
set -e
sudo apt-get update
wget https://apt.llvm.org/llvm.sh -O /tmp/llvm.sh
chmod +x /tmp/llvm.sh
Expand All @@ -142,6 +145,7 @@ EOF

# Install the most recent GCC, like clang install the previous version as a transition.
RUN <<EOF
set -e
sudo git clone https://github.com/compiler-explorer/infra.git /tmp/ce-infra
(cd /tmp/ce-infra && sudo make ce)
sudo /tmp/ce-infra/bin/ce_install install compilers/c++/x86/gcc $GCC_LATEST_VERSION.1.0
Expand All @@ -155,13 +159,14 @@ EOF

RUN <<EOF
# Install a recent CMake
set -e
wget https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.sh -O /tmp/install-cmake.sh
sudo bash /tmp/install-cmake.sh --prefix=/usr --exclude-subdir --skip-license
rm /tmp/install-cmake.sh
EOF

# ===----------------------------------------------------------------------===##
# Android Buildkite Image
# Android Builder Base Image
# ===----------------------------------------------------------------------===##

FROM ubuntu:jammy AS android-builder-base
Expand All @@ -170,10 +175,11 @@ ARG ANDROID_CLANG_VERSION
ARG ANDROID_CLANG_PREBUILTS_COMMIT
ARG ANDROID_SYSROOT_BID

RUN apt-get update && apt-get install -y curl unzip git
RUN apt-get update && apt-get install -y curl bzip2 git unzip

# Install the Android platform tools (e.g. adb) into /opt/android/sdk.
RUN <<EOF
set -e
mkdir -p /opt/android/sdk
cd /opt/android/sdk
curl -LO https://dl.google.com/android/repository/platform-tools-latest-linux.zip
Expand All @@ -187,6 +193,7 @@ EOF
ENV ANDROID_CLANG_VERSION=$ANDROID_CLANG_VERSION
ENV ANDROID_CLANG_PREBUILTS_COMMIT=$ANDROID_CLANG_PREBUILTS_COMMIT
RUN <<EOF
set -e
git clone --filter=blob:none --sparse \
https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86 \
/opt/android/clang
Expand All @@ -206,26 +213,14 @@ EOF

ENV ANDROID_SYSROOT_BID=$ANDROID_SYSROOT_BID
RUN <<EOF
set -e
cd /opt/android
curl -L -o ndk_platform.tar.bz2 \
https://androidbuildinternal.googleapis.com/android/internal/build/v3/builds/${ANDROID_SYSROOT_BID}/ndk/attempts/latest/artifacts/ndk_platform.tar.bz2/url
tar xf ndk_platform.tar.bz2
rm ndk_platform.tar.bz2
EOF

# Install Docker
RUN <<EOF
curl -fsSL https://get.docker.com -o /tmp/get-docker.sh
sh /tmp/get-docker.sh
rm /tmp/get-docker.sh

# Install Docker. Mark the binary setuid so it can be run without prefixing it
# with sudo. Adding the container user to the docker group doesn't work because
# /var/run/docker.sock is owned by the host's docker GID, not the container's
# docker GID.
chmod u+s /usr/bin/docker
EOF

# ===----------------------------------------------------------------------===##
# Buildkite Builder Image
# ===----------------------------------------------------------------------===##
Expand All @@ -243,6 +238,7 @@ WORKDIR /home/libcxx-builder
# Install the Buildkite agent and dependencies. This must be done as non-root
# for the Buildkite agent to be installed in a path where we can find it.
RUN <<EOF
set -e
cd /home/libcxx-builder
curl -sL https://raw.githubusercontent.com/buildkite/agent/main/install.sh -o /tmp/install-agent.sh
bash /tmp/install-agent.sh
Expand Down Expand Up @@ -271,6 +267,22 @@ COPY ./vendor/android/container-setup.sh /opt/android/container-setup.sh

ENV PATH="/opt/android/sdk/platform-tools:${PATH}"

USER root

# Install Docker
RUN <<EOF
set -e
curl -fsSL https://get.docker.com -o /tmp/get-docker.sh
sh /tmp/get-docker.sh
rm /tmp/get-docker.sh

# Install Docker. Mark the binary setuid so it can be run without prefixing it
# with sudo. Adding the container user to the docker group doesn't work because
# /var/run/docker.sock is owned by the host's docker GID, not the container's
# docker GID.
chmod u+s /usr/bin/docker
EOF

USER libcxx-builder
WORKDIR /home/libcxx-builder

Expand Down
2 changes: 1 addition & 1 deletion libcxx/utils/ci/vendor/android/run-buildbot-container
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ if [ -S /var/run/docker.sock ]; then
DOCKER_OPTIONS+=(--volume /var/run/docker.sock:/var/run/docker.sock)
fi

docker run "${DOCKER_OPTIONS[@]}" libcxx-builder-android \
docker run "${DOCKER_OPTIONS[@]}" ghcr.io/libcxx/android-buildkite-builder \
bash -c 'git config --global --add safe.directory /llvm; (/opt/android/container-setup.sh && exec bash)'
Loading