Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Github] Build PGO optimized toolchain in container #80096

Merged
merged 1 commit into from
Jan 31, 2024

Conversation

boomanaiden154
Copy link
Contributor

This patch adjusts the Docker container intended for CI use to contain a PGO+ThinLTO+BOLT optimized clang. The toolchain is built within a Github action and takes ~3.5 hours. No caching is utilized. The current PGO optimization is fairly minimal, only running clang over hello world. This can be adjusted as needed.

This patch adjusts the Docker container intended for CI use to contain a
PGO+ThinLTO+BOLT optimized clang. The toolchain is built within a Github
action and takes ~3.5 hours. No caching is utilized. The current PGO
optimization is fairly minimal, only running clang over hello world.
This can be adjusted as needed.
@llvmbot
Copy link
Collaborator

llvmbot commented Jan 31, 2024

@llvm/pr-subscribers-github-workflow

Author: Aiden Grossman (boomanaiden154)

Changes

This patch adjusts the Docker container intended for CI use to contain a PGO+ThinLTO+BOLT optimized clang. The toolchain is built within a Github action and takes ~3.5 hours. No caching is utilized. The current PGO optimization is fairly minimal, only running clang over hello world. This can be adjusted as needed.


Full diff: https://github.com/llvm/llvm-project/pull/80096.diff

1 Files Affected:

  • (modified) .github/workflows/containers/github-action-ci/Dockerfile (+22-16)
diff --git a/.github/workflows/containers/github-action-ci/Dockerfile b/.github/workflows/containers/github-action-ci/Dockerfile
index d91a7ad3a9d06..6c8a2045bbdc2 100644
--- a/.github/workflows/containers/github-action-ci/Dockerfile
+++ b/.github/workflows/containers/github-action-ci/Dockerfile
@@ -2,29 +2,35 @@ FROM docker.io/library/ubuntu:22.04 as base
 ENV LLVM_SYSROOT=/opt/llvm/
 
 FROM base as toolchain
-ENV LLVM_MAJOR=17
-ENV LLVM_VERSION=${LLVM_MAJOR}.0.6
-ENV LLVM_DIRNAME=clang+llvm-${LLVM_VERSION}-x86_64-linux-gnu-ubuntu-22.04
-ENV LLVM_FILENAME=${LLVM_DIRNAME}.tar.xz
+ENV LLVM_VERSION=17.0.6
 
 RUN apt-get update && \
     apt-get install -y \
-    curl \
-    xz-utils
+    wget \
+    gcc \
+    g++ \
+    cmake \
+    ninja-build \
+    python3
+
+RUN wget https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-$LLVM_VERSION.tar.gz && tar -xf llvmorg-$LLVM_VERSION.tar.gz
 
-RUN mkdir -p $LLVM_SYSROOT/bin/ $LLVM_SYSROOT/lib/
+WORKDIR /llvm-project-llvmorg-$LLVM_VERSION
 
-RUN curl -O -L https://github.com/llvm/llvm-project/releases/download/llvmorg-$LLVM_VERSION/$LLVM_FILENAME
+RUN mkdir build
 
-RUN tar -C $LLVM_SYSROOT --strip-components=1 -xJf $LLVM_FILENAME \
-    $LLVM_DIRNAME/bin/clang \
-    $LLVM_DIRNAME/bin/clang++ \
-    $LLVM_DIRNAME/bin/clang-cl \
-    $LLVM_DIRNAME/bin/clang-$LLVM_MAJOR \
-    $LLVM_DIRNAME/bin/lld \
-    $LLVM_DIRNAME/bin/ld.lld \
-    $LLVM_DIRNAME/lib/clang/
+RUN cmake -B ./build -G Ninja ./llvm \
+  -C ./clang/cmake/caches/BOLT-PGO.cmake \
+  -DBOOTSTRAP_LLVM_ENABLE_LLD=ON \
+  -DBOOTSTRAP_BOOTSTRAP_LLVM_ENABLE_LLD=ON \
+  -DPGO_INSTRUMENT_LTO=Thin \
+  -DLLVM_ENABLE_RUNTIMES="compiler-rt" \
+  -DCMAKE_INSTALL_PREFIX="$LLVM_SYSROOT" \
+  -DLLVM_ENABLE_PROJECTS="bolt;clang;lld;clang-tools-extra" \
+  -DLLVM_DISTRIBUTION_COMPONENTS="lld;compiler-rt;clang-format" \
+  -DCLANG_DEFAULT_LINKER="lld"
 
+RUN ninja -C ./build stage2-install-distribution && ninja -C ./build install-distribution && rm -rf ./build
 
 FROM base
 

@boomanaiden154
Copy link
Contributor Author

The container ends up being 808MB fully unpacked. It's only a couple hundred MB in download size (not sure what the full one is currently). I think that should probably be fine. I've seen pretty good performance improvements with just this configuration (~20% when I last tested it), but it's not the theoretical max. I think we can land this, start moving jobs over to the container, and then once some other patches land, we can do better perf-training and see what the results are like.

@tstellar
Copy link
Collaborator

#78879 should make it possible to optimize the build further.

@boomanaiden154
Copy link
Contributor Author

#78879 should make it possible to optimize the build further.

Yep. My plan is to adjust this script once that lands and do some proper compile time experiments.

@boomanaiden154 boomanaiden154 merged commit 9107904 into llvm:main Jan 31, 2024
6 checks passed
carlosgalvezp pushed a commit to carlosgalvezp/llvm-project that referenced this pull request Feb 1, 2024
This patch adjusts the Docker container intended for CI use to contain a
PGO+ThinLTO+BOLT optimized clang. The toolchain is built within a Github
action and takes ~3.5 hours. No caching is utilized. The current PGO
optimization is fairly minimal, only running clang over hello world.
This can be adjusted as needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants