Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build release binaries for multiple targets #98431

Merged
merged 5 commits into from
Jul 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/release-binaries-all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Release Binaries All

permissions:
contents: read # Default everything to read-only

on:
workflow_dispatch:
inputs:
release-version:
description: 'Release Version'
required: true
type: string
upload:
description: 'Upload binaries to the release page'
required: true
default: false
type: boolean

workflow_call:
inputs:
release-version:
description: 'Release Version'
required: true
type: string
upload:
description: 'Upload binaries to the release page'
required: true
default: false
type: boolean

pull_request:
types:
- opened
- synchronize
- reopened
# When a PR is closed, we still start this workflow, but then skip
# all the jobs, which makes it effectively a no-op. The reason to
# do this is that it allows us to take advantage of concurrency groups
# to cancel in progress CI jobs whenever the PR is closed.
- closed
paths:
- '.github/workflows/release-binaries-all.yml'
- '.github/workflows/release-binaries.yml'
- '.github/workflows/release-binaries-setup-stage/*'
- '.github/workflows/release-binaries-save-stage/*'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || 'dispatch' }}
cancel-in-progress: True

jobs:
setup-variables:
if: >-
(github.event_name != 'pull_request' || github.event.action != 'closed')
runs-on: ubuntu-22.04
outputs:
release-version: ${{ steps.vars.outputs.release-version }}
upload: ${{ steps.vars.outputs.upload }}
steps:
- shell: bash
id: vars
run: |
upload="${{ inputs.upload }}"
release_version="${{ inputs.release-version }}"
if [ "${{ github.event_name }}" = "pull_request" ]; then
upload="false"
release_version=""
fi
echo "release-version=$release_version" >> "$GITHUB_OUTPUT"
echo "upload=$upload" >> "$GITHUB_OUTPUT"

release-binaries-all:
name: Build Release Binaries
needs:
- setup-variables
permissions:
contents: write # For release uploads
id-token: write # For artifact attestations
attestations: write # For artifact attestations
strategy:
fail-fast: false
matrix:
runs-on:
- ubuntu-22.04
- windows-2022
- macos-13
- macos-14

uses: ./.github/workflows/release-binaries.yml
with:
release-version: "${{ needs.setup-variables.outputs.release-version }}"
upload: ${{ needs.setup-variables.outputs.upload == 'true'}}
runs-on: "${{ matrix.runs-on }}"

38 changes: 38 additions & 0 deletions .github/workflows/release-binaries-save-stage/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Save Stage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get a comment somewhere in here about what this exactly is used for? Something like the description flag in the setup-stage action below?

description: >-
Upload the source and binary directories from a build stage so that they
can be re-used in the next stage. This action is used to the release
binaries workflow into multiple stages to avoid the 6 hour timeout on
the GitHub hosted runners.
inputs:
build-prefix:
description: "Directory containing the build directory."
required: true
type: 'string'

runs:
using: "composite"
steps:
# We need to create an archive of the build directory, because it has too
# many files to upload.
- name: Package Build and Source Directories
shell: bash
run: |
# Windows does not support symlinks, so we need to dereference them.
tar --exclude build/ ${{ (runner.os == 'Windows' && '-h') || '' }} -c . | zstd -T0 -c > ../llvm-project.tar.zst
mv ../llvm-project.tar.zst .
tar -C ${{ inputs.build-prefix }} -c build/ | zstd -T0 -c > build.tar.zst

- name: Upload Stage 1 Source
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
with:
name: ${{ runner.os }}-${{ runner.arch }}-${{ github.job }}-source
path: llvm-project.tar.zst
retention-days: 2

- name: Upload Stage 1 Build Dir
uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
with:
name: ${{ runner.os}}-${{ runner.arch }}-${{ github.job }}-build
path: build.tar.zst
retention-days: 2
59 changes: 59 additions & 0 deletions .github/workflows/release-binaries-setup-stage/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Setup Stage
description: >-
Setup the next stage of the release binaries workflow. This sets up the
environment correctly for a new stage of the release binaries workflow
and also restores the source and build directory from the previous stage.
inputs:
previous-artifact:
description: >-
A unique descriptor for the artifact from the previous stage. This will
be used to construct the final artifact pattern, which is:
$RUNNER_OS-$RUNNER_ARCH-$PREVIOUS_ARTIFACT-*
required: false
type: 'string'

outputs:
build-prefix:
description: "Directory containing the build directory."
value: ${{ steps.build-prefix.outputs.build-prefix }}

runs:
using: "composite"
steps:
- name: Install Ninja
uses: llvm/actions/install-ninja@22e9f909d35b50bd1181709564bfe816eaeaae81 # main
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: Is there a reason these actions are still in a separate repo and not in the mono-repo?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only advantage of having them in a separate repo is that you can use them without checking out the project source code. However, for this specific action I don't think it would ever be used without already having the source code, so there is probably no reason to keep it here.


- name: Setup Windows
if: startsWith(runner.os, 'Windows')
uses: llvm/actions/setup-windows@main
with:
arch: amd64

- name: Set Build Prefix
id: build-prefix
shell: bash
run: |
build_prefix=`pwd`
if [ "${{ runner.os }}" = "Linux" ]; then
sudo chown $USER:$USER /mnt/
build_prefix=/mnt/
fi
echo "build-prefix=$build_prefix" >> $GITHUB_OUTPUT
- name: Download Previous Stage Artifact
if: ${{ inputs.previous-artifact }}
id: download
uses: actions/download-artifact@6b208ae046db98c579e8a3aa621ab581ff575935 # v4.1.1
with:
pattern: ${{ runner.os }}-${{ runner.arch }}-${{ inputs.previous-artifact }}-*
merge-multiple: true

- name: Unpack Artifact
if: ${{ steps.download.outputs.download-path }}
shell: bash
run: |
tar --zstd -xf llvm-project.tar.zst
rm llvm-project.tar.zst
tar --zstd -C ${{ steps.build-prefix.outputs.build-prefix}} -xf build.tar.zst
rm build.tar.zst
Loading
Loading