Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release workflow #961

Merged
merged 13 commits into from
Jan 18, 2024
478 changes: 478 additions & 0 deletions .github/workflows/build-binaries.yml

Large diffs are not rendered by default.

File renamed without changes.
33 changes: 33 additions & 0 deletions .github/workflows/publish-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Publish a release to PyPI.
#
# Assumed to run as a subworkflow of .github/workflows/release.yml; specifically, as a publish job
# within `cargo-dist`.
name: "Publish to PyPI"

on:
workflow_call:
inputs:
plan:
required: true
type: string

jobs:
pypi-publish:
name: Upload to PyPI
runs-on: ubuntu-latest
environment:
name: release
permissions:
# For PyPI's trusted publishing.
id-token: write
steps:
- uses: actions/download-artifact@v3
with:
name: wheels
path: wheels
- name: Publish to PyPi
uses: pypa/gh-action-pypi-publish@release/v1
with:
skip-existing: true
packages-dir: wheels
verbose: true
220 changes: 220 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
# Copyright 2022-2023, axodotdev
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Generated file.)

# SPDX-License-Identifier: MIT or Apache-2.0
#
# CI that:
#
# * checks for a Git Tag that looks like a release
# * builds artifacts with cargo-dist (archives, installers, hashes)
# * uploads those artifacts to temporary workflow zip
# * on success, uploads the artifacts to a Github Release
#
# Note that the Github Release will be created with a generated
# title/body based on your changelogs.

name: Release

permissions:
contents: write

# This task will run whenever you workflow_dispatch with a tag that looks like a version
# like "1.0.0", "v0.1.0-prerelease.1", "my-app/0.1.0", "releases/v1.0.0", etc.
# Various formats will be parsed into a VERSION and an optional PACKAGE_NAME, where
# PACKAGE_NAME must be the name of a Cargo package in your workspace, and VERSION
# must be a Cargo-style SemVer Version (must have at least major.minor.patch).
#
# If PACKAGE_NAME is specified, then the announcement will be for that
# package (erroring out if it doesn't have the given version or isn't cargo-dist-able).
#
# If PACKAGE_NAME isn't specified, then the announcement will be for all
# (cargo-dist-able) packages in the workspace with that version (this mode is
# intended for workspaces with only one dist-able package, or with all dist-able
# packages versioned/released in lockstep).
#
# If you push multiple tags at once, separate instances of this workflow will
# spin up, creating an independent announcement for each one. However Github
# will hard limit this to 3 tags per commit, as it will assume more tags is a
# mistake.
#
# If there's a prerelease-style suffix to the version, then the release(s)
# will be marked as a prerelease.
on:
workflow_dispatch:
inputs:
tag:
description: Release Tag
required: true
default: dry-run
type: string

jobs:
# Run 'cargo dist plan' (or host) to determine what tasks we need to do
plan:
runs-on: ubuntu-latest
outputs:
val: ${{ steps.plan.outputs.manifest }}
tag: ${{ (inputs.tag != 'dry-run' && inputs.tag) || '' }}
tag-flag: ${{ inputs.tag && inputs.tag != 'dry-run' && format('--tag={0}', inputs.tag) || '' }}
publishing: ${{ inputs.tag && inputs.tag != 'dry-run' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install cargo-dist
# we specify bash to get pipefail; it guards against the `curl` command
# failing. otherwise `sh` won't catch that `curl` returned non-0
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.7.3-prerelease.3/cargo-dist-installer.sh | sh"
# sure would be cool if github gave us proper conditionals...
# so here's a doubly-nested ternary-via-truthiness to try to provide the best possible
# functionality based on whether this is a pull_request, and whether it's from a fork.
# (PRs run on the *source* but secrets are usually on the *target* -- that's *good*
# but also really annoying to build CI around when it needs secrets to work right.)
- id: plan
run: |
cargo dist ${{ inputs.tag && (inputs.tag != 'dry-run' && format('host --steps=create --tag={0}', inputs.tag) || format('host --steps=create', inputs.tag)) || (github.event.pull_request.head.repo.fork && 'plan' || 'host --steps=check') }} --output-format=json > dist-manifest.json
echo "cargo dist ran successfully"
cat dist-manifest.json
echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT"
- name: "Upload dist-manifest.json"
uses: actions/upload-artifact@v3
with:
name: artifacts
path: dist-manifest.json

custom-build-binaries:
needs:
- plan
if: ${{ needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload' || inputs.tag == 'dry-run' }}
uses: ./.github/workflows/build-binaries.yml
with:
plan: ${{ needs.plan.outputs.val }}
secrets: inherit

# Build and package all the platform-agnostic(ish) things
build-global-artifacts:
needs:
- plan
- custom-build-binaries
runs-on: "ubuntu-20.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install cargo-dist
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.7.3-prerelease.3/cargo-dist-installer.sh | sh"
# Get all the local artifacts for the global tasks to use (for e.g. checksums)
- name: Fetch local artifacts
uses: actions/download-artifact@v3
with:
name: artifacts
path: target/distrib/
- id: cargo-dist
shell: bash
run: |
cargo dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json
echo "cargo dist ran successfully"

# Parse out what we just built and upload it to scratch storage
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
jq --raw-output ".artifacts[]?.path | select( . != null )" dist-manifest.json >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"

cp dist-manifest.json "$BUILD_MANIFEST_NAME"
- name: "Upload artifacts"
uses: actions/upload-artifact@v3
with:
name: artifacts
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Determines if we should publish/announce
host:
needs:
- plan
- build-global-artifacts
# Only run if we're "publishing", and only if local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-20.04"
outputs:
val: ${{ steps.host.outputs.manifest }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install cargo-dist
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.7.3-prerelease.3/cargo-dist-installer.sh | sh"
# Fetch artifacts from scratch-storage
- name: Fetch artifacts
uses: actions/download-artifact@v3
with:
name: artifacts
path: target/distrib/
# This is a harmless no-op for Github Releases, hosting for that happens in "announce"
- id: host
shell: bash
run: |
cargo dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json
echo "artifacts uploaded and released successfully"
cat dist-manifest.json
echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT"
- name: "Upload dist-manifest.json"
uses: actions/upload-artifact@v3
with:
name: artifacts
path: dist-manifest.json

custom-publish-pypi:
needs:
- plan
- host
if: ${{ !fromJson(needs.plan.outputs.val).announcement_is_prerelease || fromJson(needs.plan.outputs.val).publish_prereleases }}
uses: ./.github/workflows/publish-pypi.yml
with:
plan: ${{ needs.plan.outputs.val }}
secrets: inherit
# publish jobs get escalated permissions
permissions:
id-token: write
packages: write

# Create a Github Release while uploading all files to it
announce:
needs:
- plan
- host
- custom-publish-pypi
# use "always() && ..." to allow us to wait for all publish jobs while
# still allowing individual publish jobs to skip themselves (for prereleases).
# "host" however must run to completion, no skipping allowed!
if: ${{ always() && needs.host.result == 'success' && (needs.custom-publish-pypi.result == 'skipped' || needs.custom-publish-pypi.result == 'success') }}
runs-on: "ubuntu-20.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: "Download Github Artifacts"
uses: actions/download-artifact@v3
with:
name: artifacts
path: artifacts
- name: Cleanup
run: |
# Remove the granular manifests
rm -f artifacts/*-dist-manifest.json
- name: Create Github Release
uses: ncipollo/release-action@v1
with:
tag: ${{ needs.plan.outputs.tag }}
name: ${{ fromJson(needs.host.outputs.val).announcement_title }}
body: ${{ fromJson(needs.host.outputs.val).announcement_github_body }}
prerelease: ${{ fromJson(needs.host.outputs.val).announcement_is_prerelease }}
artifacts: "artifacts/*"
42 changes: 38 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.74"
homepage = "https://astral.sh"
documentation = "https://astral.sh"
repository = "https://github.com/astral-sh/puffin"
authors = ["Astral Software Inc. <[email protected]>"]
homepage = "https://pypi.org/project/puffin-alpha/"
documentation = "https://pypi.org/project/puffin-alpha/"
repository = "https://pypi.org/project/puffin-alpha/"
authors = ["Puffin"]
license = "MIT OR Apache-2.0"

[workspace.dependencies]
Expand Down Expand Up @@ -133,3 +133,37 @@ rest_pat_in_fully_bound_structs = "warn"
[profile.profiling]
inherits = "release"
debug = true

# The profile that 'cargo dist' will build with.
[profile.dist]
inherits = "release"
lto = "thin"

# Config for 'cargo dist'
[workspace.metadata.dist]
# The preferred cargo-dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.7.3-prerelease.3"
# CI backends to support
ci = ["github"]
# The installers to generate for each app
installers = ["shell", "powershell"]
# The archive format to use for windows builds (defaults .zip)
windows-archive = ".zip"
# The archive format to use for non-windows builds (defaults .tar.xz)
unix-archive = ".tar.gz"
# Target platforms to build apps for (Rust target-triple syntax)
targets = ["aarch64-unknown-linux-gnu", "x86_64-unknown-linux-gnu", "i686-unknown-linux-gnu", "aarch64-apple-darwin", "x86_64-apple-darwin", "aarch64-unknown-linux-musl", "x86_64-unknown-linux-musl", "i686-unknown-linux-musl", "aarch64-pc-windows-msvc", "x86_64-pc-windows-msvc", "i686-pc-windows-msvc", "armv7-unknown-linux-gnueabihf", "powerpc64-unknown-linux-gnu", "powerpc64le-unknown-linux-gnu", "s390x-unknown-linux-gnu"]
# Whether to auto-include files like READMEs and CHANGELOGs (default true)
auto-includes = false
# Whether cargo-dist should create a Github Release or use an existing draft
create-release = true
# Publish jobs to run in CI
pr-run-mode = "skip"
# Whether CI should trigger releases with dispatches instead of tag pushes
dispatch-releases = true
# Whether CI should include auto-generated code to build local artifacts
build-local-artifacts = false
# Local artifacts jobs to run in CI
local-artifacts-jobs = ["./build-binaries"]
# Publish jobs to run in CI
publish-jobs = ["./publish-pypi"]
2 changes: 1 addition & 1 deletion crates/distribution-types/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ distribution-filename = { path = "../distribution-filename", features = ["serde"
pep440_rs = { path = "../pep440-rs" }
pep508_rs = { path = "../pep508-rs" }
platform-tags = { path = "../platform-tags" }
puffin-git = { path = "../puffin-git" }
puffin-git = { path = "../puffin-git", features = ["vendored-openssl"] }
puffin-normalize = { path = "../puffin-normalize" }
pypi-types = { path = "../pypi-types" }
requirements-txt = { path = "../requirements-txt" }
Expand Down
2 changes: 1 addition & 1 deletion crates/pep440-rs/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "pep440_rs"
version = "0.3.12"
description = "A library for python version numbers and specifiers, implementing PEP 440"
license = "Apache-2.0 OR BSD-2-Clause"
include = ["/src", "Changelog.md", "License-Apache", "License-BSD", "Readme.md", "pyproject.toml"]
include = ["/src", "Changelog.md", "License-Apache", "License-BSD", "README.md", "pyproject.toml"]

edition = { workspace = true }
rust-version = { workspace = true }
Expand Down
2 changes: 1 addition & 1 deletion crates/puffin-distribution/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ puffin-cache = { path = "../puffin-cache" }
puffin-client = { path = "../puffin-client" }
puffin-extract = { path = "../puffin-extract" }
puffin-fs = { path = "../puffin-fs" }
puffin-git = { path = "../puffin-git" }
puffin-git = { path = "../puffin-git", features = ["vendored-openssl"] }
puffin-normalize = { path = "../puffin-normalize" }
puffin-traits = { path = "../puffin-traits" }
pypi-types = { path = "../pypi-types" }
Expand Down
2 changes: 1 addition & 1 deletion crates/puffin-installer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ puffin-client = { path = "../puffin-client" }
puffin-distribution = { path = "../puffin-distribution" }
puffin-extract = { path = "../puffin-extract" }
puffin-fs = { path = "../puffin-fs" }
puffin-git = { path = "../puffin-git" }
puffin-git = { path = "../puffin-git", features = ["vendored-openssl"] }
puffin-interpreter = { path = "../puffin-interpreter" }
puffin-normalize = { path = "../puffin-normalize" }
puffin-traits = { path = "../puffin-traits" }
Expand Down
2 changes: 1 addition & 1 deletion crates/puffin-resolver/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ platform-tags = { path = "../platform-tags" }
puffin-cache = { path = "../puffin-cache" }
puffin-client = { path = "../puffin-client" }
puffin-distribution = { path = "../puffin-distribution" }
puffin-git = { path = "../puffin-git" }
puffin-git = { path = "../puffin-git", features = ["vendored-openssl"] }
puffin-interpreter = { path = "../puffin-interpreter" }
puffin-normalize = { path = "../puffin-normalize" }
puffin-traits = { path = "../puffin-traits" }
Expand Down
20 changes: 20 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[build-system]
requires = ["maturin>=1.0,<2.0"]
build-backend = "maturin"

[project]
name = "puffin-alpha"
Copy link
Member Author

@charliermarsh charliermarsh Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this would publish a binary named puffin to the PyPI package puffin-alpha. (puffin is taken, and it doesn't really matter what we use in the interim.)

version = "0.0.1"
authors = [{ name = "Puffin" }]
requires-python = ">=3.7"
keywords = []
classifiers = []
readme = "README.md"

[tool.maturin]
bindings = "bin"
manifest-path = "crates/puffin-cli/Cargo.toml"
module-name = "puffin"
python-source = "python"
strip = true
include = ["rust-toolchain.toml"]
Empty file added python/puffin/__init__.py
Empty file.
Loading