Skip to content

Commit

Permalink
Cargo.toml, .github/workflows/*: Separate profiles into dev, `f…
Browse files Browse the repository at this point in the history
…ast-dev`, `release`, `checked-release` and update CI accordingly.

Previously, `profile.dev` had `opt-level = 1` for fast development,
but that's bad for debugging (`profile.dev` is also confusingly `debug`).
This separates the two into `profile.dev`/`debug` for debugging
and `profile.fast-dev` for the fastest local development cycle.

This also splits up `profile.release` into `profile.release`
and `profile.checked-release`, which is `profile.release`
with runtime checks and minimal debug info.
This is run in CI now (as opposed to a mix of `profile.dev` and `profile.release`),
except for `profile.release`, which is used for benchmarking.

This also consolidates some of the GitHub Actions workflows
to more easily manage the profiles, features, as some things were missing.
Specifically,
* argon tests were not run with `debug_assertions`
* arm tests were not run with `debug_assertions` or `overflow_checks`
  • Loading branch information
kkysen committed Dec 6, 2023
1 parent 3bc2e0b commit 2f05cdd
Show file tree
Hide file tree
Showing 6 changed files with 158 additions and 156 deletions.
61 changes: 0 additions & 61 deletions .github/workflows/build-and-test-aarch64.yml

This file was deleted.

95 changes: 95 additions & 0 deletions .github/workflows/build-and-test-arm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: build and test on arm with QEMU
on: [push, pull_request]
jobs:
test-on-qemu:
strategy:
matrix:
target: [
{
rust: "aarch64-unknown-linux-gnu",
gcc: "g++-aarch64-linux-gnu",
libc_cross: "libc6-dev-arm64-cross",
docker_image: "debian-bullseye-aarch64",
docker_platform: "linux/arm64",
},
{
rust: "armv7-unknown-linux-gnueabihf",
gcc: "g++-arm-linux-gnueabihf",
libc_cross: "libc6-dev-armhf-cross",
docker_image: "debian-bullseye-arm7",
docker_platform: "linux/arm/v7",
},
]
build: [
{profile: "checked-release", features: "asm", timeout_multiplier: 2},
# The asm fallback "should" be the same on all platforms,
# so if we tested it for x86, we shouldn't need to for arm either
# (which is slower due to qemu).
# {profile: "checked-release", features: "" , timeout_multiplier: 2},
]
runs-on: ubuntu-latest
name: meson tests on ${{ matrix.target.rust }} (qemu) with --features ${{ matrix.build.features }}
steps:
- name: install prerequisites
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: ${{ matrix.target.gcc }} ${{ matrix.target.libc_cross }}
version: 1.0 # version of cache to load
- name: git checkout
uses: actions/checkout@v3
with:
fetch-depth: 1
- name: set up qemu
uses: docker/setup-qemu-action@v2
with:
platforms: arm64
- name: cache rust dependencies
uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ matrix.target.rust }}-cargo-and-target-${{ hashFiles('**/Cargo.lock') }}
# - name: cache dav1d object files
# uses: actions/cache@v3
# with:
# path: build/
# key: ${{ matrix.target.rust }}-c-object-files-${{ hashFiles('**/meson.build', '**/*.c', '**/*.h') }}
- name: cargo build
run: |
rustup target add ${{ matrix.target.rust }}
cargo build \
--target ${{ matrix.target.rust }} \
--profile ${{ matrix.build.profile }} \
--no-default-features --features=bitdepth_8,bitdepth_16,${{ matrix.build.features }}
- name: docker pull
run: docker pull ghcr.io/immunant/rav1d/${{ matrix.target.docker_image }}:latest
- name: build and run tests in docker
run: |
docker run \
--rm \
-v ~/.cargo/git:/home/prossimo/.cargo/git \
-v ~/.cargo/registry:/home/prossimo/.cargo/registry \
-v $(pwd):/${{ github.workspace }} \
-w ${{ github.workspace }} \
--platform ${{ matrix.target.docker_platform }} \
ghcr.io/immunant/rav1d/${{ matrix.target.docker_image }}:latest \
.github/workflows/test.sh \
-r ../target/${{ matrix.target.rust }}/${{ matrix.build.profile }}/dav1d \
-s ../target/${{ matrix.target.rust }}/${{ matrix.build.profile }}/seek_stress \
-t ${{ matrix.build.timeout_multiplier }}
- name: copy log files
if: ${{ !cancelled() }}
run: |
cp ${{ github.workspace }}/build/meson-logs/testlog.txt \
${{ github.workspace }}/build/meson-logs/testlog-${{ matrix.target }}-${{ matrix.build.features }}.txt
- name: upload build artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v3
with:
name: meson-test-logs
path: |
${{ github.workspace }}/build/meson-logs/testlog-*.txt
61 changes: 0 additions & 61 deletions .github/workflows/build-and-test-arm7.yml

This file was deleted.

25 changes: 12 additions & 13 deletions .github/workflows/build-and-test-x86-extra.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,14 @@ jobs:
strategy:
matrix:
target: [
"x86_64-unknown-linux-gnu"
"x86_64-unknown-linux-gnu",
]
build: [ # we do release with overflow checks since debug takes forever
# release build with overflow checks without optimized assembly routines
{name: "release", flags: "--release --no-default-features --features=bitdepth_8,bitdepth_16"},
# release build with overflow checks and optimized assembly routines
{name: "release", flags: "--release"}
build: [
{profile: "checked-release", features: "asm"},
{profile: "checked-release", features: ""},
]
runs-on: ubuntu-latest
name: argon tests on ${{ matrix.target }} with --features ${{ matrix.build.features }}
steps:
- name: install prerequisites
uses: awalsh128/cache-apt-pkgs-action@latest
Expand Down Expand Up @@ -58,12 +57,13 @@ jobs:
key: argon_coveragetool_av1_base_and_extended_profiles_v2.1.1.zip
restore-keys: | # it is perfectly fine to restore from other branches
argon_coveragetool_av1_base_and_extended_profiles_v2.1.1.zip
- name: cargo build for ${{ matrix.target }} ${{ matrix.build.name }}
- name: cargo build
run: |
rustup target add ${{ matrix.target }}
cargo build --target ${{ matrix.target }} ${{ matrix.build.flags }}
env:
RUSTFLAGS: "-C overflow-checks=on"
cargo build \
--target ${{ matrix.target }} \
--profile ${{ matrix.build.profile }} \
--no-default-features --features=bitdepth_8,bitdepth_16,${{ matrix.build.features }}
- name: download, check, and unpack argon test vectors
if: ${{ steps.cache-argon.outputs.cache-hit != 'true' }}
run: | # delete the downloaded .zip to avoid out-of-diskspace errors
Expand All @@ -76,7 +76,6 @@ jobs:
env:
ARGON_URL: https://storage.googleapis.com/downloads.aomedia.org/assets/zip/argon_coveragetool_av1_base_and_extended_profiles_v2.1.1.zip
ARGON_URL_MD5: https://storage.googleapis.com/downloads.aomedia.org/assets/zip/argon_coveragetool_av1_base_and_extended_profiles_v2.1.1.zip.md5sum
- name: run argon tests for ${{ matrix.target }} ${{ matrix.build.name }}
- name: run argon tests
run: |
tests/dav1d_argon.bash -d target/${{ matrix.target }}/${{ matrix.build.name }}/dav1d -j `nproc`
tests/dav1d_argon.bash -d target/${{ matrix.target }}/${{ matrix.build.profile }}/dav1d -j $(nproc)
36 changes: 19 additions & 17 deletions .github/workflows/build-and-test-x86.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,14 @@ jobs:
matrix:
target: [
"x86_64-unknown-linux-gnu",
"i686-unknown-linux-gnu"
"i686-unknown-linux-gnu",
]
build: [
# release build without optimized assembly routines
{name: "release", flags: "--release --no-default-features --features=bitdepth_8,bitdepth_16", timeout_multiplier: 1},
# debug build to catch overflows with optimized assembly routines
{name: "debug", flags: "", timeout_multiplier: 2}
{profile: "checked-release", features: "asm", timeout_multiplier: 2},
{profile: "checked-release", features: "" , timeout_multiplier: 2},
]
runs-on: ubuntu-latest
name: meson tests on ${{ matrix.target }} with --features ${{ matrix.build.features }}
steps:
- name: install prerequisites
uses: awalsh128/cache-apt-pkgs-action@latest
Expand Down Expand Up @@ -43,22 +42,25 @@ jobs:
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: cargo build for ${{ matrix.target }} ${{ matrix.build.name }}
- name: cargo build
run: |
cargo clean
rustup target add ${{ matrix.target }}
cargo build --target ${{ matrix.target }} ${{ matrix.build.flags }}
- name: meson test for ${{ matrix.target }} ${{ matrix.build.name }}
cargo build \
--target ${{ matrix.target }} \
--profile ${{ matrix.build.profile }} \
--no-default-features --features=bitdepth_8,bitdepth_16,${{ matrix.build.features }}
- name: meson test
run: |
.github/workflows/test.sh \
-r ../target/${{ matrix.target }}/${{ matrix.build.name }}/dav1d \
-s ../target/${{ matrix.target }}/${{ matrix.build.name }}/seek_stress \
-r ../target/${{ matrix.target }}/${{ matrix.build.profile }}/dav1d \
-s ../target/${{ matrix.target }}/${{ matrix.build.profile }}/seek_stress \
-t ${{ matrix.build.timeout_multiplier }}
- name: copy log files
if: ${{ !cancelled() }}
run: |
cp ${{ github.workspace }}/build/meson-logs/testlog.txt \
${{ github.workspace }}/build/meson-logs/testlog-${{ matrix.target }}-${{ matrix.build.name }}.txt
${{ github.workspace }}/build/meson-logs/testlog-${{ matrix.target }}-${{ matrix.build.features }}.txt
- name: upload build artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v3
Expand Down Expand Up @@ -95,18 +97,18 @@ jobs:
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: cargo build for x86_64-apple-darwin
- name: cargo build for x86_64-apple-darwin with --features asm
run: |
cargo build --release
- name: meson test for x86_64-apple-darwin
cargo build --profile checked-release
- name: meson test for x86_64-apple-darwin with --features asm
run: |
.github/workflows/test.sh -r ../target/release/dav1d \
-s ../target/release/seek_stress
.github/workflows/test.sh -r ../target/checked-release/dav1d \
-s ../target/checked-release/seek_stress
- name: copy log files
if: ${{ !cancelled() }}
run: |
cp ${{ github.workspace }}/build/meson-logs/testlog.txt \
${{ github.workspace }}/build/meson-logs/testlog-x86_64-apple-darwin.txt
${{ github.workspace }}/build/meson-logs/testlog-x86_64-apple-darwin-asm.txt
- name: upload build artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v3
Expand Down
36 changes: 32 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,40 @@ asm = []
bitdepth_8 = []
bitdepth_16 = []

# For debugging.
#
# This enables no optimizations, so optimizations can't interfere with debugging,
# but it will probably be very slow.
[profile.dev]
# As we port, we increasingly use operations that require more optimizations to be zero-cost,
# so while `--release` perf stays the same, debug perf suffers a lot,
# unless we enable the basic optimizations (`opt-level = 1`),
# which don't increase compile-time that much.

# Fastest for local development.
#
# `debug`/`dev` builds are very slow*, so this enables some optimizations
# to be fastest for the local development cycle of compiling and running tests.
# Only some optimizations are enabled, so compile time doesn't increase very much.
# Full debug info is still enabled, so this can also be used for debugging,
# but there are still some cases where optimizations can still interfere with debugging.
#
# * Especially as we increasingly use more zero-cost abstractions,
# which are only zero-cost with optimizations.
[profile.fast-dev]
inherits = "dev"
opt-level = 1

# Fastest.
# For release builds.
[profile.release]
codegen-units = 1

# Fast but checked.
# For CI and sometimes local development with qemu.
#
# That is, it compiles with all runtime checks and limited debug info (as opposed to none).
[profile.checked-release]
inherits = "release"
opt-level = 3
debug = "limited"
debug-assertions = true
overflow-checks = true
lto = "thin"
codegen-units = 16 # release default

0 comments on commit 2f05cdd

Please sign in to comment.