Skip to content

Commit

Permalink
Add codespeed for continuous (and sudoku)
Browse files Browse the repository at this point in the history
While pubgrub's performance is critical for both uv and cargo, there currently no benchmarking happening in the repository. [Codspeed](https://codspeed.io) runs our benchmarks with instruction counting, reporting more numbers with less variance than wall time.

We get feedback on every PR, can see trends over time and there are flamegraphs and flamegraph diffs in the web view. We've made good experiences with it in both ruff and uv.

With codspeed installed, we can start adding real-world benchmarks for uv and cargo to the pubgrub repo, and then optimize those.
  • Loading branch information
konstin committed Nov 1, 2024
1 parent 3fab343 commit 05f7395
Show file tree
Hide file tree
Showing 4 changed files with 327 additions and 18 deletions.
29 changes: 29 additions & 0 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Benchmarks (CodSpeed)

on:
push:
branches:
- main
pull_request:
workflow_dispatch:

jobs:
benchmarks:
name: Run benchmarks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup rust toolchain, cache and cargo-codspeed binary
uses: moonrepo/setup-rust@v1
with:
channel: stable
cache-target: release
bins: cargo-codspeed

- name: Build the benchmark target(s)
run: cargo codspeed build

- name: Run the benchmarks
uses: CodSpeedHQ/action@v3
with:
run: cargo codspeed run
133 changes: 116 additions & 17 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ thiserror = "1.0"
version-ranges = { version = "0.1.0", path = "version-ranges" }

[dev-dependencies]
criterion = "0.5"
criterion = { version = "2.7.2", package = "codspeed-criterion-compat" }
env_logger = "0.11.5"
proptest = "1.5.0"
ron = "=0.9.0-alpha.0"
Expand All @@ -46,3 +46,7 @@ serde = ["dep:serde", "version-ranges/serde"]
name = "large_case"
harness = false
required-features = ["serde"]

[[bench]]
name = "sudoku"
harness = false
Loading

0 comments on commit 05f7395

Please sign in to comment.