Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[libc++] Unify the benchmarks with the test suite #101399

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[libc++] Unify the benchmarks with the test suite #101399

wants to merge 4 commits into from
+322 −454

Commits on Oct 30, 2024

  1. [libc++] Make benchmarks forward-compatible with the test suite 

    This patch fixes a few warnings and errors when running the benchmarks
as part of the test suite. It also adds most of the necessary Lit
annotations to make it pass on various configurations.
    @ldionne
    ldionne committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    a50fa3f View commit details
    Browse the repository at this point in the history

  2. [libc++] Unify the benchmarks with the test suite 

    Instead of building the benchmarks separately via CMake and running them
separately from the test suite, this patch merges the benchmarks into
the test suite and handles both uniformly.

As a result:
- It is now possible to run individual benchmarks like we run tests
  (e.g. using libcxx-lit), which is a huge quality-of-life improvement.

- The benchmarks will be run under exactly the same configuration as
  the rest of the tests, which is a nice simplification. This does
  mean that one has to be careful to enable the desired optimization
  flags when running benchmarks, but that is easy with e.g.
  `libcxx-lit <...> --param optimization=speed`.

- Benchmarks can use the same annotations as the rest of the test
  suite, such as `// UNSUPPORTED` & friends.

When running the tests via `check-cxx`, we only compile the benchmarks
because running them would be too time consuming. This introduces a bit
of complexity in the testing setup, and instead it would be better to
allow passing a --dry-run flag to GoogleBenchmark executables, which is
the topic of a GoogleBenchmark issue.

I am not really satisfied with the layering violation of adding the
%{benchmark_flags} substitution to cmake-bridge, however I believe
this can be improved in the future.
    @ldionne
    ldionne committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    24d3c2b View commit details
    Browse the repository at this point in the history

  3. Upload additional artifacts to debug configuration issues

    @ldionne
    ldionne committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    2128aa5 View commit details
    Browse the repository at this point in the history

  4. Pass additional libraries to google benchmark

    @ldionne
    ldionne committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    55800ff View commit details
    Browse the repository at this point in the history