Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix bench-e2e single mode and keep results #1693

Merged
merged 16 commits into from
Oct 10, 2024
Merged

Fix bench-e2e single mode and keep results #1693

merged 16 commits into from
Oct 10, 2024
+384 −419

Commits on Oct 10, 2024

  1. Fix hydra-cluster bench single by setting a hard-coded fee 

    This is not ideal, but a lot simpler than doing proper fee calculation.
It's unclear why fee calculation was removed before, it is needed when
running benchmark scenarios.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    dbc0d84 View commit details
    Browse the repository at this point in the history

  2. Remove redundant bench-e2e mode of single + workdir set 

    This is redundant and can be achieved by using the 'datasets'
subcommand.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    ea1d38f View commit details
    Browse the repository at this point in the history

  3. Write generated dataset to outputDirectory if given 

    Before it was written to a random temporary directory, which makes it
annoying to generate datasets with this mode.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    d905ddb View commit details
    Browse the repository at this point in the history

  4. Write results.csv in --output-directory 

    They hydra-cluster benchmarks now only uses a single directory to store
the whole state, which is temporary unless a specific output-directory
is requested.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    8f878a0 View commit details
    Browse the repository at this point in the history

  5. Small module re-org in HydraNode.hs

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    2b1dfe5 View commit details
    Browse the repository at this point in the history

  6. Drop need of party in benchmark scenario

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    215bc71 View commit details
    Browse the repository at this point in the history

  7. Use a bit more non empty lists in hydra-cluster

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    46ee433 View commit details
    Browse the repository at this point in the history

  8. Switch to only use self-transfers in benchmarks 

    This reduces some code duplication without much loss of
expressiveness (which key we use does not matter).
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    a6f74a6 View commit details
    Browse the repository at this point in the history

  9. Separate client keys and hydra node keys

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    f4733d4 View commit details
    Browse the repository at this point in the history

  10. Fail if output directory is not empty

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    2c5d063 View commit details
    Browse the repository at this point in the history

  11. Update datasets with new format 

    Same transaction style (single repending txs), but deliberately smaller
length of transactions (3000 -> 300) to have shorter benchmark
run-times, while sequence should be long enough to identify regressions.

Generated with invocations:

cabal run bench-e2e -- single --cluster-size 1 --scaling-factor 10

and

cabal run bench-e2e -- single --cluster-size 3 --scaling-factor 10

Plus some manual amending of the JSON to contain a "title".
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    66a2b18 View commit details
    Browse the repository at this point in the history

  12. Start with an empty output directory

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    9317e88 View commit details
    Browse the repository at this point in the history

  13. Only show quantiles if they can be computed

    @noonio @ch1bo
    noonio authored and ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    b0bade5 View commit details
    Browse the repository at this point in the history

  14. Use and seed hydraNodeKeys in demo mode 

    As before, the bench-e2e does not assume the hydra node keys to be
seeded. This ties the way bench-e2e binary (which hard-codes Alice, Bob
and Carol) to the configurable list of --hydra-client to connect to.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    5a4718c View commit details
    Browse the repository at this point in the history

  15. Use seed-devnet and not assume node keys in bench-e2e 

    This decouples the bench-e2e binary which just produces load and
provides statistics more from how the hydra-nodes are run.

Now the only assumption is that the
'hydra-cluster/config/credentials/faucet.sk' owns funds on the given
network.
    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    a48d056 View commit details
    Browse the repository at this point in the history

  16. Add a comment to the benchmark scenario

    @ch1bo
    ch1bo committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    ec21ac0 View commit details
    Browse the repository at this point in the history