Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[benchmarking] Reset to genesis storage after each run #5655

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

ggwpez
Copy link
Member

@ggwpez ggwpez commented Sep 9, 2024

The genesis state is currently partially provided via OverlayedChanges, but these changes are reset by the runtime after the first repetition, causing the second repitition to use an invalid genesis state.

Changes:

  • Provide the genesis state as a Storage without any OverlayedChanges to make it work correctly with repetitions.
  • Add --genesis-builder-preset option to use different genesis preset names.
  • Improve error messages.

@ggwpez ggwpez marked this pull request as draft September 9, 2024 17:37
Signed-off-by: Oliver Tale-Yazdi <[email protected]>
@ggwpez ggwpez added the T1-FRAME This PR/Issue is related to core FRAME, the framework. label Sep 9, 2024
@ggwpez ggwpez marked this pull request as ready for review September 9, 2024 18:10
@ggwpez ggwpez requested a review from a team as a code owner September 9, 2024 18:10
@ggwpez ggwpez requested a review from bkontur September 9, 2024 19:30
Copy link
Contributor

@bkontur bkontur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

I will rebase #5327 tomorrow.
There is commit with enabling of the new short benchmarking CI with frame-omni-bencher, so BridgeHubRococo with frame-omni-bencher fails there, so after rebase it should work.

@ggwpez ggwpez requested review from a team as code owners September 10, 2024 17:10
doc:
- audience: Runtime Dev
description: |-
The genesis state is currently partially provided via `OverlayedChanges`, but these changes are reset by the runtime after the first repetition, causing the second repitition to use an invalid genesis state.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but these changes are reset by the runtime after the first repetition

You mean here that this was overridden and not reset?

Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay this is funny, did not see this PR but had similar thoughts today: #5694

While at it, should we fix issue 2a from the linked issue too?

})
}

/// Convert some overlayed changes to a storage.
fn changes_to_storage<H: Hash>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not get rid of all of this and genesis_from_runtime and just use GenesisConfigBuilderRuntimeCaller? It has all the machinery ready and we get rid of the manual json merging stuff too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also try using GenesisConfigBuilderRuntimeCaller to have genesis storage building in one place. If something is missing it can be adjusted / reworked.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it now.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
@ggwpez ggwpez requested a review from skunert September 13, 2024 17:14
Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

),
"build the genesis spec",
)?;
self.genesis_from_code::<EHF>(code)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You already got storage. You don't need to call this.

Copy link
Member Author

@ggwpez ggwpez Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a distinction now between using the Genesis state from the spec, and using the runtime of the spec to build the genesis state through a preset.
This function does the later - hence it generates the state from the genesis runtime.

Comment on lines 35 to 38
// Use the runtime from the Spec file to build the genesis state.
SpecRuntime,
/// Use the spec file to build the genesis state. This fails when there is no spec.
Spec,
SpecGenesis,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Maybe better name for this while enum would be GenesisBuilderPolicy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sounds better, the original name was a bit random 😅

@skunert
Copy link
Contributor

skunert commented Sep 19, 2024

@ggwpez Anything speaks against merging this? I have a follow up that could share some code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A4-needs-backport Pull request must be backported to all maintained releases. T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants