Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Remove superflous parameter overseer_enable_anyways and make parachain node type more explicit #7617

Merged
merged 5 commits into from
Aug 15, 2023

Conversation

bkchr
Copy link
Member

@bkchr bkchr commented Aug 14, 2023

This pull request removes the superfluous parameter overseer_enable_anyways and renames IsCollator to IsParachainNode. The parameter overseer_enable_anyways was actually used to express that a relay chain node is running alongside a parachain full node. This lead to things like enabling of the candidate validation system running, which is only required for relay chain validators. This pull requests makes this more explicit by introducing IsParachainNode::FullNode and ensuring that we don't enable subsystems that we don't need for parachain nodes.

Cumulus companion: paritytech/cumulus#3008

We don't need this flag, as we don't need the overseer enabled when the
node isn't a collator or validator.
@bkchr bkchr requested review from skunert and mrcnski August 14, 2023 18:13
@bkchr bkchr added A2-insubstantial Pull request requires no code review (e.g., a sub-repository hash update). B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. labels Aug 14, 2023
bkchr added a commit to paritytech/cumulus that referenced this pull request Aug 14, 2023
@skunert
Copy link
Contributor

skunert commented Aug 14, 2023

What about pov-recovery for parachain full nodes? AFAIR that was the reason for this flag.

`IsParachainNode` is more expressive and also encapsulates the state of
the parachain node being a full node. Some functionality like the
overseer needs to run always when the node runs alongside a parachain
node. The parachain node needs the overseer to e.g. recover PoVs. Other
things like candidate validation or pvf checking are only required for
when the node is running as validator.
@bkchr
Copy link
Member Author

bkchr commented Aug 14, 2023

What about pov-recovery for parachain full nodes? AFAIR that was the reason for this flag.

Ahh yes, good point! Reworked the pr.

@bkchr bkchr changed the title Remove superflous parameter overseer_enable_anyways Remove superflous parameter overseer_enable_anyways and make parachain node type more explicit Aug 14, 2023
@pepoviola
Copy link
Contributor

@bkchr zombienet tests are failing with this error, looks like an issue in the collator image.

Error: Input("Worker binaries could not be found, make sure polkadot was built/installed correctly. If you ran with `cargo run`, please run `cargo build` first. Searched given workers path (None), polkadot binary path (\"/usr/local/bin\"), and lib path (/usr/lib/polkadot), workers names: None")

Logs:
https://grafana.teleport.parity.io/goto/SXuFYj64g?orgId=1
https://grafana.teleport.parity.io/goto/JnZaLjeVg?orgId=1

I will take a look. Thx!

@skunert
Copy link
Contributor

skunert commented Aug 15, 2023

https://grafana.teleport.parity.io/goto/SXuFYj64g?orgId=1

I think this is because the undying collator is started with --validator which makes it an authority. Together with this change https://github.com/paritytech/polkadot/pull/7617/files#diff-6dcbafad4c91afb2c91f5b01429549242a45a3bb0de721c830abd0a61421ae3aL913 it will try to start the workers.

@bkchr
Copy link
Member Author

bkchr commented Aug 15, 2023

@pepoviola fixed the CI failure. It is related to how zombienet spawns the node, as @skunert explains above. I "fixed" this for now. The proper solution would be that zombienet is not using "default cli args", but that is something for another day :P

@skunert
Copy link
Contributor

skunert commented Aug 15, 2023

@pepoviola fixed the CI failure. It is related to how zombienet spawns the node, as @skunert explains above. I "fixed" this for now. The proper solution would be that zombienet is not using "default cli args", but that is something for another day :P

Still slightly confused by this, validator should not be default afaik. Also, the behaviour here is a bit different because in cumulus we handle --validator as --collator. Wondering if setting cumulus_based=false would help here. Just curious why zombienet is passing --validator in the first place 🤔

Copy link
Contributor

@mrcnski mrcnski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good. As @rphmeier has noted we may need a bigger refactor of this code in the future.

Comment on lines +61 to +63
// Zombienet is spawning all collators currently with the same CLI, this means it
// sets `--validator` and this is wrong here.
config.role = Role::Full;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have a followup to fix this on the zombienet side?

Copy link
Contributor

@pepoviola pepoviola Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here, I will create a new pr to remove this workaround.

@bkchr
Copy link
Member Author

bkchr commented Aug 15, 2023

bot merge

@paritytech-processbot paritytech-processbot bot merged commit e074364 into master Aug 15, 2023
5 checks passed
@paritytech-processbot paritytech-processbot bot deleted the bkchr-remove-overseer-anyways branch August 15, 2023 08:51
paritytech-processbot bot pushed a commit to paritytech/cumulus that referenced this pull request Aug 15, 2023
* Companion for Polkadot#7617

paritytech/polkadot#7617

* Adapt to latest Polkadot changes.

* update lockfile for {"polkadot", "substrate"}

---------

Co-authored-by: parity-processbot <>
@pepoviola
Copy link
Contributor

Thanks @bkchr / @skunert / @mrcnski, I created this followup issue to fix this in zombienet.

s0me0ne-unkn0wn pushed a commit that referenced this pull request Aug 15, 2023
…ain node type more explicit (#7617)

* Remove superflous parameter `overseer_enable_anyways`

We don't need this flag, as we don't need the overseer enabled when the
node isn't a collator or validator.

* Rename `IsCollator` to `IsParachainNode`

`IsParachainNode` is more expressive and also encapsulates the state of
the parachain node being a full node. Some functionality like the
overseer needs to run always when the node runs alongside a parachain
node. The parachain node needs the overseer to e.g. recover PoVs. Other
things like candidate validation or pvf checking are only required for
when the node is running as validator.

* FMT

* Fix CI
@rphmeier
Copy link
Contributor

rphmeier commented Aug 15, 2023

This shouldn't have been merged IMO

Subsystems shouldn't be disabling themselves internally! Just don't add the subsystem if you don't need it.

And we're baking in all of these half-formed assumptions about what a parachain node is or what a collator is. Or even what a full node of the relay chain is. These assumptions will slow us down later. The service instantiation is already a petrified lump. If it could speak, it'd say "kill me."

There is an OverseerGen trait for a reason - implement it to disable the subsystems that aren't needed. There is no one-size-fits-all solution here and we will just end up building ourselves a tower of conditional spaghetti.

@rphmeier
Copy link
Contributor

ref paritytech/polkadot-sdk#586

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A2-insubstantial Pull request requires no code review (e.g., a sub-repository hash update). B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants