Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zk_inception): Add ecosystem ports scanner #2849

Open
wants to merge 47 commits into
base: main
Choose a base branch
from

Conversation

sanekmelnikov
Copy link
Collaborator

@sanekmelnikov sanekmelnikov commented Sep 11, 2024

What ❔

Added EcosystemPortsScanner tool to scan ecosystem directory config and find all values for ports.
Returns EcosystemPorts object with ports found and custom description of a service.
EcosystemPorts also contains methods to: check whether the port is assigned, allocate a new port in range, etc.

Why ❔

With multiple chains and number of services (like explorer, etc.) growing - it is useful to have a tool (EcosystemPorts) that can be used to allocate new ports, thus, lowering the chance of port conflicts within created configuration.

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.

@sanekmelnikov sanekmelnikov marked this pull request as draft September 11, 2024 14:11
@sanekmelnikov
Copy link
Collaborator Author

sanekmelnikov commented Sep 11, 2024

Marking as draft as ports in docker files are not parsed and tool is not yet used for allocation.

@matias-gonz matias-gonz marked this pull request as ready for review September 18, 2024 11:34
@matias-gonz matias-gonz marked this pull request as draft September 18, 2024 12:39
@matias-gonz matias-gonz changed the title feat(zk_inception): [WIP] Ecosystem ports scanner feat(zk_inception): Add ecosystem ports scanner Sep 18, 2024
@matias-gonz matias-gonz marked this pull request as ready for review September 18, 2024 18:22
@matias-gonz matias-gonz requested a review from a team as a code owner September 18, 2024 18:22
Comment on lines 66 to 78
pub const DEFAULT_CONSENSUS_PORT: u16 = 3054;
/// Default port for the web3 json rpc service
pub const DEFAULT_WEB3_JSON_RPC_PORT: u16 = 3050;
/// Default port for the web3 ws rpc service
pub const DEFAULT_WEB3_WS_RPC_PORT: u16 = 3051;
/// Default port for the healthcheck service
pub const DEFAULT_HEALTHCHECK_PORT: u16 = 3071;
/// Default port for the merkle tree
pub const DEFAULT_MERKLE_TREE_PORT: u16 = 3072;
/// Default port for the prometheus service
pub const DEFAULT_PROMETHEUS_PORT: u16 = 3314;
/// Default port for the contract verifier service
pub const DEFAULT_CONTRACT_VERIFIER_PORT: u16 = 3070;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need them?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are used for default port + offset allocation

format!("{}:{}", path, key.as_str().unwrap_or_default())
};

if key.as_str() == Some("ports") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably services is fine

Comment on lines 294 to 314
HashMap::from([
(
"web3_json_rpc_http_port".to_string(),
DEFAULT_WEB3_JSON_RPC_PORT,
),
(
"web3_json_rpc_ws_port".to_string(),
DEFAULT_WEB3_WS_RPC_PORT,
),
("healthcheck_port".to_string(), DEFAULT_HEALTHCHECK_PORT),
("merkle_tree_port".to_string(), DEFAULT_MERKLE_TREE_PORT),
(
"prometheus_listener_port".to_string(),
DEFAULT_PROMETHEUS_PORT,
),
(
"contract_verifier_port".to_string(),
DEFAULT_CONTRACT_VERIFIER_PORT,
),
("consensus_port".to_string(), DEFAULT_CONSENSUS_PORT),
])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this function? We have already allocated ports in config file

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will see if I can use those instead

])
}

fn set_ports(&mut self, ports: HashMap<String, u16>) -> anyhow::Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand it right, that if we have some another port, which is not specified, it'd fail?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, i'd rather save the whole paths to the value and write a small macros for updating it

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set_ports is used here

This means set_ports uses the same ports as the ones in get_default_ports
So it wouldn't fail as long as get_default_ports is consistent with set_ports. Both functions belong to the same trait: ConfigWithChainPorts making it easy to keep it consistent.

How do you think we can improve this?

Comment on lines +72 to +73
let consensus_port_range = DEFAULT_CONSENSUS_PORT + offset..PORT_RANGE_END;
let consensus_port =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we reuse the ecosystem ports for it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean adding a function to EcosystemPorts to allocate consensus port?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants