Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zk_inception): Add ecosystem ports scanner #2849

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
3b55ab2
ecosystem ports scanner
sanekmelnikov Sep 4, 2024
14a28ca
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 16, 2024
3f6702a
Add support for docker compose
matias-gonz Sep 17, 2024
27c346c
Make sure it detects host amchine port and not container port
matias-gonz Sep 17, 2024
5d300eb
Add ZKsync repo directory to scan docker-compose files
matias-gonz Sep 17, 2024
1526dbe
Fix docker compose scan
matias-gonz Sep 18, 2024
c77a5a9
Use allocate ports instead of offset
matias-gonz Sep 18, 2024
36df52b
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 18, 2024
0d15f96
lint
matias-gonz Sep 18, 2024
861284d
lint
matias-gonz Sep 18, 2024
c75dfe3
lint
matias-gonz Sep 18, 2024
3712aee
Remove unused functions
matias-gonz Sep 18, 2024
cab60bb
Remove port offset for chain init
matias-gonz Sep 18, 2024
a72fe5e
Update explorer to ecosystem port scanner
matias-gonz Sep 18, 2024
d4e4529
Remove port offset
matias-gonz Sep 18, 2024
c1330c8
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 18, 2024
d30bae0
Remove find_offset
matias-gonz Sep 19, 2024
04c7bbe
Add default for EcosystemPorts
matias-gonz Sep 19, 2024
2ada84f
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 19, 2024
27723d7
merge
matias-gonz Sep 19, 2024
7f0623c
Update docker compose parse ports
matias-gonz Sep 19, 2024
54aec19
implement ConfigWithChainPorts for ExplorerPorts
matias-gonz Sep 20, 2024
c93d467
impl ConfigWithChainPorts for GeneralConfig
matias-gonz Sep 20, 2024
30e57c6
Update contract_verifier_prometheus_port
matias-gonz Sep 20, 2024
bf19838
Use new port allocation for en
matias-gonz Sep 20, 2024
8574c17
Delete update configs
matias-gonz Sep 20, 2024
03682ee
Refactor get_consensus_config
matias-gonz Sep 20, 2024
26188f7
Add get_consensus_port
matias-gonz Sep 20, 2024
52f3c5e
Remove PortsConfig
matias-gonz Sep 20, 2024
28272c6
Split traverse yaml into smaller functions
matias-gonz Sep 20, 2024
cffe123
Improve test_traverse_yaml
matias-gonz Sep 20, 2024
b0a673f
Add no ports reallocation option
matias-gonz Sep 23, 2024
c314a46
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 23, 2024
c36c1fc
Fix ecosystem directory
matias-gonz Sep 23, 2024
2eb62c5
Add default_value for no_port_reallocation
matias-gonz Sep 23, 2024
550c5e4
Remove default port constants
matias-gonz Sep 25, 2024
5175c07
Change name to traverse_allocate_ports_in_yaml
matias-gonz Sep 26, 2024
934c187
Use allocate_ports_in_yaml in chain init
matias-gonz Sep 26, 2024
7074c7f
Use allocate_ports_in_yaml for en
matias-gonz Sep 26, 2024
c1525aa
Add chain count to allocate_ports_in_yaml
matias-gonz Sep 26, 2024
91ab676
Fix consensus port allocation
matias-gonz Sep 26, 2024
364229c
Fix consensus port allocation in chain init
matias-gonz Sep 26, 2024
1415e2d
Fix consensus port offset
matias-gonz Sep 26, 2024
6c9494c
Fix prepare configs prot allocation
matias-gonz Sep 26, 2024
f3d97d6
Revert en changes
matias-gonz Sep 26, 2024
f13ff46
Allocate consensus port in chain init
matias-gonz Sep 26, 2024
00a934c
Merge branch 'main' into sanekmelnikov/zk-inception-ports
matias-gonz Sep 27, 2024
2405919
Use allocate_ports_in_yaml in en prepare configs
matias-gonz Sep 27, 2024
e174c3b
Change port allocation order
matias-gonz Sep 27, 2024
9ed7664
Fix offset
matias-gonz Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions zk_toolbox/crates/config/src/general.rs
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,7 @@ impl FileConfigWithDefaultName for GeneralConfig {
const FILE_NAME: &'static str = GENERAL_FILE;
}

#[derive(Debug, Default, Clone)]
pub struct PortsConfig {
pub web3_json_rpc_http_port: u16,
pub web3_json_rpc_ws_port: u16,
Expand Down
11 changes: 8 additions & 3 deletions zk_toolbox/crates/zk_inception/src/commands/chain/init.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@ use crate::{
MSG_REGISTERING_CHAIN_SPINNER, MSG_SELECTED_CONFIG,
MSG_UPDATING_TOKEN_MULTIPLIER_SETTER_SPINNER, MSG_WALLET_TOKEN_MULTIPLIER_SETTER_NOT_FOUND,
},
utils::consensus::{generate_consensus_keys, get_consensus_config, get_consensus_secrets},
utils::{
consensus::{generate_consensus_keys, get_consensus_config, get_consensus_secrets},
ports::EcosystemPortsScanner,
},
};

pub(crate) async fn run(args: InitArgs, shell: &Shell) -> anyhow::Result<()> {
Expand All @@ -54,10 +57,12 @@ pub async fn init(
ecosystem_config: &EcosystemConfig,
chain_config: &ChainConfig,
) -> anyhow::Result<()> {
let mut ecosystem_ports = EcosystemPortsScanner::scan(shell)?;
copy_configs(shell, &ecosystem_config.link_to_code, &chain_config.configs)?;

let mut general_config = chain_config.get_general_config()?;
apply_port_offset(init_args.port_offset, &mut general_config)?;
ecosystem_ports.allocate_ports(&mut general_config)?;
sanekmelnikov marked this conversation as resolved.
Show resolved Hide resolved

let ports = ports_config(&general_config).context(MSG_PORTS_CONFIG_ERR)?;

let consensus_keys = generate_consensus_keys();
Expand Down Expand Up @@ -177,7 +182,7 @@ pub async fn init(
Ok(())
}

fn apply_port_offset(port_offset: u16, general_config: &mut GeneralConfig) -> anyhow::Result<()> {
fn _apply_port_offset(port_offset: u16, general_config: &mut GeneralConfig) -> anyhow::Result<()> {
let Some(mut ports_config) = ports_config(general_config) else {
bail!("Missing ports config");
};
Expand Down
13 changes: 13 additions & 0 deletions zk_toolbox/crates/zk_inception/src/defaults.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::ops::Range;

use config::ChainConfig;
use lazy_static::lazy_static;
use url::Url;
Expand All @@ -11,6 +13,17 @@ lazy_static! {
Url::parse("postgres://postgres:notsecurepassword@localhost:5432").unwrap();
}

// Default ports for services
pub const OBSERVABILITY_PORT: u16 = 3000;
pub const POSTGRES_DB_PORT: u16 = 5432;
pub const LOCAL_HTTP_RPC_PORT: u16 = 8545;
pub const LOCAL_WS_RPC_PORT: u16 = 8546;

// Default port range
pub const PORT_RANGE_START: u16 = 3000;
pub const PORT_RANGE_END: u16 = 5000;
pub const PORT_RANGE: Range<u16> = PORT_RANGE_START..PORT_RANGE_END;

pub const ROCKS_DB_STATE_KEEPER: &str = "state_keeper";
pub const ROCKS_DB_TREE: &str = "tree";
pub const ROCKS_DB_PROTECTIVE_READS: &str = "protective_reads";
Expand Down
1 change: 1 addition & 0 deletions zk_toolbox/crates/zk_inception/src/utils/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
pub mod consensus;
pub mod forge;
pub mod ports;
pub mod rocks_db;
Loading
Loading