Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add extra binaries in the setup command #1208

Merged
merged 17 commits into from
Sep 21, 2023
Merged

Add extra binaries in the setup command #1208

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f71e2f1
Add extra binaries in the setup command
wirednkod Jul 31, 2023
2fdeccb
Prepare correctly the setup to accept polkadot prepare and execute wo…
wirednkod Aug 1, 2023
e4aae1b
minor correction
wirednkod Aug 1, 2023
1f1a310
Add README
wirednkod Aug 1, 2023
228c899
minor changes
wirednkod Aug 2, 2023
63d65ea
Fix readme
wirednkod Aug 2, 2023
421d1ac
Fix typo
wirednkod Aug 2, 2023
b63b123
Merge branch 'main' into nik-extra-setup-binaries
wirednkod Aug 2, 2023
dcd48a8
Fix erroneously skipped comment
wirednkod Aug 7, 2023
0c938ac
Merge branch 'nik-extra-setup-binaries' of github.com:paritytech/zomb…
wirednkod Aug 7, 2023
f93e78a
Merge branch 'main' into nik-extra-setup-binaries
wirednkod Aug 7, 2023
d5d5256
replace log with throw Error
wirednkod Aug 7, 2023
e5f5e80
Merge branch 'main' into nik-extra-setup-binaries
wirednkod Sep 5, 2023
284c6fc
Update javascript/packages/cli/src/actions/setup.ts
pepoviola Sep 21, 2023
e020257
merge
wirednkod Sep 21, 2023
ee87512
nik-extra-setup-binaries
wirednkod Sep 21, 2023
9e242d2
fix minor changes on setup command
wirednkod Sep 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,7 @@ This allows to use the `setup` script, making everything ready for a ZombieNet d
You can use the following arguments:

`--help` shows the different options and commands for using the Zombienet CLI.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `polkadot`, `polkadot-parachain`.

`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `all`, `polkadot`, `polkadot-parachain`. *Note:* Downloading `polkadot` will automatically download also the binaries of `polkadot-prepare-worker`, `polkadot-execute-worker`. Since Polkadot v1.0 all 3 binaries are needed for the node to run as a validator;
For example:

```bash
Expand Down
2 changes: 1 addition & 1 deletion javascript/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ This allows to use the `setup` script, making everything ready for a ZombieNet d
You can use the following arguments:

`--help` shows the different options and commands for using the Zombienet CLI.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `polkadot`, `polkadot-parachain`.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `all`, `polkadot`, `polkadot-parachain`. *Note:* Downloading `polkadot` will automatically download also the binaries of `polkadot-prepare-worker`, `polkadot-execute-worker`. Since Polkadot v1.0 all 3 binaries are needed for the node to run as a validator;

For example:

Expand Down
2 changes: 1 addition & 1 deletion javascript/packages/cli/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ This allows to use the `setup` script, making everything ready for a ZombieNet d
You can use the following arguments:

`--help` shows the different options and commands for using the Zombienet CLI.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `polkadot`, `polkadot-parachain`.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `all`, `polkadot`, `polkadot-parachain`. *Note:* Downloading `polkadot` will automatically download also the binaries of `polkadot-prepare-worker`, `polkadot-execute-worker`. Since Polkadot v1.0 all 3 binaries are needed for the node to run as a validator;

For example:

Expand Down
90 changes: 75 additions & 15 deletions javascript/packages/cli/src/actions/setup.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ interface OptIf {
[key: string]: { name: string; url?: string; size?: string };
}

const POLKADOT = "polkadot";
const POLKADOT_PREPARE_WORKER = "polkadot-prepare-worker";
const POLKADOT_EXECUTE_WORKER = "polkadot-execute-worker";
const CUMULUS = "cumulus";
const POLKADOT_PARACHAIN = "polkadot-parachain";

const POSSIBLE_BINARIES = [POLKADOT, POLKADOT_PARACHAIN];
const POLKADOT_WORKERS = [POLKADOT_PREPARE_WORKER, POLKADOT_EXECUTE_WORKER];

const options: OptIf = {};
/**
* Setup - easily download latest artifacts and make them executable in order to use them with zombienet
Expand All @@ -16,8 +25,6 @@ const options: OptIf = {};
* @returns
*/
export async function setup(params: any, opts?: any) {
const POSSIBLE_BINARIES = ["polkadot", "polkadot-parachain"];

console.log(decorators.green("\n\n🧟🧟🧟 ZombieNet Setup 🧟🧟🧟\n\n"));
if (!["linux", "darwin"].includes(process.platform)) {
console.log(
Expand All @@ -28,10 +35,36 @@ export async function setup(params: any, opts?: any) {

console.log(decorators.green("Gathering latest releases' versions...\n"));
await new Promise<void>((resolve) => {
latestPolkadotReleaseURL("polkadot", "polkadot").then(
latestPolkadotReleaseURL(POLKADOT, POLKADOT).then(
(res: [string, string]) => {
options[POLKADOT] = {
name: POLKADOT,
url: res[0],
size: res[1],
};
resolve();
},
);
});

await new Promise<void>((resolve) => {
latestPolkadotReleaseURL(POLKADOT, POLKADOT_PREPARE_WORKER).then(
(res: [string, string]) => {
options[POLKADOT_PREPARE_WORKER] = {
name: POLKADOT_PREPARE_WORKER,
url: res[0],
size: res[1],
};
resolve();
},
);
});

await new Promise<void>((resolve) => {
latestPolkadotReleaseURL(POLKADOT, POLKADOT_EXECUTE_WORKER).then(
(res: [string, string]) => {
options.polkadot = {
name: "polkadot",
options[POLKADOT_EXECUTE_WORKER] = {
name: POLKADOT_EXECUTE_WORKER,
url: res[0],
size: res[1],
};
Expand All @@ -41,10 +74,10 @@ export async function setup(params: any, opts?: any) {
});

await new Promise<void>((resolve) => {
latestPolkadotReleaseURL("cumulus", "polkadot-parachain").then(
latestPolkadotReleaseURL(CUMULUS, POLKADOT_PARACHAIN).then(
(res: [string, string]) => {
options["polkadot-parachain"] = {
name: "polkadot-parachain",
options[POLKADOT_PARACHAIN] = {
name: POLKADOT_PARACHAIN,
url: res[0],
size: res[1],
};
Expand All @@ -55,39 +88,60 @@ export async function setup(params: any, opts?: any) {

// If the platform is MacOS then the polkadot repo needs to be cloned and run locally by the user
// as polkadot do not release a binary for MacOS
if (process.platform === "darwin" && params.includes("polkadot")) {
if (params[0] === "all") {
mrcnski marked this conversation as resolved.
Show resolved Hide resolved
params = [POLKADOT, POLKADOT_PARACHAIN];
}
if (process.platform === "darwin" && params.includes(POLKADOT)) {
console.log(
`${decorators.yellow(
"Note: ",
)} You are using MacOS. Please, clone the polkadot repo ` +
decorators.cyan("(https://github.com/paritytech/polkadot)") +
` and run it locally.\n At the moment there is no polkadot binary for MacOs.\n\n`,
);
params = params.filter((param: string) => param !== "polkadot");
params = params.filter((param: string) => param !== POLKADOT);
}

if (params.length === 0) {
console.log(decorators.green("No binaries to download. Exiting..."));
return;
}
let count = 0;

console.log("Setup will start to download binaries:");

params.forEach((a: any) => {
if (!POSSIBLE_BINARIES.includes(a)) {
params = params.filter((param: any) => param !== a);
console.log(
decorators.red(
`"${a}" is not one of the possible options for this setup and will be skipped;`,
),
decorators.green(` Valid options: polkadot polkadot-parachain`),
decorators.green(
` Valid options: 'polkadot', 'polkadot-parachain', 'all'`,
),
);
return;
}
const size = parseInt(options[a]?.size || "0", 10);
count += size;
console.log("-", a, "\t Approx. size ", size, " MB");
let size = 0;
if (a === POLKADOT) {
size = parseInt(options[a]?.size || "0", 10);
count += size;
console.log("-", a, "\t\t Approx. size ", size, " MB");

POLKADOT_WORKERS.forEach((b) => {
params.push(b);
size = parseInt(options[b]?.size || "0", 10);
count += size;
console.log("-", b, "\t\t Approx. size ", size, " MB");
});
} else {
size = parseInt(options[a]?.size || "0", 10);
count += size;
console.log("-", a, "\t\t Approx. size ", size, " MB");
}
});
console.log("Total approx. size: ", count, "MB");
console.log("Total approx. size:\t\t ", count, "MB");
if (!opts?.yes) {
const response = await askQuestion(
decorators.yellow("\nDo you want to continue? (y/n)"),
Expand Down Expand Up @@ -209,6 +263,12 @@ const latestPolkadotReleaseURL = async (
return Boolean(obj);
});

if (!release) {
throw Error(
`In repo '${repo}', there is no release for: '${name}'! Exiting...`,
Copy link
Collaborator

@pepoviola pepoviola Aug 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to throw here, since the next line
const { tag_name } = release;
will generate an error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - good catch. I meant to throw error here - but I never did. Thank you

);
}

const { tag_name } = release;

if (!tag_name) {
Expand Down
2 changes: 1 addition & 1 deletion javascript/packages/cli/src/cli.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ program
"<binaries...>",
`the binaries that you want to be downloaded, provided in a row without any separators;\nThey are downloaded in current directory and appropriate executable permissions are assigned.\nPossible options: 'polkadot', 'polkadot-parachain'\n${decorators.blue(
"zombienet setup polkadot polkadot-parachain",
)}`,
)}\nNote: Downloading 'polkadot' downloads also 'polkadot-prepare-worker' and 'polkadot-execute-worker'`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package polkadot into npm. inventing own package manager is dead end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dzmitry-lahoda can you elaborate a bit more on this - what exactly you mean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe he means that zombienet setup is doing a job that can already be done by a package manager?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the case then i think it is not correct; Cause not everyone is using zombienet by cloning the repo; There are cases of using it as a release - and thus the release can help setting up the binaries;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. just ask user to use apt npm nix curl dpkg etc.

adhoc pm is dead end. let start that it is not secure nor it will work on proxyvpnedfirewalled envs.

parity to release apt or and npm.

parity already releases nix polkadot. waiting for nix common good nodes;

Copy link
Contributor

@dzmitry-lahoda dzmitry-lahoda Sep 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just ask user to install, like you ask to install k8s, podman, etc.

on windows can use scoop or winget.

on mac brew or nix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was created as an easy way for non-advanced linux users to crash-test zombienet; It retrieves these from the release packages of gitlab and its not the "mandatory way to go";

I am fine for removing it if @pepoviola agrees - I do not see any value though on removing this;

)
.addOption(new Option("-y, --yes", "Bypass confirmation"))
.action(asyncAction(setup));
Expand Down
2 changes: 1 addition & 1 deletion javascript/packages/orchestrator/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ This allows to use the `setup` script, making everything ready for a ZombieNet d
You can use the following arguments:

`--help` shows the different options and commands for using the Zombienet CLI.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `polkadot`, `polkadot-parachain`.
`--binaries` or `-b`: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: `all`, `polkadot`, `polkadot-parachain`. *Note:* Downloading `polkadot` will automatically download also the binaries of `polkadot-prepare-worker`, `polkadot-execute-worker`. Since Polkadot v1.0 all 3 binaries are needed for the node to run as a validator;

For example:

Expand Down
Loading