Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

garaga_rs WASM bindings #180

Merged
merged 85 commits into from
Sep 10, 2024

Conversation

raugfer
Copy link
Collaborator

@raugfer raugfer commented Aug 30, 2024

Pull Request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no API changes)
  • Build-related changes
  • Documentation content changes
  • Testing
  • Other (please describe):

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Does this introduce a breaking change?

  • Yes
  • No

Other information

@raugfer raugfer marked this pull request as ready for review September 5, 2024 02:57
@raugfer
Copy link
Collaborator Author

raugfer commented Sep 5, 2024

I have marked it for review but there are 2 steps that we could still include in the PR:

1- Commit tools/garaga_rs/pkg folder with the code generated by the wasm-pack-build-and-patch script (except .wasm files which are large), this can help detecting unexpected changes on the wasm-pack output that could mess the patched configuration

2- Include a folder with a minimal project that imports the garaga_rs npm package as an end-to-end test

@feltroidprime feltroidprime changed the base branch from main to wasm September 5, 2024 09:05
@feltroidprime
Copy link
Collaborator

feltroidprime commented Sep 5, 2024

I have marked it for review but there are 2 steps that we could still include in the PR:

1- Commit tools/garaga_rs/pkg folder with the code generated by the wasm-pack-build-and-patch script (except .wasm files which are large), this can help detecting unexpected changes on the wasm-pack output that could mess the patched configuration

  • The .wasm isn't that large, it's about 1.2mB, i think we an we commit this package indeed, but can we config the files so it builds it in tools/pkg instead of tools/garaga_rs/pkg ? (or tools/npm if we can choose a different name)
    Then the CI would build again and compare against the committed one, maybe we can do something like https://github.com/hazae41/ed25519.wasm?tab=readme-ov-file#automated-checks for the commit part

  • I made a sample .js test file to put in the pkg folder

// test.js
import * as garaga_rs from 'garaga_rs';
async function main() {
    await garaga_rs.init();
    const result = garaga_rs.msm_calldata_builder([1, 2], [10], 0);
    console.log("Output of msm_calldata_builder:", result);
}
// Execute the main function
main().catch(console.error);

and i run it with node pkg/test.js

How can we make a separate test folder with this file that imports it and runs this file as well in the CI ?

More generally how can set up the config so that the pkg/ folder built from the script has an automated part that is generated by wasm-pack, and a more high level part with wrappers that we don't delete each time the wasm-pack-build-and-patch is ran ?

The idea is that the bindings can be regenerated but the high level functions that uses the binding don't change and are part of the same package

@raugfer
Copy link
Collaborator Author

raugfer commented Sep 5, 2024

  • The .wasm isn't that large, it's about 1.2mB, i think we an we commit this package indeed, but can we config the files so it builds it in tools/pkg instead of tools/garaga_rs/pkg ? (or tools/npm if we can choose a different name)
    Then the CI would build again and compare against the committed one, maybe we can do something like https://github.com/hazae41/ed25519.wasm?tab=readme-ov-file#automated-checks for the commit part

Yes, we can redirect the output to any folder using wasm-pack build --out-dir and use a strategy like ed25519.wasm does to overwrite the generated code and test for discrepancies.

  • I made a sample .js test file to put in the pkg folder
// test.js
import * as garaga_rs from 'garaga_rs';
async function main() {
    await garaga_rs.init();
    const result = garaga_rs.msm_calldata_builder([1, 2], [10], 0);
    console.log("Output of msm_calldata_builder:", result);
}
// Execute the main function
main().catch(console.error);

and i run it with node pkg/test.js

How can we make a separate test folder with this file that imports it and runs this file as well in the CI ?

Ideally we would setup a minimal/complete typescript project that imports garaga_rs. In fact we could use your code above but have a small set of config variations in subfolders (to test compatibility e.g. commonjs vs esm, browser vs nodejs, javascript vs typescript) and setup an umbrella project (monorepo style) that compiles and tests them altogether to be used on CI. We could have a structure like

tools/npm/pkg
tools/npm/integration-test-suite

or better

tools/npm/garaga_rs
tools/npm/integration-test-suite

More generally how can set up the config so that the pkg/ folder built from the script has an automated part that is generated by wasm-pack, and a more high level part with wrappers that we don't delete each time the wasm-pack-build-and-patch is ran ?
The idea is that the bindings can be regenerated but the high level functions that uses the binding don't change and are part of the same package

I think you are talking about a hybrid package that has both WASM module and additional TypeScript, like we have for Pyhton, right?

In that case I would do just like ed25519.wasm and setup a full NPM project that is built with rollup.js and integrates both parts. Right now, I have simplified that setup under the assumption that the NPM package would only include the WASM code.

In that case we could have borrow of structure ed25519.wasm like

tools/npm/garaga_rs/src/node/
tools/npm/garaga_rs/src/wasm/pkg
tools/npm/garaga_rs/integration-test-suite

where tools/npm/garaga_rs would be the root of the NPM package built with rollup.js

@feltroidprime feltroidprime changed the base branch from wasm to main September 9, 2024 17:01
feltroidprime and others added 27 commits September 9, 2024 19:02
@feltroidprime feltroidprime merged commit a055ebb into keep-starknet-strange:main Sep 10, 2024
26 checks passed
@raugfer raugfer deleted the rust-wasm-binding branch September 10, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants