Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): Check that generated bindings are up to date #1955

Merged
merged 32 commits into from
Aug 6, 2024

Conversation

bitdivine
Copy link
Member

@bitdivine bitdivine commented Jul 31, 2024

Motivation

There is no check in CI that candid files are up to date.

Secondary (& real!!) motivation: Frustration with dfx generate requiring deployment. Really annoying when trying to iterate an API. But might as well use the fast command for something useful, now that we have it.

Changes

  • Added a script that generates candid bindings reasonably quickly & without local deployment.

Tests

  • Added a test to CI that the bindings are up to date.

@bitdivine bitdivine marked this pull request as ready for review August 5, 2024 10:58
@bitdivine bitdivine requested a review from a team as a code owner August 5, 2024 10:58
@@ -4,9 +4,10 @@ function generate_did() {
local canister=$1
canister_root="src/$canister"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recompile the backend only if the Wasm is not there already.

Copy link
Member

@peterpeterparker peterpeterparker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

# Create local .did files
./scripts/did.sh
# Download .did files without an explicit url in dfx.json
./scripts/download.icp.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't require download icp, ckbtc and cketh as we do no generate their types. Any particular reason why you download those here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running a blanket dfx generate, as in the original package.json, will fail if some candid files are missing, like this:

Generating type declarations for canister icp_ledger:
Error: Failed while trying to generate type declarations for 'icp_ledger'.
Caused by: Failed while trying to generate type declarations for 'icp_ledger'.
  Candid file: /home/max/dfn/oisy-wallet/branches/check-generate-in-ci/target/ic/icp_ledger.did doesn't exist.

I guess one could generate only the canisters for which there are directories in declarations, like this:

for canister in $(ls src/declarations/) ; do echo "Generating bindings for $canister" ; dfx generate "$canister" ; done

Is that what you mean? I think it's a good idea. Less files to download => faster execution. :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not knew. I think we should just get rid of dfx generate and directly use candid-extractor.

That's what I do in Juno. e.g. https://github.com/junobuild/juno/blob/main/scripts/did.sh

I can provide a PR to do so once we have merged this PR.

#
# .did files are placed in the local .dfx directory, where the .did files are expected by `dfx generate` and other commands.
function install_did_files() {
jq -r '.canisters | to_entries | .[] | select(.value.candid != null) | "\(.key) \(.value.candid)"' dfx.json |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a check that assert if jq is installed at the top of the script? "You don't have jq, install jq to execute this script"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, can do. Added to the --help and added a check.

.github/workflows/binding-checks.yml Outdated Show resolved Hide resolved
@bitdivine bitdivine merged commit 3a0fe52 into main Aug 6, 2024
9 checks passed
@bitdivine bitdivine deleted the check-generate-in-ci branch August 6, 2024 12:02
@bitdivine
Copy link
Member Author

Merged but happy to revisit if any of the changes were not as imagined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants