Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(IDX): allow tests to depend on mainnet pocket-ic #1707

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
load("@bazel_gazelle//:def.bzl", "gazelle")
load("@bazel_skylib//rules:common_settings.bzl", "string_setting")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")

package(default_visibility = ["//visibility:public"])
Expand Down Expand Up @@ -98,3 +99,54 @@ test_suite(
tags = ["manual"],
tests = ["//rs/tests/testing_verification/testnets:single_large_node"],
)

### Pocket IC

# The pocket-ic server binary. Use this as a test dependency if the test
# does not require a specific pocket-ic version (see ":pocket-ic-variant"
# for details).
# By default returns the pocket-ic server from the source tree to ensure
# consistency within the source tree. See 'pocket_ic_mainnet_test' for
# overrides.
alias(
name = "pocket-ic-server",
actual = select({
":pocket-ic-variant-mainnet": "//:pocket-ic-mainnet",
":pocket-ic-variant-head": "//rs/pocket_ic_server:pocket-ic-server",
"//conditions:default": "//rs/pocket_ic_server:pocket-ic-server",
}),
)

# A setting to switch between different variants of pocket-ic. The
# default pocket-ic variant/version (head) is the one as in the
# source tree.
string_setting(
name = "pocket-ic-variant",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about being explicit about which pocket-ic "thing" we're mentioning here? pocket-ic usually refers to the library while the following is about the server:

Suggested change
name = "pocket-ic-variant",
name = "pocket-ic-server-variant",

Same comment for the other things below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep that makes sense!

build_setting_default = "head",
visibility = ["//visibility:public"],
)

config_setting(
name = "pocket-ic-variant-head",
flag_values = {
":pocket-ic-variant": "head",
},
)

# A "mainnet" variant of the pocket-ic server which represents a
# released version of pocket-ic.
config_setting(
name = "pocket-ic-variant-mainnet",
flag_values = {
":pocket-ic-variant": "mainnet",
},
)

# The pocket-ic as released; use this for tests that need to ensure consistency
# with a release pocket-ic/replica.
genrule(
name = "pocket-ic-mainnet",
srcs = ["@pocket-ic-mainnet-gz//file"],
outs = ["pocket-ic"],
cmd = "gunzip -c $< > $@",
)
6 changes: 6 additions & 0 deletions WORKSPACE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -981,3 +981,9 @@ http_file(
sha256 = "2a428e8d35518ce22002e498f7d618a9eeeddf57371f965e92cf480dd3cbd046",
url = "https://github.com/dfinity/examples/releases/download/rust-basic-bitcoin-24-09-16/basic_bitcoin.wasm.gz",
)

http_file(
name = "pocket-ic-mainnet-gz",
sha256 = "454891cac2421f3f894759ec5e6b6e48fbb544d79197bc29b88d34b93d78a4f1",
url = "https://download.dfinity.systems/ic/52ebccfba8855e23dcad9657a8d6e6be01df71f9/binaries/x86_64-linux/pocket-ic.gz",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see mainnet is using c66489953ed34670667ad81ce42a172948f289f5.

Will you address syncing this with testnet/mainnet_revisions.json#L3 in another PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you address syncing this with testnet/mainnet_revisions.json#L3 in another PR?

Yep, talked to DRE team and we'll be taking over the update of that file too. Let's sync offline

)
94 changes: 94 additions & 0 deletions bazel/pocket-ic-tests.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
"""Build helpers for using the pocket-ic"""

load("@bazel_skylib//lib:paths.bzl", "paths")

# This defines a transition used in the pocket-ic declaration.
# This allows tests to depend on an arbitrary version of pocket-ic
# and for dependents to override that version.
def _pocket_ic_mainnet_transition_impl(_settings, _attr):
return {
"//:pocket-ic-variant": "mainnet",
}

pocket_ic_mainnet_transition = transition(
implementation = _pocket_ic_mainnet_transition_impl,
inputs = [],
outputs = [
"//:pocket-ic-variant",
],
)

# Provider that allows wrapping/transitioning a test executable. This
# ensure all the test data & env is forwarded.
# Adapted from github.com/aherrman/bazel-transitions-demo:
# https://github.com/aherrmann/bazel-transitions-demo/blob/f22cf40a62131eace14829f262e8d7c00b0a9a19/flag/defs.bzl#L124
TestAspectInfo = provider("some descr", fields = ["args", "env"])

def _test_aspect_impl(_target, ctx):
data = getattr(ctx.rule.attr, "data", [])
args = getattr(ctx.rule.attr, "args", [])
env = getattr(ctx.rule.attr, "env", [])
args = [ctx.expand_location(arg, data) for arg in args]
env = {k: ctx.expand_location(v, data) for (k, v) in env.items()}
return [TestAspectInfo(
args = args,
env = env,
)]

_test_aspect = aspect(_test_aspect_impl)

def _pocket_ic_mainnet_test_impl(ctx):
test_aspect_info = ctx.attr.test[TestAspectInfo]
(_, extension) = paths.split_extension(ctx.executable.test.path)
executable = ctx.actions.declare_file(
ctx.label.name + extension,
)
ctx.actions.write(
output = executable,
content = """\
#!/usr/bin/env bash
set -euo pipefail
{commands}
""".format(
commands = "\n".join([
" \\\n".join([
'{}="{}"'.format(k, v)
for k, v in test_aspect_info.env.items()
] + [
ctx.executable.test.short_path,
] + test_aspect_info.args),
]),
),
is_executable = True,
)

runfiles = ctx.runfiles(files = [executable, ctx.executable.test] + ctx.files.data)
runfiles = runfiles.merge(ctx.attr.test[DefaultInfo].default_runfiles)
for data_dep in ctx.attr.data:
runfiles = runfiles.merge(data_dep[DefaultInfo].default_runfiles)

return [DefaultInfo(
executable = executable,
files = depset(direct = [executable]),
runfiles = runfiles,
)]

# Rule to override another (test) rule. The resulting rule
# will use the mainnet pocket-ic varaint.
pocket_ic_mainnet_test = rule(
_pocket_ic_mainnet_test_impl,
attrs = {
"_allowlist_function_transition": attr.label(
default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),
"data": attr.label_list(allow_files = True),
"test": attr.label(
aspects = [_test_aspect],
cfg = "target",
executable = True,
),
},
cfg = pocket_ic_mainnet_transition,
executable = True,
test = True,
)
6 changes: 6 additions & 0 deletions rs/bitcoin/ckbtc/kyt/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")
load("//bazel:canisters.bzl", "rust_canister")
load("//bazel:defs.bzl", "rust_ic_test")
load("//bazel:pocket-ic-tests.bzl", "pocket_ic_mainnet_test")

package(default_visibility = ["//visibility:public"])

Expand Down Expand Up @@ -72,3 +73,8 @@ rust_ic_test(
"@crate_index//:candid",
],
)

pocket_ic_mainnet_test(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just an example, not sure if meaningful or not

name = "kyt_integration_test_mainnet",
test = ":kyt_integration_test",
)
4 changes: 2 additions & 2 deletions rs/bitcoin/kyt/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,13 @@ rust_ic_test(
env = {
"CARGO_MANIFEST_DIR": "rs/bitcoin/kyt",
"IC_BTC_KYT_CANISTER_WASM_PATH": "$(rootpath :btc_kyt_canister)",
"POCKET_IC_BIN": "$(rootpath //rs/pocket_ic_server:pocket-ic-server)",
"POCKET_IC_BIN": "$(rootpath //:pocket-ic-server)",
},
deps = [
# Keep sorted.
":btc_kyt_lib",
"//:pocket-ic-server",
"//packages/pocket-ic",
"//rs/pocket_ic_server:pocket-ic-server",
"//rs/test_utilities/load_wasm",
"//rs/types/base_types",
"//rs/types/types",
Expand Down
Loading