Skip to content

Commit

Permalink
ci: nightly fuzzing (#1407)
Browse files Browse the repository at this point in the history
<!--- Please provide a general summary of your changes in the title
above -->

<!-- Give an estimate of the time you spent on this PR in terms of work
days.
Did you spend 0.5 days on this PR or rather 2 days?  -->

Time spent on this PR:

## Pull request type

<!-- Please try to limit your pull request to one type,
submit multiple pull requests if needed. -->

Please check the type of change your PR introduces:

- [ ] Bugfix
- [x] Feature
- [ ] Code style update (formatting, renaming)
- [ ] Refactoring (no functional changes, no api changes)
- [ ] Build related changes
- [ ] Documentation content changes
- [ ] Other (please describe):

## What is the current behavior?

<!-- Please describe the current behavior that you are modifying,
or link to a relevant issue. -->

Resolves #1403

## What is the new behavior?

- A nightly run is triggered with
[schedule](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#schedule)
at 0:20 UTC for unit test with 1500 examples for hypothesis
- CI runs is decreased to 100 examples
- refactoring of a few patches (30% faster on 1000 example for
`test_raise_gas_limit_too_high`)
- update of hypothesis packages: should increase perfs for binary (see
[changelog
here](https://hypothesis.readthedocs.io/en/latest/changes.html))

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg"
height="34" align="absmiddle"
alt="Reviewable"/>](https://reviewable.io/reviews/kkrt-labs/kakarot/1407)
<!-- Reviewable:end -->
  • Loading branch information
obatirou committed Sep 10, 2024
1 parent e8cd987 commit 428fd68
Show file tree
Hide file tree
Showing 4 changed files with 81 additions and 18 deletions.
63 changes: 63 additions & 0 deletions .github/workflows/nightly-fuzzing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: NIGHTLY-FUZZING

on:
schedule:
- cron: 20 0 * * *

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

permissions: read-all

jobs:
tests-unit:
runs-on: ubuntu-latest-16-cores
env:
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10.14
uses: actions/setup-python@v5
with:
python-version: 3.10.14
- name: Load cached Poetry installation
id: cached-poetry
uses: actions/cache@v4
with:
path: ~/.local
key: poetry-${{ runner.os }}
- name: Install Poetry
if: steps.cached-poetry.outputs.cache-hit != 'true'
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
virtualenvs-path: .venv
installer-parallel: true
- name: Enforce poetry config
run: |
poetry config virtualenvs.in-project true
poetry config virtualenvs.create true
poetry config virtualenvs.path .venv
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: make setup
- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
with:
version: nightly
- name: Run tests
env:
HYPOTHESIS_PROFILE: nightly
run: make test-unit
- name: Upload coverage report to codecov
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
directory: ./coverage/
10 changes: 5 additions & 5 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 7 additions & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,16 @@ def seed(request):

pytest_plugins = ["tests.fixtures.starknet"]

settings.register_profile(
"nightly",
deadline=None,
max_examples=1500,
phases=[Phase.explicit, Phase.reuse, Phase.generate, Phase.target],
)
settings.register_profile(
"ci",
deadline=None,
max_examples=1000,
max_examples=100,
phases=[Phase.explicit, Phase.reuse, Phase.generate, Phase.target],
)
settings.register_profile(
Expand Down
18 changes: 6 additions & 12 deletions tests/src/kakarot/test_kakarot.py
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,7 @@ def test_should_raise_invalid_nonce(self, cairo_run, tx):
)

@given(gas_limit=integers(min_value=2**64, max_value=2**248 - 1))
@SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [34])
def test_raise_gas_limit_too_high(self, cairo_run, gas_limit):
tx = {
"type": 2,
Expand All @@ -468,17 +469,15 @@ def test_raise_gas_limit_too_high(self, cairo_run, gas_limit):
}
tx_data = list(rlp_encode_signed_data(tx))

with (
SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [tx["nonce"]]),
cairo_error(message="Gas limit too high"),
):
with cairo_error(message="Gas limit too high"):
cairo_run(
"test__eth_send_raw_unsigned_tx",
tx_data_len=len(tx_data),
tx_data=tx_data,
)

@given(maxFeePerGas=integers(min_value=2**128, max_value=2**248 - 1))
@SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [34])
def test_raise_max_fee_per_gas_too_high(self, cairo_run, maxFeePerGas):
tx = {
"type": 2,
Expand All @@ -494,10 +493,7 @@ def test_raise_max_fee_per_gas_too_high(self, cairo_run, maxFeePerGas):
}
tx_data = list(rlp_encode_signed_data(tx))

with (
SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [tx["nonce"]]),
cairo_error(message="Max fee per gas too high"),
):
with cairo_error(message="Max fee per gas too high"):
cairo_run(
"test__eth_send_raw_unsigned_tx",
tx_data_len=len(tx_data),
Expand Down Expand Up @@ -543,6 +539,7 @@ def max_priority_fee_too_high(draw):
return (max_fee_per_gas, max_priority_fee_per_gas)

@SyscallHandler.patch("Kakarot_block_gas_limit", TRANSACTION_GAS_LIMIT)
@SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [34])
@given(max_priority_fee_too_high())
def test_raise_max_priority_fee_too_high(
self, cairo_run, max_priority_fee_too_high
Expand All @@ -561,10 +558,7 @@ def test_raise_max_priority_fee_too_high(
}
tx_data = list(rlp_encode_signed_data(tx))

with (
SyscallHandler.patch("IAccount.get_nonce", lambda _, __: [tx["nonce"]]),
cairo_error(message="Max priority fee greater than max fee per gas"),
):
with cairo_error(message="Max priority fee greater than max fee per gas"):
cairo_run(
"test__eth_send_raw_unsigned_tx",
tx_data_len=len(tx_data),
Expand Down

0 comments on commit 428fd68

Please sign in to comment.