Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(l1): reduce unnecessary docker builds and remove duplicate workflow executions in main #1032

Merged
merged 22 commits into from
Oct 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
489253d
Inter workflow artifact download test
rodrigo-o Oct 31, 2024
a04c8a4
Renamed the workflow correctly
rodrigo-o Oct 31, 2024
cea1fc0
moving the docker image download to hive
rodrigo-o Oct 31, 2024
5eac784
Correctly format the download job
rodrigo-o Oct 31, 2024
bb41063
Make hive depend on docker image upload
rodrigo-o Oct 31, 2024
d846f08
Checking adding steps to the job pointing to the toher workflow
rodrigo-o Oct 31, 2024
264430c
Fixed a typo
rodrigo-o Oct 31, 2024
32addef
Test workflow_run as dependency
rodrigo-o Oct 31, 2024
943bc63
Removed all call to the other workflow from hive
rodrigo-o Oct 31, 2024
e2b5126
Moving to just depending on the docker build job
rodrigo-o Oct 31, 2024
6a4b5ef
Check readding branches and making the name shorter and without spaces
rodrigo-o Oct 31, 2024
851bb80
Testing removing also concurrency and just letting the on workflow_run
rodrigo-o Oct 31, 2024
f3cab3c
defaulted to share the docker image in the same job due to the workfl…
rodrigo-o Oct 31, 2024
9d9c0dd
Moved assertoor to the same workflow as hive to share the docker image
rodrigo-o Oct 31, 2024
731eb1e
Correctly add needs
rodrigo-o Oct 31, 2024
08d2749
Check that now the workflows run
rodrigo-o Oct 31, 2024
8d75341
Merge branch 'main' into reuse-docker-build-on-other-jobs
rodrigo-o Oct 31, 2024
0e1bc33
Added checkout to the assertoor job
rodrigo-o Oct 31, 2024
60e3fd1
Remove Docker build from the hive steps
rodrigo-o Oct 31, 2024
99974cd
Reverted CI changes
rodrigo-o Oct 31, 2024
ccb4104
Resolve the duplication upon merging in main related to push + merge_…
rodrigo-o Oct 31, 2024
84eb616
Fixed a typo
rodrigo-o Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 0 additions & 43 deletions .github/workflows/assertoor.yaml

This file was deleted.

2 changes: 0 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
name: CI
on:
merge_group:
push:
branches: [main]
pull_request:
branches: ["**"]
paths-ignore:
Expand Down
67 changes: 0 additions & 67 deletions .github/workflows/hive.yaml

This file was deleted.

124 changes: 124 additions & 0 deletions .github/workflows/hive_and_assertoor.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
name: "Hive & Assertoor"
on:
merge_group:
paths-ignore:
- "crates/l2/**"
- 'README.md'
- 'LICENSE'
- "**/README.md"
- "**/docs/**"
pull_request:
branches: ["**"]
paths-ignore:
- "crates/l2/**"
- 'README.md'
- 'LICENSE'
- "**/README.md"
- "**/docs/**"

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

env:
RUST_VERSION: 1.80.1

jobs:
docker-build:
name: Docker Build image
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
load: true
tags: ethereum_rust
outputs: type=docker,dest=/tmp/ethereum_rust_image.tar

- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: ethereum_rust_image
path: /tmp/ethereum_rust_image.tar

run-hive:
name: Hive - ${{ matrix.name }}
needs: [docker-build]
runs-on: ubuntu-latest
strategy:
matrix:
include:
- simulation: rpc-compat
name: "Rpc Compat tests"
run_command: make run-hive-on-latest SIMULATION=ethereum/rpc-compat TEST_PATTERN="/eth_chainId|eth_getTransactionByBlockHashAndIndex|eth_getTransactionByBlockNumberAndIndex|eth_getCode|eth_getStorageAt|eth_call|eth_getTransactionByHash|eth_getBlockByHash|eth_getBlockByNumber|eth_createAccessList|eth_getBlockTransactionCountByNumber|eth_getBlockTransactionCountByHash|eth_getBlockReceipts|eth_getTransactionReceipt|eth_blobGasPrice|eth_blockNumber|ethGetTransactionCount|debug_getRawHeader|debug_getRawBlock|debug_getRawTransaction|debug_getRawReceipts|eth_estimateGas|eth_getBalance|eth_sendRawTransaction|eth_getProof|eth_getLogs"
- simulation: rpc-auth
name: "Rpc Auth tests"
run_command: make run-hive-on-latest SIMULATION=ethereum/rpc-compat TEST_PATTERN="/engine-auth"
- simulation: discv4
name: "Devp2p discv4 tests"
run_command: make run-hive-on-latest SIMULATION=devp2p TEST_PATTERN="discv4"
- simulation: snap
name: "Devp2p snap tests"
run_command: make run-hive-on-latest SIMULATION=devp2p TEST_PATTERN="/AccountRange"
- simulation: engine
name: "Engine tests"
run_command: make run-hive-on-latest SIMULATION=ethereum/engine TEST_PATTERN="/Blob Transactions On Block 1, Cancun Genesis|Blob Transactions On Block 1, Shanghai Genesis|Blob Transaction Ordering, Single Account, Single Blob|Blob Transaction Ordering, Single Account, Dual Blob|Blob Transaction Ordering, Multiple Accounts|Replace Blob Transactions|Parallel Blob Transactions|ForkchoiceUpdatedV3 Modifies Payload ID on Different Beacon Root|NewPayloadV3 After Cancun|NewPayloadV3 Versioned Hashes|ForkchoiceUpdated Version on Payload Request"
- simulation: engine-cancun
name: "Cancun Engine tests"
run_command: make run-hive-on-latest SIMULATION=ethereum/engine TEST_PATTERN="cancun/Unique Payload ID|ParentHash equals BlockHash on NewPayload|Re-Execute Payload|Payload Build after New Invalid Payload|RPC|Build Payload with Invalid ChainID|Invalid PayloadAttributes, Zero timestamp, Syncing=False|Invalid PayloadAttributes, Parent timestamp, Syncing=False|Invalid PayloadAttributes, Missing BeaconRoot, Syncing=False|Suggested Fee Recipient Test|PrevRandao Opcode Transactions Test|Invalid Missing Ancestor ReOrg, StateRoot"
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: ethereum_rust_image
path: /tmp

- name: Load image
run: |
docker load --input /tmp/ethereum_rust_image.tar

- name: Checkout sources
uses: actions/checkout@v3

- name: Rustup toolchain install
uses: dtolnay/rust-toolchain@stable
with:
toolchain: ${{ env.RUST_VERSION }}

- name: Setup Go
uses: actions/setup-go@v3

- name: Run Hive Simulation
run: ${{ matrix.run_command }}

run-assertoor:
name: Assertoor - Stability Check
runs-on: ubuntu-latest
needs: [docker-build]
steps:
- uses: actions/checkout@v4

- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: ethereum_rust_image
path: /tmp

- name: Load image
run: |
docker load --input /tmp/ethereum_rust_image.tar

- name: Setup kurtosis testnet and run assertoor tests
uses: ethpandaops/kurtosis-assertoor-github-action@v1
with:
kurtosis_version: '1.3.1'
ethereum_package_url: 'github.com/lambdaclass/ethereum-package'
ethereum_package_branch: 'ethereum-rust-integration'
ethereum_package_args: './test_data/network_params.yaml'
2 changes: 0 additions & 2 deletions .github/workflows/levm_bench.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@ name: LEVM benchmarks

on:
merge_group:
push:
paths:
- 'crates/vm/levm/**'
branches: [ main ]
pull_request:
paths:
- 'crates/vm/levm/**'
Expand Down
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,9 @@ TEST_PATTERN ?= /
run-hive: build-image setup-hive ## 🧪 Run Hive testing suite
cd hive && ./hive --sim $(SIMULATION) --client ethereumrust --sim.limit "$(TEST_PATTERN)"

run-hive-on-latest: setup-hive ## 🧪 Run Hive testing suite with the latest docker image
fkrause98 marked this conversation as resolved.
Show resolved Hide resolved
cd hive && ./hive --sim $(SIMULATION) --client ethereumrust --sim.limit "$(TEST_PATTERN)"

run-hive-debug: build-image setup-hive ## 🐞 Run Hive testing suite in debug mode
cd hive && ./hive --sim $(SIMULATION) --client ethereumrust --sim.limit "$(TEST_PATTERN)" --docker.output

Expand Down
Loading