feat: Optimism Derivation Proof Composition (#69)

* transactions module

* TxEssence trait

* Generic Transaction struct

* tx essence as a generic parameter

* OptimismTxEssence

* op chain spec

* OpTxExecStrategy

* redundant block builder type params

* strategy bundles

* host binary parameters

* profiling flag

* optimism

* new derivation binary skeleton, copy over libs

* providers and conversion utils

* epoch transitioning

* basic derive flow

* host-side derivation

* read metadata from op head

* op-derive guest

* disable guest memory leaks

* heapless batch derivation

* Add missing import

* Remove heapless BinaryHeap

* Remove heapless

* Introduce op-derive tool

* Remove ethers types from BatcherDb trait

* Verify new op block has correct transaction list

* Move derive logic into library

* Fix bug in transaction trie reconstruction

* Introduce get_op_header() to BatcherDb

* Clippy warning

* Default Serde value for FileProvider::receipts

* Disable bloom filter checks

* Clippy

* Clippy

* Fix parsing of from and to fields for deposits

* Reintroduce filtering by log bloom

* fmt

* Add support for local exec to op-derive

* Enforce block_number is correct in MemDb

* Split derive() into multiple functions

* Remove redundant check for batch parent hash

* Remove redundant copy of system config

* Remove redundant block number check

* Cleanup

* Remove redundant vector of eth blocks

* Add base_fee_per_gas to Epoch

* Store deposits in Epoch

* Add Eth tail to DeriveOutput

* Cleanup

* Move deque_next_epoch_if_none to State

* Move eth block processing to Batches

* update zkvm, basic guest

* more compose guest code, comment bonsai code

* untested composition draft guest code

* Reorg and cleanup

* More cleanup

* run cargo fmt --all

* fix clippy warnings

* remove unused imports

* basic prep/left/fin in-memory flow

* in memory aggregation workflow

* format guest code

* composition with receipts

* add op-derive cmd test

* add cmd tests

* cleanup optimism/mod

* Rename command line args

* Fix test arguments

* Rename config field to max_channel_bank_size

* Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL

* Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity

* Import from std instead of alloc/core

* Re-enable core::mem::forget() optimization

* Replace asserts with ensures; enforce absence of receipts for Op blocks

* Simplfy iteration through derived transactions

* Add Bonsai support to op-derive. Also add Bonsai session status to output when polling

* More println

* Rework Batcher initialization

* More logging

* More log output if Bonsai workflow fails

* refactor rpc db

* variable derive step support

* re-enable profiling

* ignore rpc_cache dir

* add -profile support to compose binary

* upgrade zkvm, modify code comments

* bump zkvm, fix CI

* clippies

* fixes and changes

* vs code change

* disambiguate merkle ranges and proofs

Co-authored-by: Wolfgang Welz <[email protected]>

* update risc0 to release v0.20

* update GH action to 0.20

* use old actions

* changes

* refactor: Unified zeth utility (#72)

* feat: Provably build derived op blocks (#78)

* feat: Composition via Bonsai (#79)

* feat: Deterministic builds with receipt cache (+misc) (#80)

* cleanup toml files

* remove unused imports

* cli cleanup

* change block count to u32

* test composition

* fix arguments

* update risc0 to 0.20.1

* compact json files

* update lock files

* test for warnings

* nits

* docker in readme

* create cache file if not found

* update copyright

* update copyright

---------

Co-authored-by: Timothy Carstens <[email protected]>
Co-authored-by: Wolfgang Welz <[email protected]>

.github/workflows/ci.yml

@@ -8,7 +8,7 @@ on:
 env:
   CARGO_TERM_COLOR: always
   GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-  RISC0_VERSION: 0.19.1
+  RISC0_VERSION: 0.20.1
   RISC0_TOOLCHAIN_VERSION: test-release-2
 
 concurrency:
@@ -31,7 +31,7 @@ jobs:
         crate: cargo-binstall
     - run: cargo binstall -y --force cargo-risczero@${{ env.RISC0_VERSION }}
     - run: cargo risczero install --version $RISC0_TOOLCHAIN_VERSION
-    - run: cargo test --workspace --all-targets --all-features
+    - run: cargo test --workspace --all-targets -F ef-tests,debug-guest-build
 
   clippy:
     name: clippy
@@ -45,7 +45,7 @@ jobs:
       with:
         reporter: 'github-pr-check'
         fail_on_error: true
-        clippy_flags: --workspace --all-targets --all-features -- -Dwarnings
+        clippy_flags: --workspace --all-targets -- -Dwarnings
 
   fmt:
     name: fmt

.gitignore

@@ -1,7 +1,9 @@
 .DS_Store
 target/
 tmp/
+cache_rpc/
+cache_zkp/
 log.txt
 *.pb
-rpc_cache.json
-.idea
+*.zkp
+.idea

.vscode/settings.json

@@ -4,6 +4,7 @@
         "./Cargo.toml",
         "./guests/eth-block/Cargo.toml",
         "./guests/op-block/Cargo.toml",
+        "./guests/op-compose/Cargo.toml",
         "./guests/op-derive/Cargo.toml",
         "./testing/ef-tests/testguest/Cargo.toml"
     ]