feat: Deterministic builds with receipt cache (+misc) #80
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hashcashier
changed the title
hashcashier
changed the title
Wollac
approved these changes
Jan 31, 2024
lib/src/builder/execute/optimism.rs
Outdated
Comment on lines
76
to
83
| trace!("Block no. {}", header.number); | ||
| trace!(" EVM spec ID: {:?}", spec_id); | ||
| trace!(" Timestamp: {}", dt); | ||
| trace!(" Transactions: {}", block_builder.input.transactions.len()); | ||
| trace!(" Fee Recipient: {:?}", block_builder.input.beneficiary); | ||
| trace!(" Gas limit: {}", block_builder.input.gas_limit); | ||
| trace!(" Base fee per gas: {}", header.base_fee_per_gas); | ||
| trace!(" Extra data: {:?}", block_builder.input.extra_data); |
There was a problem hiding this comment.
This is much less verbose, so I think it should be debug
There was a problem hiding this comment.
it gets really spammy and repetitive, especially during composition, which already shows block number and hash
There was a problem hiding this comment.
That's why I like it being debug and not info, it only comes once per block, the logs on tx level are even more frequent and trace
…to rkhalil/polish
hashcashier
added a commit
that referenced
this pull request
Feb 5, 2024
…ceipt cache (+misc) (#80) * update crates * prove on bonsai * verify bonsai receipts * repeat bonsai requests until success * wider bonsai loop * switch to async bonsai * patch blocking code * feat: Deterministic builds with receipt cache (+misc) (#80) * readme,cache,cuda,no-dev-mode,receipt uuid output, etc.. * logging nit * update readme * rpc_cache -> cache_rpc * det build, receipt cache, info->debug * fix nondet receipt label * logging and TransactionKind rlp decode fix * nit * move more info output to trace * fix CI
hashcashier
added a commit
that referenced
this pull request
Feb 5, 2024
hashcashier
added a commit
that referenced
this pull request
Feb 5, 2024
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * test for warnings * nits * docker in readme * create cache file if not found * update copyright * update copyright --------- Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Wolfgang Welz <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * initial monolith * refactor prompt style * op-info support * delete old host binaries * nits * fix tests * help nit * short params * bump zkvm, fix CI * clippies * save receipts * refactor * move clap structs to cli module * refactor * refactor main monolith * fixes and changes * vs code change * default cache folder nit * build after derive * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * missing cache files for test * update crates * prove on bonsai * verify bonsai receipts * readme,cache,cuda,no-dev-mode,receipt uuid output, etc.. * logging nit * rlp decoding transactions * provably bad blocks * build on derive * derivation test fix * support erroneous preflights * host: derive with receipts * fix op head * verification nit * repeat bonsai requests until success * wider bonsai loop * switch to async bonsai * update readme * patch blocking code * patch blocking code * rpc_cache -> cache_rpc * det build, receipt cache, info->debug * fix nondet receipt label * logging and TransactionKind rlp decode fix * update risc0 to release v0.20 * cargo locks * nit * move more info output to trace * update GH action to 0.20 * use old actions * stark2snark * changes * refactor rlp & comments * update to v20.1 * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * fmt * SNARK Verification via RPC call * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * test for warnings * nits * docker in readme * remove env unset code because #91 --------- Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Wolfgang Welz <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 12, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * deterministic cache * test for warnings * nits * docker in readme * create cache file if not found * fmt * update copyright * update copyright --------- Co-authored-by: Rami Khalil <[email protected]> Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Rami <[email protected]>
hashcashier
added a commit
that referenced
this pull request
Feb 13, 2024
* transactions module * TxEssence trait * Generic Transaction struct * tx essence as a generic parameter * OptimismTxEssence * op chain spec * OpTxExecStrategy * redundant block builder type params * strategy bundles * host binary parameters * profiling flag * optimism * new derivation binary skeleton, copy over libs * providers and conversion utils * epoch transitioning * basic derive flow * host-side derivation * read metadata from op head * op-derive guest * disable guest memory leaks * heapless batch derivation * Add missing import * Remove heapless BinaryHeap * Remove heapless * Introduce op-derive tool * Remove ethers types from BatcherDb trait * Verify new op block has correct transaction list * Move derive logic into library * Fix bug in transaction trie reconstruction * Introduce get_op_header() to BatcherDb * Clippy warning * Default Serde value for FileProvider::receipts * Disable bloom filter checks * Clippy * Clippy * Fix parsing of from and to fields for deposits * Reintroduce filtering by log bloom * fmt * Add support for local exec to op-derive * Enforce block_number is correct in MemDb * Split derive() into multiple functions * Remove redundant check for batch parent hash * Remove redundant copy of system config * Remove redundant block number check * Cleanup * Remove redundant vector of eth blocks * Add base_fee_per_gas to Epoch * Store deposits in Epoch * Add Eth tail to DeriveOutput * Cleanup * Move deque_next_epoch_if_none to State * Move eth block processing to Batches * update zkvm, basic guest * more compose guest code, comment bonsai code * untested composition draft guest code * Reorg and cleanup * More cleanup * run cargo fmt --all * fix clippy warnings * remove unused imports * basic prep/left/fin in-memory flow * in memory aggregation workflow * format guest code * composition with receipts * add op-derive cmd test * add cmd tests * cleanup optimism/mod * Rename command line args * Fix test arguments * Rename config field to max_channel_bank_size * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity * Import from std instead of alloc/core * Re-enable core::mem::forget() optimization * Replace asserts with ensures; enforce absence of receipts for Op blocks * Simplfy iteration through derived transactions * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling * More println * Rework Batcher initialization * More logging * More log output if Bonsai workflow fails * refactor rpc db * variable derive step support * re-enable profiling * ignore rpc_cache dir * add -profile support to compose binary * upgrade zkvm, modify code comments * bump zkvm, fix CI * clippies * fixes and changes * vs code change * disambiguate merkle ranges and proofs Co-authored-by: Wolfgang Welz <[email protected]> * update risc0 to release v0.20 * update GH action to 0.20 * use old actions * changes * refactor: Unified zeth utility (#72) * feat: Provably build derived op blocks (#78) * feat: Composition via Bonsai (#79) * feat: Deterministic builds with receipt cache (+misc) (#80) * cleanup toml files * remove unused imports * cli cleanup * change block count to u32 * test composition * fix arguments * update risc0 to 0.20.1 * compact json files * update lock files * improve execution * test for warnings * nits * docker in readme * create cache file if not found * update copyright * update copyright --------- Co-authored-by: Rami Khalil <[email protected]> Co-authored-by: Timothy Carstens <[email protected]> Co-authored-by: Rami <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves the usability of zeth as follows:
info-level spam (messages moved to debug)debug-guest-buildfeature)