Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(l2): integrate ExecutionDB, prove execution #956

Merged
merged 83 commits into from
Oct 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
45c9a95
add libmdbx features
fborello-lambda Oct 16, 2024
0b6658b
bump-up versions
fborello-lambda Oct 16, 2024
b154097
add de/serialization for Block and Transaction
fborello-lambda Oct 16, 2024
ab231dd
add zkvm program ci job
xqft Oct 16, 2024
8957079
misc fixes on ci job
xqft Oct 16, 2024
330177c
change names ci jobs
xqft Oct 16, 2024
88d0f71
wip: add basic structure
fborello-lambda Oct 16, 2024
cf4d670
fix: run tests with no-default-features
fborello-lambda Oct 16, 2024
6a794f2
Merge branch 'l2/prover_integration' into l2/prover_base
fborello-lambda Oct 16, 2024
518c30d
add de/serialization for Receipt
fborello-lambda Oct 16, 2024
0703ce6
Merge branch 'l2/prover_integration' into l2/prover_base
fborello-lambda Oct 16, 2024
687619c
Revert "change names ci jobs"
xqft Oct 17, 2024
33db110
fix prover job
xqft Oct 17, 2024
2190010
fix job binary path
xqft Oct 17, 2024
1604131
fix cargo prove
xqft Oct 17, 2024
0312121
nit
xqft Oct 17, 2024
4d01428
add prover crate to workspace
fborello-lambda Oct 17, 2024
f49e67e
fix: rm clone
fborello-lambda Oct 17, 2024
408e256
fix: replace todo! with custom error
fborello-lambda Oct 17, 2024
056094d
rm derive Deserialize for EIP1559
fborello-lambda Oct 17, 2024
e573135
refactor transactions deser and derive deserialize on enum
xqft Oct 17, 2024
7f0f7e8
Revert "refactor transactions deser and derive deserialize on enum"
xqft Oct 17, 2024
24bc2d2
refactor transactions deser and derive deserialize on enum
xqft Oct 17, 2024
702c11b
rm underscore form path variable
fborello-lambda Oct 17, 2024
2698e5b
add missing Serialize impl for EIP1559
fborello-lambda Oct 17, 2024
6ecac43
Merge branch 'l2/prover_integration' into l2/prover_base
fborello-lambda Oct 17, 2024
980fea6
wip
fborello-lambda Oct 17, 2024
bbb3976
Merge branch 'main' into l2/prover_base
xqft Oct 18, 2024
ce5c073
[wip] run prover with ethereum_rust_l2 cli
fborello-lambda Oct 18, 2024
c1498ab
integrate prover's client with existing setup
fborello-lambda Oct 18, 2024
4e4e7e5
Merge branch 'main' into l2/prover_base
fborello-lambda Oct 18, 2024
02282f1
fix
fborello-lambda Oct 18, 2024
bba68f1
add Makefile target
fborello-lambda Oct 21, 2024
9227e46
use libmdbx
fborello-lambda Oct 21, 2024
14bf100
Merge branch 'main' into l2/prover_base
fborello-lambda Oct 21, 2024
1733e0d
renaming
fborello-lambda Oct 21, 2024
32a0328
revert .gitignore
xqft Oct 21, 2024
bedc3e6
change MemoryDB to more descriptive
xqft Oct 21, 2024
83875cb
remove commented dependency
xqft Oct 21, 2024
ea7abc3
remove unwrap
xqft Oct 21, 2024
e9772cd
add revm dep comments
xqft Oct 21, 2024
5522b6b
fix comment
xqft Oct 21, 2024
49ba37e
tidy mock zkvm program
xqft Oct 21, 2024
54a2e51
improve revm dep comment
xqft Oct 21, 2024
8d54fe5
Merge branch 'main' into l2/prover_base
xqft Oct 21, 2024
1eefc32
change unused fn name
xqft Oct 21, 2024
27b56a6
Merge branch 'main' into l2/prover_base
fborello-lambda Oct 22, 2024
34ec094
fix zkvm ci job
xqft Oct 23, 2024
d507c03
Merge branch 'main' into l2/prover_base
xqft Oct 23, 2024
589a6b4
refactor(l2): new zkvm (#939)
fborello-lambda Oct 23, 2024
e2e15be
renaming and fix CI
fborello-lambda Oct 23, 2024
65d8c39
fix ci
fborello-lambda Oct 23, 2024
5b3f84a
chore: docs
fborello-lambda Oct 23, 2024
5bfee10
add cache crate and mock struct
xqft Oct 17, 2024
bcc356d
fix new EvmState
xqft Oct 17, 2024
2271eec
move to vm crate, rename
xqft Oct 18, 2024
08f3a3e
remove newline
xqft Oct 18, 2024
972d552
rename file
xqft Oct 18, 2024
076b6b3
define db fields
xqft Oct 18, 2024
28acd02
fix doc
xqft Oct 18, 2024
82fdfe2
integrate new EvmState
xqft Oct 21, 2024
67236b1
add from_exec()
xqft Oct 23, 2024
d072af3
add/fix comments
xqft Oct 23, 2024
bb1e4b6
add impl for executiondb to get_state_transitions()
xqft Oct 23, 2024
8adff1b
fix database() function, added chain_config()
xqft Oct 23, 2024
9feaf96
replace MemoryDB with ExecutionDB
xqft Oct 23, 2024
baa9fe7
create prover input when handling request
xqft Oct 23, 2024
f188f19
add debug log
xqft Oct 23, 2024
accc380
tidy up zkvm program, remove revm deps
xqft Oct 24, 2024
0f0d864
fix compilation
fborello-lambda Oct 24, 2024
4532715
remove parent header
xqft Oct 24, 2024
499d43e
add better info log
xqft Oct 24, 2024
7db34c6
fix log level
xqft Oct 24, 2024
d249f39
change prover log level
xqft Oct 24, 2024
48cfa4c
change prover log level
xqft Oct 24, 2024
5fcfcd7
fix log level
xqft Oct 24, 2024
8796417
remove unused dep
xqft Oct 24, 2024
d948827
update example env
xqft Oct 24, 2024
2c0f597
feat(l2): generate groth16 proof (#971)
xqft Oct 24, 2024
7f84c9e
feat(l2): prove block validation (#973)
xqft Oct 25, 2024
bdca908
rm [old] .example.env
fborello-lambda Oct 28, 2024
ee89df8
Merge branch 'main' into l2/integrate_cache_db
fborello-lambda Oct 30, 2024
1e7cd7f
fix merge
fborello-lambda Oct 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion crates/blockchain/blockchain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,10 @@ pub fn is_canonical(
}
}

fn validate_gas_used(receipts: &[Receipt], block_header: &BlockHeader) -> Result<(), ChainError> {
pub fn validate_gas_used(
receipts: &[Receipt],
block_header: &BlockHeader,
) -> Result<(), ChainError> {
if let Some(last) = receipts.last() {
if last.cumulative_gas_used != block_header.gas_used {
return Err(ChainError::InvalidBlock(InvalidBlockError::GasUsedMismatch));
Expand Down
1 change: 1 addition & 0 deletions crates/l2/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ ethereum_rust-rlp.workspace = true
ethereum_rust-rpc.workspace = true
ethereum_rust-blockchain.workspace = true
ethereum_rust-storage.workspace = true
ethereum_rust-vm.workspace = true
ethereum_rust-dev = { path = "../../crates/blockchain/dev" }
hex.workspace = true
bytes.workspace = true
Expand Down
46 changes: 37 additions & 9 deletions crates/l2/proposer/prover_server.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use crate::utils::eth_client::RpcResponse;
use ethereum_rust_storage::Store;
use ethereum_rust_vm::execution_db::ExecutionDB;
use reqwest::Client;
use serde::{Deserialize, Serialize};
use std::{
Expand All @@ -14,15 +15,11 @@ use ethereum_rust_core::types::{Block, BlockHeader};

#[derive(Debug, Serialize, Deserialize, Default)]
pub struct ProverInputData {
pub db: MemoryDB,
pub parent_block_header: BlockHeader,
pub db: ExecutionDB,
pub block: Block,
pub parent_header: BlockHeader,
}

// Placeholder structure until we have ExecutionDB on L1
#[derive(Debug, Serialize, Deserialize, Default)]
pub struct MemoryDB;

use crate::utils::config::prover_server::ProverServerConfig;

use super::errors::ProverServerError;
Expand Down Expand Up @@ -177,22 +174,22 @@ impl ProverServer {
) -> Result<(), String> {
debug!("Request received");

//let last_block_number = Self::get_last_block_number().await?;
let last_block_number = self
.store
.get_latest_block_number()
.map_err(|e| e.to_string())?
.ok_or("missing latest block number".to_string())?;
let input = self.create_prover_input(last_block_number)?;

let response = if last_block_number > last_proved_block {
ProofData::Response {
block_number: Some(last_block_number),
input: ProverInputData::default(),
input,
}
} else {
ProofData::Response {
block_number: None,
input: ProverInputData::default(),
input,
}
};
let writer = BufWriter::new(stream);
Expand All @@ -211,4 +208,35 @@ impl ProverServer {
let writer = BufWriter::new(stream);
serde_json::to_writer(writer, &response).map_err(|e| e.to_string())
}

fn create_prover_input(&self, block_number: u64) -> Result<ProverInputData, String> {
let header = self
.store
.get_block_header(block_number)
.map_err(|err| err.to_string())?
.ok_or("block header not found")?;
let body = self
.store
.get_block_body(block_number)
.map_err(|err| err.to_string())?
.ok_or("block body not found")?;

let block = Block::new(header, body);

let db = ExecutionDB::from_exec(&block, &self.store).map_err(|err| err.to_string())?;

let parent_header = self
.store
.get_block_header_by_hash(block.header.parent_hash)
.map_err(|err| err.to_string())?
.ok_or("missing parent header".to_string())?;

debug!("Created prover input for block {block_number}");

Ok(ProverInputData {
db,
block,
parent_header,
})
}
}
16 changes: 11 additions & 5 deletions crates/l2/prover/src/prover.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
use ethereum_rust_core::types::Block;
use tracing::info;

// risc0
use zkvm_interface::methods::{ZKVM_PROGRAM_ELF, ZKVM_PROGRAM_ID};

use risc0_zkvm::{default_prover, ExecutorEnv, ExecutorEnvBuilder};
use risc0_zkvm::{default_prover, ExecutorEnv, ExecutorEnvBuilder, ProverOpts};

use ethereum_rust_rlp::encode::RLPEncode;

Expand Down Expand Up @@ -34,12 +35,12 @@ impl<'a> Prover<'a> {

pub fn set_input(&mut self, input: ProverInputData) -> &mut Self {
let head_block_rlp = input.block.encode_to_vec();
let parent_block_header_rlp = input.parent_block_header.encode_to_vec();
let parent_header_rlp = input.parent_header.encode_to_vec();

// We should pass the inputs as a whole struct
self.env_builder.write(&head_block_rlp).unwrap();
self.env_builder.write(&parent_block_header_rlp).unwrap();
self.env_builder.write(&input.db).unwrap();
self.env_builder.write(&parent_header_rlp).unwrap();

self
}
Expand All @@ -59,13 +60,18 @@ impl<'a> Prover<'a> {
// Proof information by proving the specified ELF binary.
// This struct contains the receipt along with statistics about execution of the guest
let prove_info = prover
.prove(env, self.elf)
.prove_with_opts(env, self.elf, &ProverOpts::groth16())
.map_err(|_| "Failed to prove".to_string())?;

// extract the receipt.
let receipt = prove_info.receipt;

info!("Successfully generated Receipt!");
let executed_block: Block = receipt.journal.decode().map_err(|err| err.to_string())?;

info!(
"Successfully generated execution proof receipt for block {}",
executed_block.header.compute_block_hash()
);
Ok(receipt)
}

Expand Down
11 changes: 5 additions & 6 deletions crates/l2/prover/zkvm/interface/guest/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,8 @@ ethereum_rust-rlp = { path = "../../../../../common/rlp" }
ethereum_rust-vm = { path = "../../../../../vm", default-features = false }
ethereum_rust-blockchain = { path = "../../../../../blockchain", default-features = false }

# revm
revm = { version = "14.0.3", features = [
"std",
"serde",
"kzg-rs",
], default-features = false }
[patch.crates-io]
crypto-bigint = { git = "https://github.com/risc0/RustCrypto-crypto-bigint", tag = "v0.5.5-risczero.0" }
k256 = { git = "https://github.com/risc0/RustCrypto-elliptic-curves", tag = "k256/v0.13.3-risczero.0" }
sha2 = { git = "https://github.com/risc0/RustCrypto-hashes", tag = "sha2-v0.10.6-risczero.0" }
secp256k1 = { git = "https://github.com/sp1-patches/rust-secp256k1", branch = "patch-secp256k1-v0.29.1" }
120 changes: 24 additions & 96 deletions crates/l2/prover/zkvm/interface/guest/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,111 +1,39 @@
use ethereum_rust_rlp::{decode::RLPDecode, error::RLPDecodeError};
use risc0_zkvm::guest::env;

//use ethereum_rust_blockchain::validate_gas_used;
use ethereum_rust_core::types::{Receipt, Transaction};
// We have to import the ExecutionDB.
use ethereum_rust_vm::{block_env, tx_env};

use revm::{
db::CacheDB, inspectors::TracerEip3155, primitives::ResultAndState as RevmResultAndState,
Evm as Revm,
};
use ethereum_rust_blockchain::{validate_block, validate_gas_used};
use ethereum_rust_core::types::{Block, BlockHeader};
use ethereum_rust_vm::{execute_block, execution_db::ExecutionDB, get_state_transitions, EvmState};

fn main() {
// Read the input
let head_block_bytes = env::read::<Vec<u8>>();
let parent_header_bytes = env::read::<Vec<u8>>();
//let execution_db = env::read::<ExecutionDB>();
let (block, execution_db, parent_header) = read_inputs().expect("failed to read inputs");
let mut state = EvmState::from_exec_db(execution_db.clone());

// SetUp data from inputs
let block = <ethereum_rust_core::types::Block as ethereum_rust_rlp::decode::RLPDecode>::decode(
&head_block_bytes,
)
.unwrap();
// Validate the block pre-execution
validate_block(&block, &parent_header, &state).expect("invalid block");

let parent_header =
<ethereum_rust_core::types::BlockHeader as ethereum_rust_rlp::decode::RLPDecode>::decode(
&parent_header_bytes,
)
.unwrap();
let receipts = execute_block(&block, &mut state).unwrap();

// Make DataInputs public.
env::commit(&block);
env::commit(&parent_header);
//env::commit(&execution_db);

// SetUp CacheDB in order to use execute_block()
//let mut cache_db = CacheDB::new(execution_db);
println!("executing block");
validate_gas_used(&receipts, &block.header).expect("invalid gas used");

//let block_receipts = execute_block(&block, &mut cache_db).unwrap();
// TODO
// Handle the case in which the gas used differs and throws an error.
// Should the zkVM panic? Should it generate a dummy proof?
// Private function
//let _ = validate_gas_used(&block_receipts, &block.header);
let _account_updates = get_state_transitions(&mut state);

//env::commit(&block_receipts);
// TODO: compute new state root from account updates and check it matches with the block's
// header one.
}

// Modified from ethereum_rust-vm
/*
fn execute_block(
block: &ethereum_rust_core::types::Block,
db: &mut CacheDB<ExecutionDB>,
) -> Result<Vec<Receipt>, ethereum_rust_vm::EvmError> {
let spec_id = revm::primitives::SpecId::CANCUN;
let mut receipts = Vec::new();
let mut cumulative_gas_used = 0;

for transaction in block.body.transactions.iter() {
let result = execute_tx(transaction, &block.header, db, spec_id)?;
cumulative_gas_used += result.gas_used();
let receipt = Receipt::new(
transaction.tx_type(),
result.is_success(),
cumulative_gas_used,
result.logs(),
);
receipts.push(receipt);
}
fn read_inputs() -> Result<(Block, ExecutionDB, BlockHeader), RLPDecodeError> {
let head_block_bytes = env::read::<Vec<u8>>();
let execution_db = env::read::<ExecutionDB>();
let parent_header_bytes = env::read::<Vec<u8>>();

Ok(receipts)
}
let block = Block::decode(&head_block_bytes)?;
let parent_header = BlockHeader::decode(&parent_header_bytes)?;

// Modified from ethereum_rust-vm
fn execute_tx(
transaction: &Transaction,
block_header: &ethereum_rust_core::types::BlockHeader,
db: &mut CacheDB<ExecutionDB>,
spec_id: revm::primitives::SpecId,
) -> Result<ethereum_rust_vm::ExecutionResult, ethereum_rust_vm::EvmError> {
let block_env = block_env(block_header);
let tx_env = tx_env(transaction);
run_evm(tx_env, block_env, db, spec_id)
.map(Into::into)
.map_err(ethereum_rust_vm::EvmError::from)
}
// make inputs public
env::commit(&block);
env::commit(&execution_db);
env::commit(&parent_header);

// Modified from ethereum_rust-vm
fn run_evm(
tx_env: revm::primitives::TxEnv,
block_env: revm::primitives::BlockEnv,
db: &mut CacheDB<ExecutionDB>,
spec_id: revm::primitives::SpecId,
) -> Result<ethereum_rust_vm::ExecutionResult, ethereum_rust_vm::EvmError> {
// let chain_spec = db.get_chain_config()?;
let mut evm = Revm::builder()
.with_db(db)
.with_block_env(block_env)
.with_tx_env(tx_env)
// If the chain_id is not correct, it throws:
// Transaction(InvalidChainId)
// TODO: do not hardcode the chain_id
.modify_cfg_env(|cfg| cfg.chain_id = 1729)
.with_spec_id(spec_id)
.with_external_context(TracerEip3155::new(Box::new(std::io::stderr())).without_summary())
.build();
let RevmResultAndState { result, state: _ } = evm.transact().unwrap();
Ok(result.into())
Ok((block, execution_db, parent_header))
}
*/
4 changes: 4 additions & 0 deletions crates/vm/vm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ pub enum EvmState {
}

impl EvmState {
pub fn from_exec_db(db: ExecutionDB) -> Self {
EvmState::Execution(revm::db::CacheDB::new(db))
}

/// Get a reference to inner `Store` database
pub fn database(&self) -> Option<&Store> {
if let EvmState::Store(db) = self {
Expand Down
Loading