Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferred calls #4723

Draft
wants to merge 103 commits into
base: mainnet_2_3
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
19e89b4
Add structure async call and async call id
AurelienFT May 9, 2024
4407e86
progress
damip May 9, 2024
b6948d0
progress
damip May 9, 2024
bcfaed3
todos
damip May 9, 2024
eb59c84
Improve general behavior
damip May 10, 2024
5315ea4
typo
damip May 10, 2024
d4300c3
reformat registry
damip May 10, 2024
60cbf28
make things compile
damip May 10, 2024
42c47cd
updates
damip May 21, 2024
37a1961
Merge remote-tracking branch 'aurelien/new_asc' into deferred_calls
modship Jul 8, 2024
94016cb
update new interface
modship Jul 8, 2024
d62dbee
deferred calls first step
modship Jul 9, 2024
d18ed1b
rename massa_asc package to massa_deferred_calls
modship Jul 9, 2024
8b3b2da
TDD for ser_deser validation
modship Jul 10, 2024
7d7e850
Test(DeferredRegistrySlotChanges) : Ser/deser
modship Jul 10, 2024
f945386
Test(DeferredRegistryChanges) : Ser/Deser
modship Jul 10, 2024
9814537
update StateChanges Deserializer
modship Jul 10, 2024
90202d3
create macros, put_entry , delete_entry on DB
modship Jul 12, 2024
698a033
refactor + get_total_gas
modship Jul 12, 2024
cfe824a
get_slot_calls && get_slot_base_fee
modship Jul 15, 2024
1a86834
add mod tests / deferred_call_quote
modship Jul 16, 2024
9d74423
get_deferred_call_quote (interface_impl)
modship Jul 16, 2024
e1f2eca
register call
modship Jul 17, 2024
1bcc026
register set gas
modship Jul 17, 2024
14ca8d5
deferred_call_cancel
modship Jul 17, 2024
4a0c7db
execute deferred call, advance_slot, snapshot
modship Jul 24, 2024
ee901f1
execution trace && execution info
modship Jul 24, 2024
693cf0e
delete call from DB
modship Jul 24, 2024
43923d1
slot remaining gas for async msg
modship Jul 25, 2024
e94c57f
in case of fail, refound sender
modship Jul 25, 2024
6297693
get_slot from DeferredCallId
modship Jul 25, 2024
8bbb687
Fix(ser/deser): error and update test
modship Jul 26, 2024
801e65c
Fix(exec_deferred_call) : Fix lock
modship Jul 26, 2024
d59a556
Test(deferred_call) : first execution test
modship Jul 26, 2024
cd44aa1
fix(deferred_call) : remove double delete_call
modship Jul 26, 2024
cc14cdc
Test(abi) : First test for register deferred_call
modship Jul 29, 2024
1668f4d
Test(register) : more assert
modship Jul 30, 2024
24e09fe
Fix(set_total_gas) : fix set / keep
modship Jul 31, 2024
64f6c24
Test(deferred_call) : Register call fail (total_gas)
modship Jul 31, 2024
9a022ef
Fix(execution) : cancel deferred on check target sc + test refound
modship Jul 31, 2024
08bbef2
update wasm file
modship Jul 31, 2024
9fb3c0e
Fix(deferred_call) : fix compute call fee
modship Aug 1, 2024
82938ec
Fix(deferred_call) : some fix
modship Aug 1, 2024
3ce418f
Test(deferred_call) : Add slot base fee
modship Aug 1, 2024
baa1c3a
rename DEFERRED_CALLS_SLOT_PREFIX to DEFERRED_CALLS_PREFIX
modship Aug 14, 2024
87bb3a2
deferred calls ser/deser error msg
modship Aug 14, 2024
1dcab48
deferred_call_id deserializer limit
modship Aug 14, 2024
ac7ccde
models error
modship Aug 14, 2024
ebe33e1
update max_async_gas for async msg
modship Aug 16, 2024
12971b5
remove useless condition
modship Aug 16, 2024
1d61d14
overbooking_fee return Result
modship Aug 16, 2024
8f82d3e
fix operator
modship Aug 16, 2024
6c1caec
add penalty constant
modship Aug 16, 2024
f3331b5
pr comment # 1
modship Aug 26, 2024
9664ca7
Update massa-execution-worker/src/context.rs
modship Aug 26, 2024
a08664e
pr comment # 2
modship Aug 26, 2024
b53c77f
create DeferredCallsConfig
modship Aug 26, 2024
91ac5e3
use config struct
modship Aug 27, 2024
fe347bb
clippy
modship Aug 27, 2024
07863fc
allow unused_macros
modship Aug 27, 2024
5f28f44
fix : massa-bootstrap import
modship Aug 27, 2024
fd69b2e
wrap deferred call execution into a closure
modship Aug 28, 2024
67ad55f
update sc runtime && wasm test file
modship Aug 28, 2024
6815f44
update interface params from bytes to string
modship Aug 29, 2024
db87465
first metrics
modship Aug 29, 2024
d49dda7
Test: Add check on total_gas after execution
modship Aug 29, 2024
f43fdea
Test : Add 2nd call in scenario and check total_gas
modship Aug 30, 2024
5a58f5f
move type Set Update Delete from Ledger to massa_models
modship Sep 2, 2024
3f5f3cc
remove todo
modship Sep 3, 2024
4df7053
add vm allocation gas cost
modship Sep 4, 2024
c8ca2a8
remove todo
modship Sep 5, 2024
3dcd4ab
Fee: calc relu_occupancy_before
modship Sep 5, 2024
b5dae7c
update sc runtime
modship Sep 5, 2024
89c4d1d
add abi gas cost
modship Sep 5, 2024
039e9ba
Fix : total gas booked
modship Sep 6, 2024
562c16f
BugFix && add nb_call_registered
modship Sep 18, 2024
9583fc7
API : Jsonrpc DeferredCallQuote
modship Sep 20, 2024
341db02
API : JsonRpc get_deferred_call_info && list_deferred_calls_by_slot
modship Sep 23, 2024
4f7524d
API : openrpc spec
modship Sep 24, 2024
5b68c00
grpc api
modship Sep 26, 2024
815ed4c
Fix grpc log error
modship Sep 27, 2024
fd9d945
API : Update list call ids by slot
modship Sep 27, 2024
ed88731
rename openrpc method
modship Sep 27, 2024
7cd95b3
Fix : reset_to_snapshot
modship Oct 1, 2024
08be8a9
Fix: reset_to_snapshot
modship Oct 1, 2024
bc225ec
FIX : Api double alloc vm cst
modship Oct 2, 2024
a4e21a3
Fix : compute_fee get_effective_total_gas()
modship Oct 2, 2024
a4f3a6d
Update runtime
modship Oct 2, 2024
34d9975
Fix abi gas cost
modship Oct 2, 2024
9e57b53
Test : compute_call_fee setup && first basic test
modship Oct 2, 2024
6d25ee6
Test : add test for deferred call exists
modship Oct 3, 2024
2aea5a0
clippy
modship Oct 3, 2024
46fccbe
Fix : fee calculation
modship Oct 7, 2024
8ebf906
add storage cost for deferred call
modship Oct 16, 2024
2af6725
Merge remote-tracking branch 'origin/mainnet_2_3' into deferred_calls
modship Oct 18, 2024
88ecb0d
Storage: Refund sender (storage cost for params)
modship Oct 18, 2024
ff28375
Test : deferred call quote
modship Oct 21, 2024
64f4cdb
update runtime
modship Oct 21, 2024
95e1bc8
Fix all feature && update runtime
modship Oct 21, 2024
c8e45d8
fix doc
modship Oct 21, 2024
2453a80
Update storage cost for deferred calls and set MAX_FUNCTION_NAME_LENG…
modship Oct 30, 2024
1ef9d71
Fix comment
modship Oct 30, 2024
dd31e59
condom limits default with gas_calibration
modship Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,609 changes: 778 additions & 831 deletions Cargo.lock

Large diffs are not rendered by default.

13 changes: 9 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ massa_consensus_exports = { path = "./massa-consensus-exports" }
massa_consensus_worker = { path = "./massa-consensus-worker" }
massa_db_exports = { path = "./massa-db-exports" }
massa_db_worker = { path = "./massa-db-worker" }
massa_deferred_calls = { path = "./massa-deferred-calls" }
massa_executed_ops = { path = "./massa-executed-ops" }
massa_execution_exports = { path = "./massa-execution-exports" }
massa_execution_worker = { path = "./massa-execution-worker" }
Expand Down Expand Up @@ -106,12 +107,16 @@ massa_versioning = { path = "./massa-versioning" }
massa_wallet = { path = "./massa-wallet" }

# Massa projects dependencies
massa-proto-rs = { git = "https://github.com/massalabs/massa-proto-rs", branch = "mainnet_2_3" }
massa-sc-runtime = { git = "https://github.com/massalabs/massa-sc-runtime", "branch" = "next_breaking_update" }
# massa-proto-rs = { git = "https://github.com/massalabs/massa-proto-rs", branch = "deferred_calls" }
# massa-sc-runtime = { git = "https://github.com/massalabs/massa-sc-runtime", "branch" = "deferred_calls" }
massa-proto-rs = { git = "https://github.com/massalabs/massa-proto-rs", "rev" = "b5267178eaf266ec724691d7de163e4c34343416" }
massa-sc-runtime = { git = "https://github.com/massalabs/massa-sc-runtime", "rev" = "82dd714d38cbcd863ae0bb215c06442917bb6404" }


peernet = { git = "https://github.com/massalabs/PeerNet", "rev" = "04b05ddd320fbe76cc858115af7b5fc28bdb8310" }
# Dev only - use local dependencies
# massa-proto-rs = { path = "../massa-proto-rs" }
# massa-sc-runtime = { path = "../massa-sc-runtime_3" }
# massa-sc-runtime = { path = "../massa-sc-runtime" }
# peernet = { path = "../peernet" }

# Common dependencies
Expand Down Expand Up @@ -179,7 +184,7 @@ serde = "1.0"
serde_json = "1.0"
serde_qs = "0.12"
serde_with = "3.3"
serde_yaml = "0.9.34" # 0.9.34+deprecated
serde_yaml = "0.9.34" # 0.9.34+deprecated
serial_test = "2.0"
sha2 = "=0.10"
sha3 = "=0.10"
Expand Down
1 change: 1 addition & 0 deletions massa-api-exports/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ massa_final_state = {workspace = true}
massa_hash = {workspace = true}
massa_wallet = {workspace = true}
massa_versioning = {workspace = true}
massa_deferred_calls = { workspace = true }

[dev-dependencies]
serial_test = {workspace = true}
3 changes: 3 additions & 0 deletions massa-api-exports/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Copyright (c) 2022 MASSA LABS <[email protected]>

use massa_deferred_calls::config::DeferredCallsConfig;
use massa_models::amount::Amount;
use massa_signature::KeyPair;
use massa_time::MassaTime;
Expand Down Expand Up @@ -84,4 +85,6 @@ pub struct APIConfig {
pub deferred_credits_delta: MassaTime,
/// minimal fees to include an operation in a block
pub minimal_fees: Amount,
/// deferred calls config
pub deferred_calls_config: DeferredCallsConfig,
}
43 changes: 43 additions & 0 deletions massa-api-exports/src/execution.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Copyright (c) 2022 MASSA LABS <[email protected]>

use massa_deferred_calls::DeferredCall;
use massa_final_state::StateChanges;
use massa_models::{
address::Address, amount::Amount, block_id::BlockId, operation::OperationId,
Expand Down Expand Up @@ -121,3 +122,45 @@ pub struct Transfer {
/// Context
pub context: TransferContext,
}

/// request for deferred call quote
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct DeferredCallsQuoteRequest {
/// The slot at which the deferred call is to be executed.
pub target_slot: Slot,
/// The maximum gas requested.
pub max_gas_request: u64,
/// Size of parameters
pub params_size: u64,
}

/// The response to a request for a deferred call quote.
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct DeferredCallsQuoteResponse {
/// The slot at which the deferred call is to be executed.
pub target_slot: Slot,
/// The maximum gas requested.
pub max_gas_request: u64,
/// if the slot is bookable
pub available: bool,
/// the cost for booking the call
pub price: Amount,
}

/// response for deferred call
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct DeferredCallResponse {
/// deferred call id
pub call_id: String,
/// deferred call
pub call: DeferredCall,
}

/// response for deferred calls by slot
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct DeferredCallsSlotResponse {
/// deferred calls
pub slot: Slot,
/// deferred calls
pub call_ids: Vec<String>,
}
26 changes: 25 additions & 1 deletion massa-api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@ use jsonrpsee::proc_macros::rpc;
use jsonrpsee::server::middleware::http::HostFilterLayer;
use jsonrpsee::server::{BatchRequestConfig, PingConfig, ServerBuilder, ServerHandle};
use jsonrpsee::RpcModule;
use massa_api_exports::execution::Transfer;
use massa_api_exports::execution::{
DeferredCallResponse, DeferredCallsQuoteRequest, DeferredCallsQuoteResponse,
DeferredCallsSlotResponse, Transfer,
};
use massa_api_exports::{
address::{AddressFilter, AddressInfo},
block::{BlockInfo, BlockSummary},
Expand Down Expand Up @@ -401,6 +404,27 @@ pub trait MassaRpc {
/// Get OpenRPC specification.
#[method(name = "rpc.discover")]
async fn get_openrpc_spec(&self) -> RpcResult<Value>;

/// DeferredCall quote
#[method(name = "get_deferred_call_quote")]
async fn get_deferred_call_quote(
&self,
arg: Vec<DeferredCallsQuoteRequest>,
) -> RpcResult<Vec<DeferredCallsQuoteResponse>>;

/// DeferredCall get info
#[method(name = "get_deferred_call_info")]
async fn get_deferred_call_info(
&self,
arg: Vec<String>,
) -> RpcResult<Vec<DeferredCallResponse>>;

/// List deferred calls for given slot
#[method(name = "get_deferred_call_ids_by_slot")]
async fn get_deferred_call_ids_by_slot(
&self,
arg: Vec<Slot>,
) -> RpcResult<Vec<DeferredCallsSlotResponse>>;
}

fn wrong_api<T>() -> RpcResult<T> {
Expand Down
26 changes: 25 additions & 1 deletion massa-api/src/private.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@ use massa_api_exports::{
datastore::{DatastoreEntryInput, DatastoreEntryOutput},
endorsement::EndorsementInfo,
error::ApiError,
execution::{ExecuteReadOnlyResponse, ReadOnlyBytecodeExecution, ReadOnlyCall, Transfer},
execution::{
DeferredCallResponse, DeferredCallsQuoteRequest, DeferredCallsQuoteResponse,
DeferredCallsSlotResponse, ExecuteReadOnlyResponse, ReadOnlyBytecodeExecution,
ReadOnlyCall, Transfer,
},
node::NodeStatus,
operation::{OperationInfo, OperationInput},
page::{PageRequest, PagedVec},
Expand Down Expand Up @@ -351,6 +355,26 @@ impl MassaRpcServer for API<Private> {
)
}

async fn get_deferred_call_quote(
&self,
_req: Vec<DeferredCallsQuoteRequest>,
) -> RpcResult<Vec<DeferredCallsQuoteResponse>> {
crate::wrong_api::<Vec<DeferredCallsQuoteResponse>>()
}
async fn get_deferred_call_info(
&self,
_arg: Vec<String>,
) -> RpcResult<Vec<DeferredCallResponse>> {
crate::wrong_api::<Vec<DeferredCallResponse>>()
}

async fn get_deferred_call_ids_by_slot(
&self,
_slot: Vec<Slot>,
) -> RpcResult<Vec<DeferredCallsSlotResponse>> {
crate::wrong_api::<Vec<DeferredCallsSlotResponse>>()
}

async fn get_openrpc_spec(&self) -> RpcResult<Value> {
crate::wrong_api::<Value>()
}
Expand Down
143 changes: 134 additions & 9 deletions massa-api/src/public.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ use massa_api_exports::{
endorsement::EndorsementInfo,
error::ApiError,
execution::{
ExecuteReadOnlyResponse, ReadOnlyBytecodeExecution, ReadOnlyCall, ReadOnlyResult, Transfer,
DeferredCallResponse, DeferredCallsQuoteRequest, DeferredCallsQuoteResponse,
DeferredCallsSlotResponse, ExecuteReadOnlyResponse, ReadOnlyBytecodeExecution,
ReadOnlyCall, ReadOnlyResult, Transfer,
},
node::NodeStatus,
operation::{OperationInfo, OperationInput},
Expand All @@ -37,20 +39,17 @@ use massa_models::{
composite::PubkeySig,
config::CompactConfig,
datastore::DatastoreDeserializer,
endorsement::EndorsementId,
endorsement::SecureShareEndorsement,
deferred_calls::DeferredCallId,
endorsement::{EndorsementId, SecureShareEndorsement},
error::ModelsError,
execution::EventFilter,
node::NodeId,
operation::OperationDeserializer,
operation::OperationId,
operation::{OperationType, SecureShareOperation},
operation::{OperationDeserializer, OperationId, OperationType, SecureShareOperation},
output_event::SCOutputEvent,
prehash::{PreHashMap, PreHashSet},
secure_share::SecureShareDeserializer,
slot::{IndexedSlot, Slot},
timeslots,
timeslots::{get_latest_block_slot_at_timestamp, time_range_to_slot_range},
timeslots::{self, get_latest_block_slot_at_timestamp, time_range_to_slot_range},
version::Version,
};
use massa_pool_exports::PoolController;
Expand All @@ -63,8 +62,8 @@ use massa_versioning::versioning_factory::FactoryStrategy;
use massa_versioning::{
keypair_factory::KeyPairFactory, versioning::MipStore, versioning_factory::VersioningFactory,
};
use std::collections::BTreeMap;
use std::net::{IpAddr, SocketAddr};
use std::{collections::BTreeMap, str::FromStr};

impl API<Public> {
/// generate a new public API
Expand Down Expand Up @@ -1157,6 +1156,132 @@ impl MassaRpcServer for API<Public> {
Ok(res?)
}

async fn get_deferred_call_quote(
&self,
req: Vec<DeferredCallsQuoteRequest>,
) -> RpcResult<Vec<DeferredCallsQuoteResponse>> {
if req.len() as u64 > self.0.api_settings.max_arguments {
return Err(ApiError::BadRequest("too many arguments".into()).into());
}

let queries: Vec<ExecutionQueryRequestItem> = req
.into_iter()
.map(|call| ExecutionQueryRequestItem::DeferredCallQuote {
target_slot: call.target_slot,
max_gas_request: call.max_gas_request,
params_size: call.params_size,
})
.collect();

let result = self
.0
.execution_controller
.query_state(ExecutionQueryRequest { requests: queries })
.responses
.into_iter()
.map(|response| match response {
Ok(ExecutionQueryResponseItem::DeferredCallQuote(
target_slot,
max_gas_request,
available,
price,
)) => Ok(DeferredCallsQuoteResponse {
target_slot,
max_gas_request,
available,
price,
}),
Ok(_) => Err(ApiError::InternalServerError(
"unexpected response type".to_string(),
)),
Err(err) => Err(ApiError::InternalServerError(err.to_string())),
})
.collect::<Result<Vec<DeferredCallsQuoteResponse>, ApiError>>()?;

Ok(result)
}

async fn get_deferred_call_info(
&self,
arg: Vec<String>,
) -> RpcResult<Vec<DeferredCallResponse>> {
if arg.len() as u64 > self.0.api_settings.max_arguments {
return Err(ApiError::BadRequest("too many arguments".into()).into());
}

let requests: Vec<ExecutionQueryRequestItem> = arg
.into_iter()
.map(|id_str| {
DeferredCallId::from_str(&id_str)
.map_err(|e| ApiError::BadRequest(e.to_string()))
.map(ExecutionQueryRequestItem::DeferredCallInfo)
})
.collect::<Result<_, _>>()?;

let result: Vec<DeferredCallResponse> = self
.0
.execution_controller
.query_state(ExecutionQueryRequest { requests })
.responses
.into_iter()
.map(|exec| match exec {
Ok(ExecutionQueryResponseItem::DeferredCallInfo(id, call)) => {
Ok(DeferredCallResponse {
call_id: id.to_string(),
call,
})
}
Ok(_) => Err(ApiError::InternalServerError(
"unexpected response type".to_string(),
)),
Err(err) => Err(ApiError::InternalServerError(err.to_string())),
})
.collect::<Result<_, _>>()?;

Ok(result)
}

async fn get_deferred_call_ids_by_slot(
&self,
slots: Vec<Slot>,
) -> RpcResult<Vec<DeferredCallsSlotResponse>> {
if slots.len() as u64 > self.0.api_settings.max_arguments {
return Err(ApiError::BadRequest("too many arguments".into()).into());
}

let requests: Vec<ExecutionQueryRequestItem> = slots
.into_iter()
.map(ExecutionQueryRequestItem::DeferredCallsBySlot)
.collect();

let mut slot_calls = Vec::new();

for exec in self
.0
.execution_controller
.query_state(ExecutionQueryRequest { requests })
.responses
.into_iter()
{
match exec {
Ok(ExecutionQueryResponseItem::DeferredCallsBySlot(slot, result)) => {
let call_ids = result.into_iter().map(|id| id.to_string()).collect();

slot_calls.push(DeferredCallsSlotResponse { slot, call_ids });
}
Ok(_) => {
return Err(ApiError::InternalServerError(
"unexpected response type".to_string(),
)
.into())
}
Err(err) => return Err(ApiError::InternalServerError(err.to_string()).into()),
}
}

Ok(slot_calls)
}

/// send operations
async fn send_operations(&self, ops: Vec<OperationInput>) -> RpcResult<Vec<OperationId>> {
let mut cmd_sender = self.0.pool_command_sender.clone();
Expand Down
2 changes: 2 additions & 0 deletions massa-api/src/tests/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ pub(crate) fn get_apiv2_server(addr: &SocketAddr) -> (API<ApiV2>, APIConfig) {
chain_id: *CHAINID,
deferred_credits_delta: MassaTime::from_millis(24 * 3600 * 2),
minimal_fees: Amount::zero(),
deferred_calls_config: Default::default(),
};

// let shared_storage: massa_storage::Storage = massa_storage::Storage::create_root();
Expand Down Expand Up @@ -146,6 +147,7 @@ pub(crate) fn start_public_api(addr: SocketAddr) -> (API<Public>, APIConfig) {
chain_id: *CHAINID,
deferred_credits_delta: MassaTime::from_millis(24 * 3600 * 2),
minimal_fees: Amount::zero(),
deferred_calls_config: Default::default(),
};

let shared_storage: massa_storage::Storage = massa_storage::Storage::create_root();
Expand Down
Loading