Add exchange-oracle crate and implement get_exchange_rate for coinGecko #442

echevrier · 2021-10-06T14:10:28Z

DO NOT MERGE THIS INTO MASTER
it's a showcase that may reside on a branch for now

app-libs/exchange-oracle/src/coingecko.rs

app-libs/exchange-oracle/src/lib.rs

app-libs/exchange-oracle/src/coingecko.rs

enclave-runtime/Cargo.lock

enclave-runtime/Cargo.toml

enclave-runtime/src/lib.rs

core-primitives/enclave-api/src/teeracle_api.rs

app-libs/exchange-oracle/Cargo.toml

app-libs/exchange-oracle/src/coingecko.rs

murerfel

Good work! Some minor improvements I could identify. And I think it makes sense to wait a bit until #498 is merged and then update this PR again. Otherwise the code is already obsolete again.

app-libs/exchange-oracle/src/coingecko.rs

app-libs/exchange-oracle/src/error.rs

murerfel · 2021-11-08T15:39:16Z

core-primitives/enclave-api/src/teeracle_api.rs

+				&mut retval,
+				gen.as_ptr(),
+				gen.len() as u32,
+				&nonce,


The nonce is now handled entirely inside the enclave, so the e-call does not need to pass the nonce. Inside the enclave, we have a global accessor to the nonce. After my next PR (#498 ), we will have a proper nonce cache that can be accessed, rather than just a global static variable (which is not safe for concurrent access)

Genesis hash can also be retrieved from within the enclave

worker/enclave-runtime/src/lib.rs

Line 792 in 11e25fc

validator.genesis_hash(validator.num_relays()).unwrap(),

Validator can be unsealed like the following:

worker/enclave-runtime/src/lib.rs

Line 690 in 11e25fc

let mut validator = LightClientSeal::<PB>::unseal()?;

Though, I'm unsure what's the way to go for currently. @mullefel should the validator be unsealed for every ecall? Or is there already a lazy static like there is for the nonce?

At the moment we still unseal for every e-call. And in order to ensure data consistency, we need to acquire a lock before we unseal the light client (validator):

worker/enclave-runtime/src/lib.rs

Lines 643 to 647 in e959c9c

// we acquire lock explicitly (variable binding), since '_' will drop the lock after the statement

// see https://medium.com/codechain/rust-underscore-does-not-bind-fec6a18115a8

let _light_client_lock = EnclaveLock::write_light_client_db()?;

let mut validator = LightClientSeal::<PB>::unseal()?;

Then, after we're done using the light client, we seal it again and drop the lock (this is done either implicitly by going out of scope, or actively by calling std::mem::drop() on the lock.

worker/enclave-runtime/src/lib.rs

Lines 662 to 663 in e959c9c

// store updated state in light client in case we fail afterwards.

LightClientSeal::seal(validator)?;

If you're only using the light client to get the genesis hash, you can

acquire the light client db read lock (EnclaveLock::read_light_client_db() )

unseal the light client

get the genesis hash

seal the light client immediately afterwards

drop the lock

murerfel · 2021-11-08T15:40:17Z

enclave-runtime/src/lib.rs

+pub unsafe extern "C" fn update_market_data_xt(
+	genesis_hash: *const u8,
+	genesis_hash_size: u32,
+	nonce: &mut u32,


handle nonce inside the enclave, not from outside (see previous comment)

service/src/main.rs

app-libs/exchange-oracle/Cargo.toml

enclave-runtime/src/lib.rs

haerdib

Looks very good! Some picky comments from my side (someone has to substitute @clangenb 😋 ), but good work :)

app-libs/exchange-oracle/Cargo.toml

app-libs/exchange-oracle/src/coingecko.rs

app-libs/exchange-oracle/src/error.rs

app-libs/exchange-oracle/src/coingecko.rs

app-libs/exchange-oracle/src/lib.rs

haerdib · 2021-11-10T08:04:05Z

core-primitives/enclave-api/src/teeracle_api.rs

+				&mut retval,
+				gen.as_ptr(),
+				gen.len() as u32,
+				&nonce,


Genesis hash can also be retrieved from within the enclave

worker/enclave-runtime/src/lib.rs

Line 792 in 11e25fc

validator.genesis_hash(validator.num_relays()).unwrap(),

Validator can be unsealed like the following:

worker/enclave-runtime/src/lib.rs

Line 690 in 11e25fc

let mut validator = LightClientSeal::<PB>::unseal()?;

Though, I'm unsure what's the way to go for currently. @mullefel should the validator be unsealed for every ecall? Or is there already a lazy static like there is for the nonce?

core-primitives/settings/src/lib.rs

service/src/main.rs

author echevrier <[email protected]> 1633529369 +0200 committer echevrier <[email protected]> 1636648987 +0100 Exchange Rate Oracle Showcase Add exchange-oracle crate and implement get_exchange_rate for coinGecko Clean according review Call update market data (Exchange rate) from enclave Rename pallet market to teeracle cargo fmt Set MARKET_DATA_UPDATE_INTERVAL to 24h [GA] bump node artifact Changes from review Set substrate-fixed version to 0.5.6 (before update to new substrate) Changes from review Rebase Changes due to review Rebase to get PR #498 to rework code according review

… the thread is stopped

enclave-runtime/src/lib.rs

murerfel · 2021-11-16T06:47:03Z

@brenzi I think we need the light-client in order to get the genesis hash (in order to create an extrinsic I think?). So there is this small usage of the light-client that we cannot get rid of. That would mean we'd have to modularize it for this use case so it only provides a subset of functionality.

haerdib · 2021-11-16T07:05:25Z

As the whole TEEracle is basically one Ecall, so the genesis hash could also directly be given as a variable into the enclave. Like it's being done for the mock_register

.github/workflows/build_and_test.yml

core-primitives/settings/src/lib.rs

enclave-runtime/src/lib.rs

murerfel · 2021-11-17T10:16:22Z

enclave-runtime/src/lib.rs

+	let mut curr_slice = slice::from_raw_parts(currency, currency_size as usize);
+	let curr: String = Decode::decode(&mut curr_slice).unwrap();
+
+	let xts = match update_market_data_internal(genesis_hash, curr) {


xt is also somewhat of a gray area - probably extrinsics would be better as well. A new developer won't know that xt = extrinsic

Here again xt, xts are used for extrinsic (s) throughout the code. Even in function names. I change it here, but we should apply theses changes all over the code.

service/src/main.rs

murerfel · 2021-11-17T10:18:06Z

service/src/main.rs

+	let mut xthex = hex::encode(uxt);
+	xthex.insert_str(0, "0x");


hex_encoded_extrinsic

service/src/main.rs

murerfel

Approved. Thanks for putting my REST-client implementation to the test!

clangenb

Looks very nice in general. I only found the dependency issue with substrate fixed. Can you update that? @echevrier

enclave-runtime/Cargo.toml

clangenb · 2021-11-19T09:01:54Z

Cargo.lock

@@ -3910,7 +3926,7 @@ dependencies = [
 "sp-io 4.0.0-dev (git+https://github.com/paritytech/substrate.git?branch=master)",
 "sp-runtime",
 "sp-std",
- "substrate-fixed",
+ "substrate-fixed 0.5.7",


We seem to have two versions of substrate-fixed here.

clangenb · 2021-11-19T09:02:44Z

service/Cargo.toml

 # scs / integritee
 substrate-api-client = { git = "https://github.com/scs/substrate-api-client", branch = "master" }
 my-node-runtime = { package = "integritee-node-runtime", git = "https://github.com/integritee-network/integritee-node", branch = "master" }
+substrate-fixed = { package = "substrate-fixed", git = "https://github.com/encointer/substrate-fixed", tag = "v0.5.6" }


Also tag v0.5.7

…ko (#442) author echevrier <[email protected]> 1633529369 +0200 committer echevrier <[email protected]> 1636648987 +0100 Exchange Rate Oracle Showcase Add exchange-oracle crate and implement get_exchange_rate for coinGecko Clean according review Call update market data (Exchange rate) from enclave Rename pallet market to teeracle cargo fmt Set MARKET_DATA_UPDATE_INTERVAL to 24h [GA] bump node artifact Changes from review Set substrate-fixed version to 0.5.6 (before update to new substrate) Changes from review Rebase Changes due to review Rebase to get PR #498 to rework code according review * Rebase * cargo clippy and catch error in execute_update_market to prevent that the thread is stopped * settings for demo * update CI because this branch will not be merged in master, but in exchange_rate_oracle branch * changes from review * changes from review * Remove reference to light client for oracle * Set update interval for exchange range to 24h * Changes review * Changes review * Cargo clippy * Changes from review: improve variable names Co-authored-by: echevrier <[email protected]>

echevrier requested a review from murerfel October 6, 2021 14:10