Release 2437 distribution

[jpraynaud]

Why

In order to deploy new features, improvements and bug fixes, we need to release a new distribution.

What

Create and release a new distribution 2437

Important

• Activate Cardano transactions certification:
  • in release-mainnet
• Upgrade to Cardano node 9.1.1:
  • in pre-release-preview
  • in release-preprod
  • in release-mainnet
• Breaking change in WASM client with unstable features
• Increase signature pace on pre-release-preview

Distribution +1:

Note

• Make client for Cardano transactions stable
• Update Cardano transactions preloader refresh interval default value in signer
• Activate Cardano stake distribution certification:
  • in pre-release-preview
  • in release-preprod
  • in release-mainnet

Distribution +2:

Note

• Make client for Cardano stake distribution stable

How

Phase 1: Prepare distribution

• Test backward compatibility with previous signer version(s)
• Upgrade repository dependencies (with runbook)
• Check that all minor versions of all crates have been bumped for breaking change (if needed)
• Check if versions of repository linked crates are correct for publication to crates.io(mithril-common, mithril-client)
• Check if Cardano minimum supported versions in networks.json must be updated following the runbook
• Merge a PR for rotating the CHANGELOG.md (set the distribution version)
• Create a draft PR for rotating the documentation

Phase 2: Rollout pre-release distribution

• Tag and deploy a 2437.0-pre pre-release distribution on the pre-release-preview network
• Fill 'Highlights' and release notes section in the pre-release page on GitHub
• Make sure pre-release-preview is working as expected
• Update the configuration of pre-release-preview:
  • Upgrade the Cardano node version to 9.1.1
• Restart the pre-release-preview deployment in the CI
• Make sure pre-release-preview is working as expected
• Make sure libraries version next have been published on npm registry
• Run an all green client multi-platform test in the CI
• Run an all green aggregator stress test in the CI
• Run a Cardano transaction prover benchmark:
  • Store results in the issue
  • Make sure that there is no performance drop since last release
• Announce the pre-release on discord channel
• Announce any breaking change in:
  • #cardano-scaling slack channel for pre-release
  • #mithril-lace slack channel for pre-release
  • #blockfrost-scaling slack channel for pre-release
• Wait for at least 3 signers running the new version (could be more if era change is tested) over at least 2 epochs

Phase 2: Rollout pre-release distribution fix

• Tag and deploy a 2437.1-pre pre-release distribution on the pre-release-preview network
• Fill 'Highlights' and release notes section in the pre-release page on GitHub
• Make sure pre-release-preview is working as expected
• Announce the pre-release on discord channel
• Update announcements of breaking change in:
  • #cardano-scaling slack channel for pre-release
  • #mithril-lace slack channel for pre-release
  • #blockfrost-scaling slack channel for pre-release
• Wait for at least 3 signers running the new version (could be more if era change is tested) over at least 2 epochs

Phase 4: Rollout release distribution

• Tag and deploy a 2437.1 release distribution on the pre-release-preview network
• Fill highlights and release notes section in the release page on GitHub
• Make sure pre-release-preview is working as expected
• Update networks compatibility table in release notes
• Update the configuration of release-mainnet:
  • Upgrade the Cardano node version to 9.1.1 in release-mainnet
• Release the 2437.1 distribution
• Make sure release-mainnet is working as expected
• Make sure release-preprod is working as expected
• Update the configuration of release-preprod:
  • Upgrade the Cardano node version to 9.1.1 in release-preprod
• Restart the release-preprod deployment in the CI
• Make sure release-preprod is working as expected
• Make sure libraries have been published on crates.io
• Make sure libraries version latest have been published on npm registry
• Merge the PR for rotating the documentation
• Merge a PR for rotating the CHANGELOG.md (updating the release date of the distribution and the versions table)
• Announce the release on discord channel (with breaking-change, upcoming features activation, ...)
• Announce any breaking/new feature change in:
  • #cardano-scaling slack channel for release
  • #mithril-lace slack channel for release
  • #blockfrost-scaling slack channel for release
• Dev blog post:
  • Updates on the Certification of Cardano transactions dev blog post
  • New Mithril client WASM breaking change dev blog post.

Phase 5: Activate features

• Accelerate the pace of signature for Cardano transactions of pre-release-preview:
  • Signing configuration from 15 from tip of chain and every 15 blocks
• Cardano transactions:
  • When latest signer release is run by > 70% of the stake (within 2 weeks, or abort),
  • Activate the certification of Cardano transaction in release-mainnet
  • Restart the release-mainnet deployment in the CI
  • Make sure release-mainnet is working as expected
  • Announce the release on discord channel
  • Dev blog post: Activation of Cardano transaction in release-mainnet

[jpraynaud]

Cardano transactions prover benchmark

Note

run on GCP with e2-highmem-2 machine with 2 vCPUs and 15GB of RAM
not during Cardano database snapshotting

• Benchmark results:

total_requests	concurrency	transactions/request	requests/s
500	50	1	58.71
500	50	10	49.19
500	50	20	39.17
500	50	30	34.11
500	50	40	29.21
500	50	50	24.39

• Command run:

# Aggregator endpoint
export AGGREGATOR_ENDPOINT=https://aggregator.pre-release-preview.api.mithril.network/aggregator

# Transactions file to prove
export TRANSACTIONS_FILE=transactions-preview.txt

# Transactions proved per request range definition
export TRANSACTIONS_PER_REQUEST_MIN=0
export TRANSACTIONS_PER_REQUEST_MAX=50
export TRANSACTIONS_PER_REQUEST_STEP=10

# Apache benchmark total request sent per benchmark
export AB_TOTAL_REQUESTS=500

# Apache benchmark concurrency level range definition
export AB_CONCURRENCY_MIN=50
export AB_CONCURRENCY_MAX=50
export AB_CONCURRENCY_STEP=50

# Run the benchmark:
./benchmark-aggregator-prover.sh

• Command result:

MITHRIL AGGREGATOR PROVER ROUTE BENCHMARK

>> Aggregator endpoint: https://aggregator.pre-release-preview.api.mithril.network/aggregator
>> Aggregator route: /proof/cardano-transaction
>> Transactions file: transactions-preview.txt
>> Transactions available: [100]
>> Transactions per request range: [0 10 20 30 40 50]
>> AB concurrency range: [50]
>> AB total requests per run: [500]
>> AB total runs: 6
>> Output file: benchmark.csv

>> [#1/6] Running stress test with 500 requests with 1 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (58.71 requests/s)

>> [#2/6] Running stress test with 500 requests with 10 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (49.19 requests/s)

>> [#3/6] Running stress test with 500 requests with 20 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (39.17 requests/s)

>> [#4/6] Running stress test with 500 requests with 30 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (34.11 requests/s)

>> [#5/6] Running stress test with 500 requests with 40 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (29.21 requests/s)

>> [#6/6] Running stress test with 500 requests with 50 transactions per request and 50 concurrency
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
>>>> Success (24.39 requests/s)

>> Benchmark completed:

total_requests,concurrency,transactions/request,requests/s
500,50,1,58.71
500,50,10,49.19
500,50,20,39.17
500,50,30,34.11
500,50,40,29.21
500,50,50,24.39