Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(testing): Task Sender / Stress Tester #1189

Open
wants to merge 62 commits into
base: 1314-fix-sender-receive-batchinclusiondata-or-createnewtaskerror-in-validityresponsemessage
Choose a base branch
from

Conversation

PatStiles
Copy link
Contributor

@PatStiles PatStiles commented Oct 7, 2024

Note

This PR is pointed to 1314 branch because it needs the fix included in said branch.
PR of that branch is here

Changes

Adds a cli program to:

  • Generate, fund wallets, and deposit to aligned payment service
  • Send proofs with a burst of n infinitely from the generated wallets.
  • Open n connection and hang infinitely.

Testing

devnet

  • run devnet, with a new function that start anvil with many prefunded accounts:
make anvil_start_with_block_time_with_more_prefunded
  • Run Batcher
make start_batcher_local
  • Use the prefunded accounts to fund the BatcherPaymentService
    Note:
    This will fund from every private key stored in the file batcher/aligned-task-sender/wallets/devnet
    You can take any amount of wallets you want from the file batcher/aligned-task-sender/wallets/devnet-backup
make task_sender_fund_wallets_devnet
  • Generate groth16 proofs. These will be taken at random when sending them later on, so merkle_roots are not easily collided.
    This will create a proofs folder in scripts/test_files/task_sender/proofs by default.
NUMBER_PROOFS=15 make task_sender_generate_groth16_proofs 
  • Send infinite proofs
    Before running this command, you need to make sure you have first generated wallets and proofs. This command reads the default paths mentioned before.
    • BURST_SIZE is how many proofs will each sender send.
    • BURST_TIME_SECS is how much time will each sender wait between each send_burst.
BURST_SIZE=2 BURST_TIME_SECS=1000 make task_sender_infinite_proofs_devnet
  • Test connections
    This will start NUM_SENDERS connections and hang them infinitely without sending any proof
NUM_SENDERS=123 make task_sender_test_connections_devnet

Testnet

  • Generate and fund wallets
    This will generate wallets on holesky, send them AMOUNT_TO_DEPOSIT funds from FUNDING_WALLET_PRIVATE_KEY, then send AMOUNT_TO_DEPOSIT_TO_ALIGNED funds from the newly created wallet to the BatcherPaymentService, to allow submittion of proofs from this wallet.
FUNDING_WALLET_PRIVATE_KEY=xxx NUM_WALLETS=10 AMOUNT_TO_DEPOSIT=0.01 AMOUNT_TO_DEPOSIT_TO_ALIGNED=0.0099 make task_sender_generate_and_fund_wallets_holesky_stage
  • Generate groth16 proofs. These will be taken at random when sending them later on, so merkle_roots are not easily collided.
    This will create a proofs folder in scripts/test_files/task_sender/proofs by default.
NUMBER_PROOFS=15 make task_sender_generate_groth16_proofs 
  • Send infinite proofs
    Before running this command, you need to make sure you have first generated wallets and proofs. This command reads the default paths mentioned before.
    • BURST_SIZE is how many proofs will each sender send.
    • BURST_TIME_SECS is how much time will each sender wait between each send_burst.
BURST_SIZE=2 BURST_TIME_SECS=1000 make task_sender_infinite_proofs_holesky_stage
  • Test connections
    This will start NUM_SENDERS connections and hang them infinitely without sending any proof
NUM_SENDERS=123 make task_sender_test_connections_holesky_stage

Closes #1113 and Closes #1015. Reopens #1162 due to merge conflicts.

Copy link
Collaborator

@MarcosNicolau MarcosNicolau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works. But it doesn't support many open connections. I think the idea is to open millions of concurrent requests. For that, we just need to open the socket connections, it is not necessary to send proofs which might be the bottleneck.

@MarcosNicolau MarcosNicolau linked an issue Oct 9, 2024 that may be closed by this pull request
@MarcosNicolau MarcosNicolau self-assigned this Oct 10, 2024
MarcosNicolau and others added 2 commits October 23, 2024 14:54
…lusiondata-or-createnewtaskerror-in-validityresponsemessage' into 1015-task-sender
@uri-99 uri-99 self-assigned this Oct 24, 2024
@uri-99 uri-99 marked this pull request as draft October 24, 2024 15:13
@uri-99 uri-99 changed the base branch from staging to 1314-fix-sender-receive-batchinclusiondata-or-createnewtaskerror-in-validityresponsemessage October 25, 2024 04:13
@uri-99 uri-99 marked this pull request as ready for review November 5, 2024 20:50
Copy link
Contributor

@MauroToscano MauroToscano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait for SDK

@uri-99
Copy link
Contributor

uri-99 commented Nov 7, 2024

Also missing build sender in CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Write a stress test for concurrent connections
4 participants