Several problems encountered during latency testing

[JiamingSuper]

I have set up a local node for testing, following the deployment instructions in the official "running_local.md" document. During the testing process, I encountered several issues (my English is not very good, so I'm using translation software to help, I hope it can convey the meaning clearly).

1. I am using go-ethereum to send transactions and recording the time of sending, indicated as "time1" in the graph. Additionally, I am listening for new blocks on L2 and considering the block time of the block containing the transaction as "time2". I noticed that "time2" is a few milliseconds earlier than "time1". I want to understand why this is happening.

2. I would like to calculate the latency on L2. Is it reasonable to use "time2 - time1"? Or should I use "time3 - time1"? (where "time3" refers to the time when the sequencer submits user transactions as a batch to the L1 contract)

3. When I send many transactions, even though I have ensured that the transactions are sent in sequence according to the designated nonce, I still encounter errors due to "nonce too high", and these errors become more frequent as the number of transactions increases. Is this primarily caused by network latency?

[bap2pecs]

we also meet with the same issue doing load testing.

per https://github.com/0xPolygon/cdk-validium-node/blob/develop/pool/pool_test.go#L1727, it seems it's due to the account's queue limit is hit.

but now the chain is stuck and cannot send tx from any accounts.

any idea how to fix?

Edit: I submitted a separate issue at 0xPolygon#68 with details analysis