Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

l2 commitments: fix rayon collect iter order #35

Merged
merged 2 commits into from
Apr 4, 2024

Conversation

cchudant
Copy link
Member

@cchudant cchudant commented Apr 3, 2024

Fix state root computation!

i thought I understood rayon but, the problem here was that, under the hood

  • par_bridge wraps the underlying iter in a lock and is not indexedparalleliterator. workers take new items as soon as they can in an unspecified way
  • collect is specialized(ish) when collecting to a vec: it will return all the elements in order in the final vec, and every worker has its dedicated range of the destination vector it writes to -- this is what I thought was happening
  • instead, unindexed parallel iterators don't benefit from this specialized collect
  • unindexed parallel iterators always collect to a LinkedList<Vec<>> behind the scene, every worker write to its own Vec it pushes results to, and they are merged at the end into the final Vec that collect returns
    This means that, when using par_bridge + collect to a vec, the order of the items in the final vec in unspecified - and the final hash of the state root is never the same :(

This is easy to fix though: we just keep track of the keys along with the values when using par_bridge

Pull Request type

  • Bugfix

What is the current behavior?

State root dono work

Resolves: #NA

What is the new behavior?

State root work

Does this introduce a breaking change?

N/A

Other information

To be honest, after this PR i'll test deoxys with make my parallelized bonsai-trie; there is a good chance that makes these parallel iterators kinda redundant as a single bonsai-trie commit would be already parallel enough

Also, the Debug impl for Felt was driving me insane so I changed it to make reading lots of logs easier

@antiyro
Copy link
Member

antiyro commented Apr 3, 2024

Could you change the bonsai-lib to:

bonsai-trie = { default-features = false, git = "https://github.com/keep-starknet-strange/bonsai-trie.git", branch = "oss" }

@antiyro antiyro requested a review from Tbelleng April 4, 2024 07:52
Copy link
Contributor

@Tbelleng Tbelleng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me, lfg 🚀

@Tbelleng Tbelleng merged commit 31dfb78 into madara-alliance:main Apr 4, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants