Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CHIA-1562] move signature validation into run_block_generator() #18812

Merged
merged 3 commits into from
Nov 6, 2024

Conversation

arvidn
Copy link
Contributor

@arvidn arvidn commented Nov 4, 2024

This PR is best reviewed one commit at a time. However, the second commit contains the bulk of the changes.

Purpose:

By combining executing CLVM, parsing (and some validation of) conditions with signature validation, we do more work under a single GIL release, improving the concurrency of block validation.

This is arguably also a simplification of the code, making it a bit more robust. Instead of optionally validating signatures in two places, we now do it unconditionally in one place. (although, there is technically still the option to not validate the signatures, but we currently only use that in tests and in block creation when all we want to do is to compute the final cost of the block).

This is also a step twoards demoting get_name_puzzle_conditions() into a test-only function. It's used in many places where the signature is either not available or not set correctly. This PR replaces all production uses of this function.

Validating signatures in pre-validate

pre_validate_blocks_multiprocessing() no longer accepts the validate_signatures=False parameter. It's always validates signatures.

Since signature validation now happens in the pre-validation step, validate_block_body() no longer validates it. Which also means it doesn't need the BLSCache passed in to it.

This also means Blockchain.add_block() no longer needs to take the BLS cache. It validates the block body, but the signature validation has already happened in pre-validate.

Current Behavior:

The block signature validation is done in validate_block_body() or pre_validate_block(). In validate_block_body() we can optionally pass in the BLS cache, to save time on BLS pairings. We do this for new unfinished blocks, since they are likely to contain signatures that we've already validated in the mempool. The mempool adds pairings to the BLS cache.

Both of these functions can optionally not validate signatures. Sometimes we rely on this in tests, where we don't bother producing correct signatures, or want to save time. The validate_signature=False option is used when we receive a new FullBlock whose corresponding UnfinishedBlock we've already validated. We save time by not validating the signatures again.

New Behavior:

The block signature validation is done in run_block_generator() (and run_block_generator2(), post hard-fork). This happens along side of running the block generator and parsing the resulting conditions. An invalid signature generates a failure directly in this step.

This is always run as part of pre-validation, in thread pools.

run_block_generator() takes an optional BLS cache and signature to validate. We pass in the BLS cache only when validating UnfinishedBlocks. This is already done in main, in FullNode.add_unfinished_block(). What's new is that the resulting SpendBundleConditions now records that we've validated the conditions, and when adding the corresponding FullBlock, we just pick up the SpendBundleConditions from the unfinished block cache.

Testing Notes:

@arvidn arvidn added Changed Required label for PR that categorizes merge commit message as "Changed" for changelog Exclude_Notes Use this label if the changes in the PR should be excluded from the release notes labels Nov 4, 2024
@arvidn arvidn changed the title Always validate block signatures move signature validation into run_block_generator() Nov 4, 2024
@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Nov 4, 2024
Copy link
Contributor

github-actions bot commented Nov 4, 2024

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@arvidn arvidn force-pushed the always-validate-block-signatures branch from 6f35bd0 to 811781f Compare November 4, 2024 16:21
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Nov 4, 2024
Copy link
Contributor

github-actions bot commented Nov 4, 2024

Conflicts have been resolved. A maintainer will review the pull request shortly.

@arvidn arvidn force-pushed the always-validate-block-signatures branch from 811781f to 0835eb0 Compare November 4, 2024 16:29
Copy link
Contributor

github-actions bot commented Nov 4, 2024

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Nov 4, 2024
@arvidn arvidn force-pushed the always-validate-block-signatures branch from 0835eb0 to c50c7c7 Compare November 5, 2024 10:30
@github-actions github-actions bot removed the merge_conflict Branch has conflicts that prevent merge to main label Nov 5, 2024
Copy link
Contributor

github-actions bot commented Nov 5, 2024

Conflicts have been resolved. A maintainer will review the pull request shortly.

…final cost of the block we're creating. This lets us avoid validating the signature (to save time) and run in mempool mode
@arvidn arvidn force-pushed the always-validate-block-signatures branch from c50c7c7 to 592b5ea Compare November 5, 2024 14:32
@arvidn arvidn marked this pull request as ready for review November 5, 2024 14:41
@arvidn arvidn requested a review from a team as a code owner November 5, 2024 14:41
@arvidn arvidn changed the title move signature validation into run_block_generator() [CHIA-1562] move signature validation into run_block_generator() Nov 5, 2024
@arvidn
Copy link
Contributor Author

arvidn commented Nov 5, 2024

the missing test coverage is not new

@arvidn arvidn force-pushed the always-validate-block-signatures branch from 592b5ea to 69f7f68 Compare November 6, 2024 09:14
@arvidn arvidn force-pushed the always-validate-block-signatures branch from 69f7f68 to de73962 Compare November 6, 2024 09:23
@arvidn arvidn requested a review from emlowe November 6, 2024 15:44
Copy link
Contributor

github-actions bot commented Nov 6, 2024

File Coverage Missing Lines
chia/consensus/blockchain.py 60.0% lines 699, 720
chia/consensus/multiprocess_validation.py 92.0% lines 50, 147
Total Missing Coverage
94 lines 4 lines 95%

Copy link

Pull Request Test Coverage Report for Build 11700675112

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 90 of 94 (95.74%) changed or added relevant lines in 15 files are covered.
  • 18 unchanged lines in 8 files lost coverage.
  • Overall coverage decreased (-0.001%) to 90.903%

Changes Missing Coverage Covered Lines Changed/Added Lines %
chia/consensus/blockchain.py 3 5 60.0%
chia/consensus/multiprocess_validation.py 23 25 92.0%
Files with Coverage Reduction New Missed Lines %
chia/_tests/simulation/test_simulation.py 1 96.45%
chia/daemon/client.py 1 74.1%
chia/server/server.py 1 81.84%
chia/plotters/madmax.py 1 50.6%
chia/full_node/full_node.py 1 86.28%
chia/data_layer/data_layer.py 2 86.09%
chia/server/node_discovery.py 4 80.18%
chia/daemon/server.py 7 83.28%
Totals Coverage Status
Change from base Build 11691476343: -0.001%
Covered Lines: 102809
Relevant Lines: 112867

💛 - Coveralls

@pmaslana pmaslana merged commit 6ceaec4 into main Nov 6, 2024
362 of 363 checks passed
@pmaslana pmaslana deleted the always-validate-block-signatures branch November 6, 2024 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changed Required label for PR that categorizes merge commit message as "Changed" for changelog Exclude_Notes Use this label if the changes in the PR should be excluded from the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants