Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: cosmovisor batch upgrades #21790

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Conversation

psiphi5
Copy link
Contributor

@psiphi5 psiphi5 commented Sep 18, 2024

Description

Closes: #20630

This PR adds a add-batch-upgrade to cosmovisor that allows to add multiple named upgrades at specific heights.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Summary by CodeRabbit

  • New Features

    • Introduced the add-batch-upgrade command for efficient batch upgrades.
    • Added functionality to pass stdin to enhance input handling.
    • Implemented a command for managing multiple upgrades in a single operation.
  • Improvements

    • Enhanced upgrade management capabilities and usability with new commands and configurations.
    • Improved configuration flexibility by allowing specification of the CometBFT RPC endpoint via an environment variable.
  • Dependency Updates

    • Updated module dependencies for improved functionality and performance.

Copy link
Contributor

coderabbitai bot commented Sep 18, 2024

Walkthrough

Walkthrough

The pull request introduces enhancements to the tools/cosmovisor module, including the addition of a new command for batch upgrades, improvements to existing upgrade functionalities, and updates to dependency management. Key changes include the introduction of the add-batch-upgrade command, modifications to the AddUpgrade function for better parameter handling, and the implementation of features for managing multiple upgrades efficiently. The go.mod file has also been updated to reflect new direct dependencies.

Changes

Files Change Summary
tools/cosmovisor/CHANGELOG.md Updated to include a new feature (add-batch-upgrade) and an improvement for passing stdin to the binary.
tools/cosmovisor/args.go Added EnvCometBftRpcEndpoint constant and CometBftRpcEndpoint field in Config struct; updated GetConfigFromEnv and DetailString methods to include new endpoint.
tools/cosmovisor/cmd/cosmovisor/add_upgrade.go Refactored AddUpgrade function for direct parameter acceptance; added getConfigFromCmd and AddUpgradeCmd functions for improved command handling.
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Introduced add-batch-upgrade command for managing multiple upgrades, validating input, and aggregating upgrade information into a single JSON output file.
tools/cosmovisor/process.go Added loadBatchUpgradeFile for reading and sorting batch upgrade plans and BatchUpgradeWatcher for monitoring blockchain state and managing upgrades dynamically.
tools/cosmovisor/go.mod Updated to include new direct dependencies: github.com/cometbft/cometbft v0.38.9 and github.com/fsnotify/fsnotify v1.7.0, removing previous indirect declarations.

Assessment against linked issues

Objective Addressed Explanation
Support multiple upgrades in upgrade-info.json (Issue #20630)
Change upgrade-info.json structure to an array of objects (Issue #20630)
Allow staging multiple upgrades without manual intervention (Issue #20630)

Possibly related PRs

Suggested labels

C:x/upgrade, C:server/v2, C:server/v2 cometbft, C:CLI, C:x/protocolpool, backport/v0.52.x


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    -- I pushed a fix in commit <commit_id>, please review it.
    -- Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    -- @coderabbitai generate unit testing code for this file.
    -- @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    -- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    -- @coderabbitai read src/utils.ts and generate unit testing code.
    -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the C:Cosmovisor Issues and PR related to Cosmovisor label Sep 18, 2024
@julienrbrt julienrbrt self-assigned this Sep 18, 2024
@psiphi5 psiphi5 marked this pull request as ready for review September 19, 2024 06:23
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Outside diff range and nitpick comments (7)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (3)

16-16: Correct grammatical error in command's short description

The Short description should be "Add APP upgrade binaries to cosmovisor" instead of "Add APP upgrades binary to cosmovisor".

Apply this diff to fix the grammatical error:

- Short:        "Add APP upgrades binary to cosmovisor",
+ Short:        "Add APP upgrade binaries to cosmovisor",

23-24: Use GoDoc style comments for exported functions

The exported function AddBatchUpgrade should have a comment that starts with the function name to comply with GoDoc conventions.

Apply this diff to update the comment:

- // AddBatchUpgrade takes in multiple specified upgrades and creates a single
- // batch upgrade file out of them
+ // AddBatchUpgrade takes in multiple specified upgrades and creates a single
+ // batch upgrade file out of them.

Ensure the comment starts with the function name and ends with a period.


13-13: Add GoDoc comment for exported function NewBatchAddUpgradeCmd

The exported function NewBatchAddUpgradeCmd lacks a GoDoc comment. Adding a comment improves documentation and code readability.

Apply this diff to add the comment:

+ // NewBatchAddUpgradeCmd creates a new Cobra command for batch adding upgrades.
  func NewBatchAddUpgradeCmd() *cobra.Command {
tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (2)

Line range hint 68-70: Avoid using panic; return an error instead

Using panic for expected errors is discouraged as it can cause the application to crash unexpectedly. Instead, return the error to allow it to be handled gracefully by the caller. Also, consider providing a more descriptive error message.

Apply this diff to fix the issue:

 if err := plan.ValidateBasic(); err != nil {
-    panic(fmt.Errorf("something is wrong with cosmovisor: %w", err))
+    return fmt.Errorf("invalid upgrade plan: %w", err)
 }

88-123: Unexport functions that are not intended to be public

The functions GetConfig and AddUpgradeCmd are currently exported due to their capitalized names. If these functions are only used within this package, consider renaming them to getConfig and addUpgradeCmd to reflect that they are unexported, following Go conventions and the Uber Go Style Guide.

Apply this diff to make the changes:

 // GetConfig returns a Config using passed-in flag
-func GetConfig(cmd *cobra.Command) (*cosmovisor.Config, error) {
+func getConfig(cmd *cobra.Command) (*cosmovisor.Config, error) {
     // ...

 // AddUpgradeCmd parses input flags and adds upgrade info to manifest
-func AddUpgradeCmd(cmd *cobra.Command, args []string) error {
+func addUpgradeCmd(cmd *cobra.Command, args []string) error {
     // ...

 func NewAddUpgradeCmd() *cobra.Command {
     addUpgrade := &cobra.Command{
         // ...
-        RunE:         AddUpgradeCmd,
+        RunE:         addUpgradeCmd,
     }
     // ...
tools/cosmovisor/process.go (1)

47-62: Use descriptive variable names for better readability

The variable uInfos is not descriptive. Consider renaming it to upgradePlans or plans to improve readability and adhere to the style guide recommendations on naming.

Apply this diff to rename uInfos to upgradePlans:

-var uInfos []upgradetypes.Plan
+var upgradePlans []upgradetypes.Plan

And update all occurrences of uInfos within the function accordingly.

tools/cosmovisor/args.go (1)

112-112: Adjust the function comment for GoDoc style compliance

Per GoDoc conventions, the comment should start with the function name and be a complete sentence that describes its behavior.

Update the comment as follows:

-// UpgradeInfoBatchFilePath is the same as UpgradeInfoFilePath but with a batch suffix.
+// UpgradeInfoBatchFilePath returns the same path as UpgradeInfoFilePath but with a ".batch" suffix.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 81ec7ea and 3fe1cff.

Files selected for processing (7)
  • tools/cosmovisor/CHANGELOG.md (1 hunks)
  • tools/cosmovisor/args.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (4 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/root.go (1 hunks)
  • tools/cosmovisor/go.mod (1 hunks)
  • tools/cosmovisor/process.go (4 hunks)
Additional context used
Path-based instructions (6)
tools/cosmovisor/cmd/cosmovisor/root.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/CHANGELOG.md (1)

Pattern **/*.md: "Assess the documentation for misspellings, grammatical errors, missing documentation and correctness"

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/args.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Markdownlint
tools/cosmovisor/CHANGELOG.md

39-39: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines

(MD022, blanks-around-headings)


40-40: null
Lists should be surrounded by blank lines

(MD032, blanks-around-lists)

GitHub Check: CodeQL
tools/cosmovisor/process.go

[notice] 201-201: Spawning a Go routine
Spawning a Go routine may be a possible source of non-determinism

Additional comments not posted (3)
tools/cosmovisor/cmd/cosmovisor/root.go (1)

22-22: Addition of NewBatchAddUpgradeCmd() follows conventions and is well-integrated.

The new command NewBatchAddUpgradeCmd() is consistent with the existing command structure. It adheres to the naming conventions outlined in the Uber Golang style guide, using the New prefix for constructor functions that return a command.

tools/cosmovisor/go.mod (1)

8-9: Verify the necessity of adding dependencies as direct requirements

Please confirm that github.com/cometbft/cometbft v0.38.9 and github.com/fsnotify/fsnotify v1.7.0 are directly imported in the cosmovisor codebase. If these packages are only used indirectly through other dependencies, they may not need to be listed as direct requirements in the go.mod file. Keeping only necessary direct dependencies helps maintain a cleaner dependency graph and reduces potential conflicts.

Run the following script to check for direct imports of these packages:

tools/cosmovisor/args.go (1)

113-115: LGTM!

The new method correctly adds the batch upgrade info file path functionality and follows existing code patterns.

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
tools/cosmovisor/process.go (1)

Line range hint 1-600: Minor deviations from Uber Go Style Guide.

The code follows the Uber Go Style Guide for the most part. There are a few minor deviations:

  • Some lines are longer than 120 characters. Consider breaking them into multiple lines for better readability.
  • Some error messages don't start with lowercase. Error messages should start with lowercase for consistency.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3fe1cff and 7e1bf65.

Files selected for processing (4)
  • tools/cosmovisor/CHANGELOG.md (1 hunks)
  • tools/cosmovisor/args.go (5 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/process.go (3 hunks)
Files skipped from review as they are similar to previous changes (2)
  • tools/cosmovisor/CHANGELOG.md
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go
Additional context used
Path-based instructions (2)
tools/cosmovisor/args.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Additional comments not posted (8)
tools/cosmovisor/process.go (4)

47-63: LGTM!

The loadBatchUpgradeFile function correctly loads and sorts the batch upgrade plans. The error handling looks good.


68-177: LGTM!

The BatchWatcher function correctly implements the batch upgrade process. It handles new blocks, updates the upgrade-info.json file at the appropriate heights, and reloads the upgrade plans if the batch file is modified. The error handling and logging look good.


201-202: LGTM!

The changes to the Run function to start the BatchWatcher in a separate goroutine look good. Using a context to manage the lifecycle of the BatchWatcher is a nice touch.

Also applies to: 214-215


207-207: LGTM!

Calling cancel() when a signal is received is the correct way to gracefully exit the BatchWatcher.

tools/cosmovisor/args.go (4)

41-41: LGTM!

The new constant EnvCometBftRpcEndpoint is defined correctly and follows the existing naming convention.


72-72: Looks good!

The new CometBftRpcEndpoint field is added to the Config struct correctly, with appropriate tags and a reasonable default value.


114-117: LGTM!

The new UpgradeInfoBatchFilePath function is implemented correctly and provides a clear way to derive the batch upgrade info file path.


217-221: Looks good!

The GetConfigFromEnv function and DetailString method are updated appropriately to handle the new CometBftRpcEndpoint field.

Also applies to: 559-559

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
tools/cosmovisor/process.go (1)

68-177: The BatchUpgradeWatcher function logic is correct, but consider refactoring for better readability and maintainability.

The BatchUpgradeWatcher function is implemented correctly and handles the core logic of watching for upgrades and replacing the upgrade-info files. However, the function is quite long and could be refactored into smaller functions for better readability and maintainability. For example:

  • Extract the logic for replacing the upgrade-info and batch files into a separate function.
  • Consider propagating errors up to the caller instead of logging and returning.
  • Add a way to gracefully stop the watcher.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2fb3b68 and 66c7be2.

Files selected for processing (4)
  • tools/cosmovisor/args_test.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (4 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/process.go (3 hunks)
Files skipped from review as they are similar to previous changes (1)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go
Additional context used
Path-based instructions (3)
tools/cosmovisor/args_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Additional comments not posted (11)
tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (6)

22-22: LGTM!

Setting the RunE field to AddUpgradeCmd is the correct way to specify the handler function for the add-upgrade command.


31-32: Refactoring the function signature is a good improvement!

Accepting parameters directly instead of relying on command-line arguments enhances the modularity and reusability of the addUpgrade function. This change aligns with best practices for writing clean and maintainable code.


36-36: LGTM!

Converting the upgrade name to lowercase, when not disabled by configuration, is a good practice to ensure consistency in naming conventions across the codebase.


Line range hint 66-82: The code segment handles upgrades at a specific height correctly.

The logic for creating an upgrade plan, validating it, and saving it to the upgrade-info.json file is implemented correctly. The code segment aligns with the best practices mentioned in the past review comment about setting appropriate file permissions for non-executable files.


88-100: Extracting the configuration retrieval logic is a good practice.

Moving the configuration retrieval logic into the getConfigFromCmd function improves code organization and reusability. The function handles errors appropriately and returns them to the caller, which aligns with best practices for error handling in Go.


102-122: The AddUpgradeCmd function is implemented correctly.

The function follows the expected logic for a command handler. It parses the input flags, retrieves the configuration, and calls the refactored addUpgrade function with the appropriate parameters. The error handling is done appropriately, and the code segment aligns with the changes made to the addUpgrade function signature.

tools/cosmovisor/process.go (3)

47-63: LGTM!

The loadBatchUpgradeFile function is implemented correctly:

  • It handles the file not found error gracefully.
  • It uses the Config for the file path to allow configurability.
  • Error messages include the file path for easier debugging.
  • It sorts the slice of upgrade plans by height to ensure correct order of processing.

95-101: Good use of polling to wait for the chain process to be ready.

Polling the RPC endpoint to wait for the chain process to be ready is a good way to ensure the watcher starts at the correct time and doesn't miss any blocks.


214-214: Starting the BatchUpgradeWatcher in a separate goroutine is a good way to ensure it runs concurrently with the main process.

The changes to the Run function to start the BatchUpgradeWatcher in a separate goroutine are implemented correctly:

  • It allows the watcher to run concurrently with the main process.
  • It passes a context to the watcher, which allows it to be cancelled when the Run function exits.
  • It starts the watcher before waiting for an upgrade, which ensures it doesn't miss any upgrades.
tools/cosmovisor/args_test.go (2)

480-480: Approve the addition of the CometBftRpcEndpoint parameter.

The addition of the CometBftRpcEndpoint parameter to the Config struct enhances the configurability of the application and aligns with the goal of improving integration with CometBFT.


480-480: Verify the handling of the CometBftRpcEndpoint parameter.

Please ensure that the CometBftRpcEndpoint parameter is properly validated and handled throughout the codebase to maintain the integrity of the application.

Verification successful

CometBftRpcEndpoint parameter is handled consistently, but could benefit from additional validation

The CometBftRpcEndpoint parameter is properly used throughout the codebase with a default value and environment variable support. However, consider implementing these improvements:

  • Add explicit URL format validation when setting the endpoint.
  • Enhance error handling when using the endpoint in HTTP requests and client creation.
  • Include documentation about security considerations for this parameter.

These suggestions aim to improve robustness and security, but the current implementation is functional.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify if the `CometBftRpcEndpoint` parameter is properly validated and handled.

# Test: Search for occurrences of `CometBftRpcEndpoint`. 
# Expect: Proper validation and handling of the parameter.
rg --type go -A 5 $'CometBftRpcEndpoint'

Length of output: 2643

if err := cmd.Process.Signal(sig); err != nil {
l.logger.Error("terminated", "error", err, "bin", bin)
os.Exit(1)
}
}()

go BatchUpgradeWatcher(ctx, l.cfg, l.logger)

Check notice

Code scanning / CodeQL

Spawning a Go routine Note

Spawning a Go routine may be a possible source of non-determinism
@psiphi5
Copy link
Contributor Author

psiphi5 commented Sep 20, 2024

Hi @julienrbrt I've finished the implementation of this feature, would love a code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:Cosmovisor Issues and PR related to Cosmovisor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Have multiple upgrades in upgrade-info.json and support it in Cosmovisor
2 participants