Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport PR #2634 to release/v1.7 for Add option to disable dns resolve #2641

Merged

Conversation

vdaas-ci
Copy link
Collaborator

@vdaas-ci vdaas-ci commented Sep 24, 2024

Description

When we verified the mirror gateway to connect to multiple locations, we could not connect when dns resolve was enabled. Therefore, I added an interface to disable it.

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.23.1
  • Rust Version: v1.81.0
  • Docker Version: v27.2.1
  • Kubernetes Version: v1.31.0
  • Helm Version: v3.16.0
  • NGT Version: v2.2.4
  • Faiss Version: v1.8.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Introduced a method to disable DNS resolution for specific addresses, enhancing connection management flexibility.
    • Improved error handling and fallback mechanisms in the gateway service for more reliable gRPC operations.
    • Adjusted connection logic to bypass DNS resolution for certain addresses, potentially improving performance.
    • Expanded mock testing functionality to control DNS resolution behavior for more precise testing scenarios.
  • Bug Fixes

    • Enhanced robustness of the gRPC call handling, allowing for alternative connections in case of failures.

* fix: add option to disable dns resolve

Signed-off-by: hlts2 <[email protected]>

* fix: add mock function

Signed-off-by: hlts2 <[email protected]>

* fix: unimplemented error

Signed-off-by: hlts2 <[email protected]>

* fix: change ForwardedContext method to private

Signed-off-by: hlts2 <[email protected]>

---------

Signed-off-by: hlts2 <[email protected]>
Copy link

cloudflare-workers-and-pages bot commented Sep 24, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: 232a2d1
Status: ✅  Deploy successful!
Preview URL: https://af81e067.vald.pages.dev
Branch Preview URL: https://backport-release-v1-7-fix-mi-dmyq.vald.pages.dev

View logs

Copy link
Contributor

coderabbitai bot commented Sep 24, 2024

Walkthrough

Walkthrough

The changes introduce a new method, SetDisableResolveDNSAddr, to the gRPCClient struct, allowing dynamic control over DNS resolution for specific addresses. This method updates the internal state management and modifies the Connect method to check a new sync.Map for DNS resolution flags. Additionally, mock structures in tests are updated to support this new functionality, and several methods in the gateway package are refactored to enhance context handling and error management.

Changes

File Path Change Summary
internal/net/grpc/client.go Added SetDisableResolveDNSAddr method to gRPCClient for managing DNS resolution flags. Updated Connect method to utilize this new functionality.
internal/test/mock/grpc/grpc_client_mock.go Modified GRPCClientMock to include SetDisableResolveDNSAddrFunc and added a new method SetDisableResolveDNSAddr.
internal/test/mock/grpc_testify_mock.go Added SetDisableResolveDNSAddr method to ClientInternal struct with empty implementation.
pkg/gateway/mirror/service/gateway.go Renamed ForwardedContext to forwardedContext (public to private) and updated related methods for context handling and error management in Do.
pkg/gateway/mirror/service/mirror.go Added call to SetDisableResolveDNSAddr in Connect method to disable DNS resolution for specific addresses before connection attempts.
pkg/gateway/mirror/service/mirror_test.go Introduced SetDisableResolveDNSAddrFunc to mock structures in test cases, maintaining existing logic but expanding mock interface.

Possibly related PRs

  • Add option to disable dns resolve #2634: The changes in this PR directly relate to the addition of the SetDisableResolveDNSAddr method in the gRPCClient struct, which is also introduced in the main PR, enhancing DNS resolution control.

Suggested labels

priority/low, size/XXXL, area/internal, area/gateway/mirror, area/filter/ingress, area/filter/egress, actions/backport/release/v1.7

Suggested reviewers

  • datelier
  • kpango

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    -- I pushed a fix in commit <commit_id>, please review it.
    -- Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    -- @coderabbitai generate unit testing code for this file.
    -- @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    -- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    -- @coderabbitai read src/utils.ts and generate unit testing code.
    -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@vdaas-ci
Copy link
Collaborator Author

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range and nitpick comments (5)
internal/test/mock/grpc/grpc_client_mock.go (2)

37-37: LGTM! Consider adding a comment for clarity.

The addition of SetDisableResolveDNSAddrFunc is consistent with the existing structure and naming conventions.

Consider adding a brief comment explaining the purpose of this function, similar to other function pointers in the struct. For example:

// SetDisableResolveDNSAddrFunc is used to mock the behavior of setting DNS resolution disable flag for a given address.
SetDisableResolveDNSAddrFunc func(addr string, disabled bool)

75-78: LGTM! Consider adding a nil check for robustness.

The implementation of SetDisableResolveDNSAddr is consistent with other methods in the struct and correctly delegates to the function pointer.

For improved robustness, consider adding a nil check before calling the function pointer. This prevents potential panics if the mock is not fully initialized. For example:

func (gc *GRPCClientMock) SetDisableResolveDNSAddr(addr string, disabled bool) {
    if gc.SetDisableResolveDNSAddrFunc != nil {
        gc.SetDisableResolveDNSAddrFunc(addr, disabled)
    }
}
internal/test/mock/grpc_testify_mock.go (1)

216-217: LGTM. Consider adding documentation for clarity.

The addition of the SetDisableResolveDNSAddr method to the ClientInternal struct is consistent with the PR objectives. The empty implementation is appropriate for a mock object, allowing test cases to override the behavior as needed.

Consider adding a comment to explain the purpose of this method and the distributed parameter, which would improve clarity for developers working with this mock in the future. For example:

// SetDisableResolveDNSAddr mocks the behavior of disabling DNS resolution for a specific address.
// The 'distributed' parameter indicates whether the setting applies to a distributed environment (e.g., mirror gateway).
func (c *ClientInternal) SetDisableResolveDNSAddr(addr string, distributed bool) {}
pkg/gateway/mirror/service/mirror.go (1)

322-322: LGTM! Consider adding a comment for clarity.

The addition of SetDisableResolveDNSAddr aligns well with the PR objective to introduce an option for disabling DNS resolution. This change ensures that DNS resolution is disabled for specific addresses before attempting to connect, which should resolve the connection issues mentioned in the PR description.

Consider adding a brief comment explaining why DNS resolution is being disabled for these connections. This would improve code readability and maintainability. For example:

+// Disable DNS resolution for this address to avoid connection issues
m.gateway.GRPCClient().SetDisableResolveDNSAddr(addr, true)
internal/net/grpc/client.go (1)

951-955: Improve code comments for clarity and professionalism

The comments within the SetDisableResolveDNSAddr method contain personal notes and informal language. To enhance maintainability and readability, consider updating the comments to clearly explain the purpose and usage of the method.

Apply this diff to refine the comments:

-  // NOTE: When connecting to multiple locations, it was necessary to switch dynamically, so implementation was added.
-  // There is no setting for disable on the helm chart side, so I used this implementation.
+  // Allows dynamic control over DNS resolution for specific addresses.
+  // Useful when connecting to multiple locations where DNS resolution needs to be toggled.
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9fd2ea6 and 232a2d1.

Files ignored due to path filters (2)
  • apis/grpc/v1/payload/payload.pb.go is excluded by !**/*.pb.go, !**/*.pb.go
  • apis/grpc/v1/rpc/errdetails/error_details.pb.go is excluded by !**/*.pb.go, !**/*.pb.go
Files selected for processing (6)
  • internal/net/grpc/client.go (3 hunks)
  • internal/test/mock/grpc/grpc_client_mock.go (2 hunks)
  • internal/test/mock/grpc_testify_mock.go (1 hunks)
  • pkg/gateway/mirror/service/gateway.go (5 hunks)
  • pkg/gateway/mirror/service/mirror.go (1 hunks)
  • pkg/gateway/mirror/service/mirror_test.go (6 hunks)
Additional comments not posted (12)
internal/test/mock/grpc/grpc_client_mock.go (1)

Line range hint 1-78: Overall, the changes look good and align with the PR objectives.

The additions to the GRPCClientMock struct successfully introduce the functionality to disable DNS resolution for specific addresses. This aligns with the PR objective of adding an option to disable DNS resolution when connecting to multiple locations via the mirror gateway.

The changes are minimal, focused, and well-integrated with the existing code structure. They provide the necessary mock implementations to support testing of the new functionality in the main codebase.

pkg/gateway/mirror/service/mirror_test.go (5)

90-90: LGTM: New mock method added correctly.

The addition of SetDisableResolveDNSAddrFunc to the grpcmock.GRPCClientMock struct is correct and aligns with the PR objective. The empty implementation is appropriate for a mock object.


122-122: LGTM: Consistent implementation of new mock method.

The addition of SetDisableResolveDNSAddrFunc is consistent with the previous test case. The empty implementation remains appropriate for a mock object.


226-226: LGTM: Consistent implementation across test cases.

The addition of SetDisableResolveDNSAddrFunc in the Test_mirr_Disconnect function maintains consistency with previous test cases. The empty implementation and method signature remain appropriate and consistent.


258-258: LGTM: Consistent implementation maintained throughout.

The addition of SetDisableResolveDNSAddrFunc in this test case of the Test_mirr_Disconnect function maintains perfect consistency with all previous instances. The empty implementation and method signature remain appropriate and consistent throughout the file.


90-90: Summary: Consistent and appropriate implementation of new mock method.

The new method SetDisableResolveDNSAddrFunc has been consistently added to all relevant test cases in the mirror_test.go file. The implementation (empty function) and signature (two parameters: addr string and disabled bool) are uniform across all instances. These changes appropriately support the PR objective of introducing an option to disable DNS resolution without altering existing test logic.

Great job on maintaining consistency and adhering to the project's testing standards!

Also applies to: 122-122, 226-226, 258-258

pkg/gateway/mirror/service/gateway.go (4)

152-160: Verify error handling logic in 'Do' method

In the Do method, when an error occurs, the code attempts a fallback using RoundRobin. The condition strings.EqualFold(conn.Target(), target) skips over connections matching the original target. This might lead to unintended behavior if multiple connections have the same target with different cases or formats. Please verify if this logic is intended or if a more precise comparison is needed.


Line range hint 116-121: Consistent use of the new 'forwardedContext' method

The implementation of g.forwardedContext(ctx, g.podName) in the BroadCast method ensures that the forwarding context is consistently applied across all requests. This enhances encapsulation and maintains code clarity.


Line range hint 181-188: Updated 'DoMulti' method to use 'forwardedContext'

The DoMulti method now utilizes g.forwardedContext(ctx, g.podName), aligning it with other methods for consistent context handling. This promotes code consistency and reduces the risk of context-related issues.


76-78: ⚠️ Potential issue

Changing 'ForwardedContext' to private may introduce breaking changes

The method ForwardedContext has been renamed to forwardedContext, altering its visibility from public to private. If this method is used outside of this package, this change could cause breaking issues for external packages or tests relying on it. Please verify if ForwardedContext is being used externally.

Run the following script to identify external usages of ForwardedContext:

Verification successful

External usages of ForwardedContext not found

The ForwardedContext method is only utilized within internal test and mock files. There are no external package dependencies relying on it. Therefore, changing its visibility from public to private will not introduce breaking changes for external consumers.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Search for external usages of 'ForwardedContext' method.

# Test: Search the codebase for 'ForwardedContext' method calls outside 'gateway.go'.
# Expectation: No external usages found.

rg --type go 'ForwardedContext\(' | grep -v 'pkg/gateway/mirror/service/gateway.go'

Length of output: 1101

internal/net/grpc/client.go (2)

986-992: Ensure correct logic in DNS resolution decision

The logic within the WithResolveDNS closure correctly checks the disableResolveDNSAddrs map to determine if DNS resolution should be disabled for the given address. This allows for dynamic control based on specific addresses.


104-104: Verify the use of sync.Map with generics

The disableResolveDNSAddrs field is declared as sync.Map[string, bool]. Ensure that the custom sync.Map implementation supports generics and that this usage is compatible with the intended Go version and your internal packages.

If using Go's standard library sync.Map, which doesn't support generics, consider using a type-safe wrapper or alternative concurrency-safe data structures.

internal/net/grpc/client.go Show resolved Hide resolved
Copy link

codecov bot commented Sep 24, 2024

Codecov Report

Attention: Patch coverage is 13.63636% with 19 lines in your changes missing coverage. Please review.

Please upload report for BASE (release/v1.7@9fd2ea6). Learn more about missing BASE report.

Files with missing lines Patch % Lines
pkg/gateway/mirror/service/gateway.go 0.00% 11 Missing ⚠️
internal/net/grpc/client.go 0.00% 7 Missing ⚠️
internal/test/mock/grpc_testify_mock.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff               @@
##             release/v1.7    #2641   +/-   ##
===============================================
  Coverage                ?   24.35%           
===============================================
  Files                   ?      536           
  Lines                   ?    46459           
  Branches                ?        0           
===============================================
  Hits                    ?    11317           
  Misses                  ?    34375           
  Partials                ?      767           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@hlts2 hlts2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants