Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Range Queries for Value on rpc SearchVariable #1116

Open
coltmcnealy-lh opened this issue Nov 9, 2024 · 0 comments
Open

Support Range Queries for Value on rpc SearchVariable #1116

coltmcnealy-lh opened this issue Nov 9, 2024 · 0 comments
Labels
feature Issue that denotes a new feature, request, or performance improvement. server Affects the core LH Server code.

Comments

@coltmcnealy-lh
Copy link
Member

It is possible to support range queries for variables in rpc SearchVariable. This would allow useful use-cases, such as:

  • Prefix search on strings.
  • Numerical ranges, such as "find all orders with a total value between $50 and $100"

The current protobuf for rpc SearchVariable is here:

message SearchVariableRequest {
  // Bookmark for cursor-based pagination; pass if applicable.
  optional bytes bookmark = 1;
  // Maximum results to return in one request.
  optional int32 limit = 2;

  // Specifies the value that the variable must be. Exact match is required.
  VariableValue value = 3;

  // Specifies major version of the WfSpec for the associated WfRun.
  optional int32 wf_spec_major_version = 4;
  // Specifies the revision of the WfSpec for the associated WfRun.
  optional int32 wf_spec_revision = 5;

  // Specifies the name of the variable to search for. This is required.
  string var_name = 6;

  // Specifies the name of the WfSpec for the associated WfRun's. This is required.
  string wf_spec_name = 7;
}

I would think it makes the most sense to make the 3 field a oneof:

message VariableRangeQuery {
  VariableValue start = 1;
  VariableValue end = 2;
}

oneof variable_search_criteria {
  VariableValue value = 3;
  VariableRangeQuery = 8;
}

This would be protobuf-compatible with no breaking changes that violate our policy (we do introduce a oneof here, which means when upgrading the golang client you will need a tiny refactor, but old clients continue to work with the new server).

Implementation

When we add support for time filters on rpc SearchVariable, we can support that by using the Variable Range as the search delimiter (just a portion of the tag's encoded key) and manually skipping over entries outside of the time range.

We could optimize it by jumping to the next character in Ascii once we go outside of the time range; however, that would involve opening and closing multiple range scans, so it may or may not be faster depending on the exact results and query.

Gotchas

In ticket #793 we are working on encoding Tag key/value pairs in preparation for 1.0. In that ticket we should include

@coltmcnealy-lh coltmcnealy-lh added feature Issue that denotes a new feature, request, or performance improvement. server Affects the core LH Server code. labels Nov 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Issue that denotes a new feature, request, or performance improvement. server Affects the core LH Server code.
Projects
None yet
Development

No branches or pull requests

1 participant