Skip to content

Commit

Permalink
Create a physical plan for block querying (#2586)
Browse files Browse the repository at this point in the history
* Create a physical plan for block querying

This PR will reduce the work necessary to query blocks, by planning out
which blocks are fetched from which ingester/store-gateway.

That way queries will potentially not need profile-by-profile
deduplication for blocks that had been deduplicated by the compactor.

* Remove unused field

* Missing `make generate` change

* Signal that there are no profiles streamed

* Generate block metadata correctly

* Improve tracing for block plan

* Request BlockMetadata correctly

Query head first (heads and flushing) and then queriers, this will make
sure that we do not miss a block moving into flushing.

* Forward hints correctly to store-gateway

* Sort iterator properly

* Ensure block metadata repsones are ts bound for ingesters

* Ignore storeQueryAfter for plan building

* Move the signalling to close connction to the right place

* Ensure we only read replica for the correct instance type

* Handle no result profiles

* Log full plan in debug level
  • Loading branch information
simonswine authored Nov 9, 2023
1 parent 11d8627 commit d4e3b03
Show file tree
Hide file tree
Showing 31 changed files with 3,797 additions and 603 deletions.
843 changes: 575 additions & 268 deletions api/gen/proto/go/ingester/v1/ingester.pb.go

Large diffs are not rendered by default.

1,010 changes: 911 additions & 99 deletions api/gen/proto/go/ingester/v1/ingester_vtproto.pb.go

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions api/gen/proto/go/ingester/v1/ingesterv1connect/ingester.connect.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

77 changes: 43 additions & 34 deletions api/gen/proto/go/storegateway/v1/storegateway.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions api/gen/proto/go/storegateway/v1/storegateway_vtproto.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit d4e3b03

Please sign in to comment.