Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregation Extension #684

Merged
merged 23 commits into from
Jun 11, 2024
Merged

Aggregation Extension #684

merged 23 commits into from
Jun 11, 2024

Conversation

jamesfisher-geo
Copy link
Contributor

@jamesfisher-geo jamesfisher-geo commented May 6, 2024

Related Issue(s):

Description:

Adds base support for the STAC API Aggregation Extension

PR Checklist:

  • [ x ] pre-commit hooks pass locally
  • [ x ] Tests pass (run make test)
  • Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
  • Changes are added to the CHANGELOG.

@drnextgis
Copy link
Contributor

drnextgis commented May 6, 2024

That's great to see movement in that direction! I have some questions/comments.

  1. AggregationExtensionPostRequest is defined but not used.
  2. Is it meant to be used with the filter extension or solely with the search API?
  3. Are there any changes that need to be made in stac-fastapi-pgstac or in pgstac to make it compatible with that backend?

Thanks in advance.

@jamesfisher-geo
Copy link
Contributor Author

jamesfisher-geo commented May 6, 2024

Thank you for the comments @drnextgis

  1. AggregationExtensionPostRequest is defined but not used.

Ah, yes. The aggregation extension spec defines the methods for /aggregate and /aggregation should be get or post. I will add post to the available methods.

  1. Is it meant to be used with the filter extension or solely with the search API?

The aggregation extension spec as-is relies only on the core item-search afaik. However, the one implementation in stac-server includes cql2 query parameter, meaning it relies on the Filter extension. @philvarner may have an opinion on if that is something specific to the implementation, or if there are updates needed in the Aggregation extension spec.

  1. Are there any changes that need to be made in stac-fastapi-pgstac or in pgstac to make it compatible with that backend?

I am working on an implementation for elasticsearch and opensearch. I am not sure how an implementation would be handled in pgstac.

from stac_fastapi.types.search import APIRequest


@attr.s
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can filter be included in this? I'm not sure how the extensions interact.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have tested with filter included on an implementation and it works great.

I am also not clear on how extension interact. For instance, could collections, datetime, bbox, and intersects be excluded here since they are part of the core search? I'm inclined to keep so the request class correlates directly to the extension spec.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, i think you should extend BaseSearchGetRequest and BaseSearchPostRequest and just add the aggregations param. You also pickup limit that's ignored, but that's find for a datamodel like this since there's a lot less duplication

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have changed it to be build off base Search and Filter. This makes the implementation dependent on the Filter extension. But I don't think that is an issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamesfisher-gis sorry I'm just realizing this now but I don't think we should do this, at least I don't see in the aggregate extension why the filter attributes should be enabled with the aggregations ones.

I think it will be up to the implementers to add the filter extension and also handle them in the client.

Copy link
Contributor Author

@jamesfisher-geo jamesfisher-geo Jun 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey. That makes sense.

I have a couple other small bug fixes for aggregation. I will submit a PR today that removes the Filter extension dependency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe wait because I will submit a PR that takes care of #713 soon

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamesfisher-gis in fact, go ahead because my PR is a no-go in fact 😓

@jamesfisher-geo
Copy link
Contributor Author

Hey @philvarner let me know what you think of the changes to this PR. I think it is close to being complete.

@jonhealy1 jonhealy1 self-requested a review June 3, 2024 16:01
@jonhealy1
Copy link
Collaborator

Need to resolve conflicts too

@jonhealy1 jonhealy1 self-requested a review June 3, 2024 16:42
Copy link
Collaborator

@jonhealy1 jonhealy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - nice work

@jonhealy1
Copy link
Collaborator

Let's get @vincentsarago thoughts on this

@jonhealy1 jonhealy1 requested a review from philvarner June 3, 2024 16:49
Copy link
Member

@vincentsarago vincentsarago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamesfisher-gis thanks for the great work.

The overall code looks good, I just have some worries about the inter dependencies of extensions and types submodule.

cc @jonhealy1

stac_fastapi/types/stac_fastapi/types/aggregation.py Outdated Show resolved Hide resolved
stac_fastapi/types/stac_fastapi/types/aggregation.py Outdated Show resolved Hide resolved
stac_fastapi/types/stac_fastapi/types/aggregation.py Outdated Show resolved Hide resolved
stac_fastapi/types/stac_fastapi/types/core.py Outdated Show resolved Hide resolved
@jonhealy1
Copy link
Collaborator

Linting

@jamesfisher-geo
Copy link
Contributor Author

Linting

@jonhealy1 Sorry, fixed

@jonhealy1
Copy link
Collaborator

@vincentsarago Hi Vincent. How does this look now?

@vincentsarago
Copy link
Member

Great work @jamesfisher-gis

@vincentsarago vincentsarago merged commit 8075fc9 into stac-utils:main Jun 11, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants