Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamline histogram and stats generation #61

Merged
merged 8 commits into from
Jun 24, 2024

Conversation

gillins
Copy link
Member

@gillins gillins commented Jun 12, 2024

This PR does a few different things:

  1. Returns the created dataset from doTiledShepherdSegmentation_finalize/doTiledShepherdSegmentation so it is all ready to use by the caller.
  2. Writes the Histogram to the RAT by default (we have to calculate this anyway for the missing segments check)
  3. Calculates overviews in AWS Batch

@neilflood
Copy link
Member

I think tiling.py line 688 now needs to pass outDs.

@neilflood
Copy link
Member

Good work with the tests @gillins

I can see that this is because the file is now left open, since we are now passing the outDs back. However, I can no longer remember why we felt this was necessary. What would happen if we just left it as it was, and you re-opened it to do the histogram and overviews?

We can see from the way it scrambled the tests script that we have made a fundamental change to the behaviour of the API, and I don't remember why.

@gillins gillins marked this pull request as ready for review June 23, 2024 22:45
@gillins gillins merged commit 34b31ac into ubarsc:master Jun 24, 2024
2 checks passed
@gillins gillins deleted the rios_calcstats branch June 24, 2024 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants