Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typecheck some API interfaces #593

Merged
merged 28 commits into from
Sep 17, 2024

Conversation

mfisher87
Copy link
Member

@mfisher87 mfisher87 commented Aug 27, 2024

I'm going to be changing a lot in the refactor to use Harmony. The typechecker is going to help find sneaky bugs along the way. This is a start, but I'd really like feedback.

My main goal here is to avoid refactoring the code more than needed and set the stage for future cleanups and refactors. E.g. I think long-term, we will want to split up the Parameters class instead of having it handle all the kinds of parameters.

I'm using pyright here for the first time, and I'm pleased with it compared to Mypy so far. The reason I switched to Pyright: python/mypy#12098

Copy link

github-actions bot commented Aug 27, 2024

Binder 👈 Launch a binder notebook on this branch for commit 5d8c38f

I will automatically update this comment whenever this PR is modified

Binder 👈 Launch a binder notebook on this branch for commit 599a40e

Binder 👈 Launch a binder notebook on this branch for commit 1299db1

Binder 👈 Launch a binder notebook on this branch for commit 4c8ec7b

Binder 👈 Launch a binder notebook on this branch for commit bf54226

Binder 👈 Launch a binder notebook on this branch for commit cb4c043

Binder 👈 Launch a binder notebook on this branch for commit cac9d1c

Binder 👈 Launch a binder notebook on this branch for commit 181099f

Binder 👈 Launch a binder notebook on this branch for commit e6e7fe2

Binder 👈 Launch a binder notebook on this branch for commit 0440fa9

Binder 👈 Launch a binder notebook on this branch for commit df78ca2

Binder 👈 Launch a binder notebook on this branch for commit 126d3fc

Binder 👈 Launch a binder notebook on this branch for commit ee70196

Binder 👈 Launch a binder notebook on this branch for commit 434c28a

Binder 👈 Launch a binder notebook on this branch for commit 2247b2c

Binder 👈 Launch a binder notebook on this branch for commit 18bd814

Binder 👈 Launch a binder notebook on this branch for commit a8996d1

Binder 👈 Launch a binder notebook on this branch for commit dfe4c09

Binder 👈 Launch a binder notebook on this branch for commit f23f5c2

Binder 👈 Launch a binder notebook on this branch for commit fddf23a

Binder 👈 Launch a binder notebook on this branch for commit e155a9b

Binder 👈 Launch a binder notebook on this branch for commit 51b3492

Binder 👈 Launch a binder notebook on this branch for commit 7f60849

Binder 👈 Launch a binder notebook on this branch for commit bb96a5f

Binder 👈 Launch a binder notebook on this branch for commit 4cb5956

Binder 👈 Launch a binder notebook on this branch for commit 7a423a8

Binder 👈 Launch a binder notebook on this branch for commit 6a4acd9

Binder 👈 Launch a binder notebook on this branch for commit e41aed0

Binder 👈 Launch a binder notebook on this branch for commit 81d60dd

Copy link

codecov bot commented Aug 27, 2024

Codecov Report

Attention: Patch coverage is 85.52632% with 11 lines in your changes missing coverage. Please review.

Project coverage is 65.66%. Comparing base (d0bc315) to head (81d60dd).
Report is 1 commits behind head on development.

Files with missing lines Patch % Lines
icepyx/core/granules.py 50.00% 4 Missing ⚠️
icepyx/core/query.py 69.23% 4 Missing ⚠️
icepyx/core/APIformatting.py 85.71% 0 Missing and 3 partials ⚠️

❗ There is a different number of reports uploaded between BASE (d0bc315) and HEAD (81d60dd). Click for more details.

HEAD has 4 uploads less than BASE
Flag BASE (d0bc315) HEAD (81d60dd)
6 2
Additional details and impacted files
@@               Coverage Diff               @@
##           development     #593      +/-   ##
===============================================
- Coverage        71.60%   65.66%   -5.95%     
===============================================
  Files               37       38       +1     
  Lines             3064     3122      +58     
  Branches           596      599       +3     
===============================================
- Hits              2194     2050     -144     
- Misses             766      984     +218     
+ Partials           104       88      -16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from 1299db1 to 4c8ec7b Compare August 27, 2024 01:07
@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from bf54226 to cb4c043 Compare August 27, 2024 02:11
icepyx/core/types.py Outdated Show resolved Hide resolved
icepyx/core/granules.py Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
icepyx/core/types.py Outdated Show resolved Hide resolved
icepyx/core/types.py Outdated Show resolved Hide resolved
icepyx/core/granules.py Outdated Show resolved Hide resolved
icepyx/core/granules.py Outdated Show resolved Hide resolved
icepyx/core/query.py Outdated Show resolved Hide resolved
icepyx/core/granules.py Outdated Show resolved Hide resolved
@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from 2247b2c to 18bd814 Compare September 12, 2024 16:18
mfisher87 and others added 2 commits September 12, 2024 10:21
Co-Authored-By: Jessica Scheick <[email protected]>
Co-Authored-By: Jessica Scheick <[email protected]>
@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from 18bd814 to a8996d1 Compare September 12, 2024 16:22
@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from f23f5c2 to fddf23a Compare September 12, 2024 18:55
doc/source/conf.py Outdated Show resolved Hide resolved
CMRparams,
reqparams,
CMRparams: CMRParams,
reqparams: EGISpecificParamsSearch,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
reqparams: EGISpecificParamsSearch,
reqparams: EGISpecificParamsDownload,

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is Query.reqparams always of type EGISpecificParamsDownload? Or only sometimes (this is my understanding)?

Copy link
Member Author

@mfisher87 mfisher87 Sep 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4cb5956

I think this is probably the wrong way to handle things. We're telling the typechecker "Ignore your eyes and ears, just trust us" with the cast() call. Instead, we can add a "typeguard" which is a runtime check that asserts a narrower type, e.g. a literal assert isinstance(...) or if isinstance(...): raise. I am hesitant to add runtime checks without fully grokking the code, but perhaps we do know for sure that reqparams is always EGISpecificParamsDownload at the point we're cast()ing.

Copy link
Member

@JessicaS11 JessicaS11 Sep 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is Query.reqparams always of type EGISpecificParamsDownload? Or only sometimes (this is my understanding)?

The latter - it's only sometimes of type EGISpecificParamsDownload (when Query._reqparams._reqtype == "download"). Otherwise it's of type "search", which maps to our EGIRequiredParamsSearch.

but perhaps we do know for sure that reqparams is always EGISpecificParamsDownload at the point we're cast()ing.

I think this is true.

mfisher87 and others added 3 commits September 12, 2024 18:00
Co-authored-by: Jessica Scheick <[email protected]>
Excludes CMR typeddicts which contain symbols in some keys.
Is this the right way to handle this? Perhaps we need a typeguard.

Co-Authored-By: Jessica Scheick <[email protected]>
@mfisher87 mfisher87 force-pushed the refactor-and-typecheck-api-interfaces branch from bb96a5f to 4cb5956 Compare September 13, 2024 01:29
CMRparams,
reqparams,
CMRparams: CMRParams,
reqparams: EGIRequiredParamsDownload,
subsetparams,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
subsetparams,
subsetparams: Union[EGIRequiredParamsSubset, dict[Never, Never]],

I might have the syntax wrong. In some cases we specify the type from types.py, and in others from APIformatting.py (e.g. in query.py).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dict[Never, Never] means an empty dict, and this change doesn't pass type checking. In what case would reqparams={} be passed?

Copy link
Member

@JessicaS11 JessicaS11 Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was simply adding a type for subsetparams, since it wasn't specified. It sounds like it would need to be something more like Optional[EGIRequiredParamsSubset] so it doesn't otherwise require an empty dict.

In what case would reqparams={} be passed?

dict[Never, Never] is not being specified as a valid type to reqparams.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops, I meant subsetparams :)

Copy link
Member

@JessicaS11 JessicaS11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this for merging now that we've released the last of v1 (v1.3.0). I think we've worked out most of the kinks and figure any others will be ironed out as we continue moving towards the refactor in v2. I'll be on PTO Tuesday and I know in our discussion today you said this was a critical next step for you for making progress.

@mfisher87 mfisher87 merged commit 1606dd0 into development Sep 17, 2024
10 of 12 checks passed
@mfisher87 mfisher87 deleted the refactor-and-typecheck-api-interfaces branch September 17, 2024 23:48
@mfisher87
Copy link
Member Author

Thanks, @JessicaS11 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review::priority This change is preferred for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants