Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-39876: Add bandaid for pydantic v2 #859

Merged
merged 6 commits into from
Jul 5, 2023
Merged

DM-39876: Add bandaid for pydantic v2 #859

merged 6 commits into from
Jul 5, 2023

Conversation

timj
Copy link
Member

@timj timj commented Jul 3, 2023

Pydantic v2 breaks everything. Rather than pinning to <2 instead use the facility provided by v2 to import v1 APIs. This will give us some flexibility on timeline for fixing things to work only with v2.

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

@codecov
Copy link

codecov bot commented Jul 3, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01 🎉

Comparison is base (ef41fb5) 87.89% compared to head (9851811) 87.90%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #859      +/-   ##
==========================================
+ Coverage   87.89%   87.90%   +0.01%     
==========================================
  Files         270      270              
  Lines       35544    35589      +45     
  Branches     7452     7452              
==========================================
+ Hits        31241    31286      +45     
  Misses       3145     3145              
  Partials     1158     1158              
Impacted Files Coverage Δ
python/lsst/daf/butler/_quantum_backed.py 88.77% <100.00%> (+0.17%) ⬆️
python/lsst/daf/butler/core/datasets/ref.py 83.96% <100.00%> (+0.23%) ⬆️
python/lsst/daf/butler/core/datasets/type.py 84.77% <100.00%> (+0.19%) ⬆️
...thon/lsst/daf/butler/core/datastoreCacheManager.py 82.91% <100.00%> (+0.12%) ⬆️
python/lsst/daf/butler/core/datastoreRecordData.py 89.77% <100.00%> (+0.36%) ⬆️
...hon/lsst/daf/butler/core/dimensions/_coordinate.py 87.56% <100.00%> (+0.10%) ⬆️
python/lsst/daf/butler/core/dimensions/_graph.py 86.39% <100.00%> (+0.24%) ⬆️
python/lsst/daf/butler/core/dimensions/_records.py 86.06% <100.00%> (+0.25%) ⬆️
python/lsst/daf/butler/core/logging.py 95.45% <100.00%> (+0.06%) ⬆️
python/lsst/daf/butler/core/quantum.py 87.79% <100.00%> (+0.17%) ⬆️
... and 5 more

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@andy-slac andy-slac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, one minor question.

Comment on lines -178 to 186
funcName: str | None
funcName: str | None = None
process: int
processName: str
exc_info: str | None
exc_info: str | None = None
MDC: dict[str, str]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with v1? I thought that pydantic (v1) does not allow non-default attributes after default ones (but I may be thinking of dataclasses)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made that change first with the auto updater and it seemed to be okay. I am running Jenkins over night on everything so I'll see whether everything works in the morning.

timj added 2 commits July 4, 2023 08:48
pydantic v2 is not compatible with daf_butler at the moment.
Change all the imports to first try to load the v1 API explicitly
before falling back to the default API. pydantic.v1 works with
v2.
@timj
Copy link
Member Author

timj commented Jul 4, 2023

I'm not sure what to do about the mypy failures. Mypy doesn't know about pydantic.v1 and with pydantic v1 it complains about thta type not existing because it never falls back to the except block. With pydantic v2 installed it gets even more confused.

<3 to protect us against the v3 breakage.
@timj
Copy link
Member Author

timj commented Jul 4, 2023

In theory changing mypy.ini to use pydantic.v1 with pydantic v2 and then adding type: ignore on the fallback imports does work. Unfortunately it doesn't work at the moment because fastapi pins pydantic to v1 so when we install that pydantic reverts to v1 and then mypy gets confused about the pydantic.v1 import (which we have to do first), since mypy always picks up the first import it sees.

timj added 3 commits July 4, 2023 09:39
This will only work in pydantic v2 with mypy but typing with
pydantic is already broken with v1 with the changes on this
branch.
@timj timj merged commit 6234c53 into main Jul 5, 2023
13 of 14 checks passed
@timj timj deleted the tickets/DM-39876 branch July 5, 2023 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants