Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(python): Continue converting to PyO3 0.21 Bound<> APIs #16081

Merged
merged 5 commits into from
May 9, 2024

Conversation

itamarst
Copy link
Contributor

@itamarst itamarst commented May 6, 2024

Fixes #16078

@github-actions github-actions bot added internal An internal refactor or improvement rust Related to Rust Polars labels May 6, 2024
@itamarst itamarst marked this pull request as ready for review May 6, 2024 15:45
@stinodego stinodego changed the title chore(rust): Continue converting to PyO3 0.21 Bound<> APIs chore(python): Continue converting to PyO3 0.21 Bound<> APIs May 6, 2024
@stinodego stinodego removed the rust Related to Rust Polars label May 6, 2024
@github-actions github-actions bot added the python Related to Python Polars label May 6, 2024
Copy link
Member

@stinodego stinodego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

PS: Even though you're only touching Rust code, this should get a 'python' tag since it affects our Python releases, not our Rust ones.

@itamarst
Copy link
Contributor Author

itamarst commented May 7, 2024

Will use correct tag for future PRs, thanks.

Also note that I can't merge this, so someone else will have to :)

@@ -46,7 +48,7 @@ impl PyExpr {
let name_mapper = Arc::new(move |name: &str| {
Python::with_gil(|py| {
let out = name_mapper.call1(py, (name,)).unwrap();
let out: SmartString = out.extract::<&str>(py).unwrap().into();
let out: SmartString = out.extract::<Cow<str>>(py).unwrap().into();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use Cow in favor of pybackedstr here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Less typing to convert to SmartString. Could do it the other way too, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, looking at the FromPyObject implementations in Pyo3, I now see what the "fastest" way is to get a rust &str.

And that is indeed going via Cow. As they write obj.downcast::<PyString>()?.to_cow().map(Cow::into_owned) for String, where to_cow says:

"""
Converts the PyString into a Rust string, avoiding copying when possible.
"""

So I do think this is the fastest way to get a borrowing string out of python. I think we should favor this than over pybytes in other places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Is that specific to this PR too? If not, can it be merged?
  2. I am not sure that Cow<str> is always the best option. In particular, it has a lifetime that is tied to the original object. PyBackedStr in contrast is an owned object, so it's a lot easier to work with some contexts. And it has some overhead but I don't think its copying overhead, it's more "having a Py<>" overhead, so like... a Python incref basically?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I mean in the borrowing &str case. Not related to this PR indeed. Thanks!

@ritchie46 ritchie46 merged commit 1d13d62 into pola-rs:main May 9, 2024
17 checks passed
@itamarst itamarst deleted the 16078-pyo3-0.21-bound-apis branch May 10, 2024 01:27
Wouittone pushed a commit to Wouittone/polars that referenced this pull request Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal An internal refactor or improvement python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Continue the very long journey towards PyO3 0.21 Bound<> APIs
4 participants