Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(python): Properly expose InProcessQuery in docs, mark as unstable #17097

Merged
merged 4 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion py-polars/docs/source/reference/dataframe/group_by.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
GroupBy
=======

This namespace is available after calling :code:`DataFrame.group_by(...)`.
This namespace becomes available by calling `DataFrame.group_by(...)`.

.. currentmodule:: polars.dataframe.group_by
.. autosummary::
Expand Down
2 changes: 1 addition & 1 deletion py-polars/docs/source/reference/lazyframe/group_by.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
GroupBy
=======

This namespace comes available by calling `LazyFrame.group_by(..)`.
This namespace becomes available by calling `LazyFrame.group_by(...)`.

.. currentmodule:: polars.lazyframe.group_by

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
InProcessQuery
==============

This namespace comes available by calling `LazyFrame.collect(background=True)`.
This namespace becomes available by calling `LazyFrame.collect(background=True)`.

.. currentmodule:: polars.lazyframe.in_process

Expand Down
3 changes: 2 additions & 1 deletion py-polars/docs/source/reference/lazyframe/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ This page gives an overview of all public LazyFrame methods.
aggregation
attributes
descriptive
group_by
modify_select
miscellaneous
group_by
in_process

.. _lazyframe:

Expand Down
7 changes: 3 additions & 4 deletions py-polars/polars/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@
scan_parquet,
scan_pyarrow_dataset,
)
from polars.lazyframe import InProcessQuery, LazyFrame
from polars.lazyframe import LazyFrame
from polars.meta import (
build_info,
get_index_type,
Expand Down Expand Up @@ -208,10 +208,9 @@
"Expr",
"LazyFrame",
"Series",
# other classes
"InProcessQuery",
# schema
"Schema",
# polars.datatypes
# datatypes
"Array",
"Binary",
"Boolean",
Expand Down
5 changes: 3 additions & 2 deletions py-polars/polars/lazyframe/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from polars.lazyframe.frame import LazyFrame
from polars.lazyframe.in_process import InProcessQuery

__all__ = ["LazyFrame", "InProcessQuery"]
__all__ = [
"LazyFrame",
]
6 changes: 6 additions & 0 deletions py-polars/polars/lazyframe/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -1811,6 +1811,10 @@ def collect(
Run the query in the background and get a handle to the query.
This handle can be used to fetch the result or cancel the query.

.. warning::
Background mode is considered **unstable**. It may be changed
at any point without it being considered a breaking change.

Returns
-------
DataFrame
Expand Down Expand Up @@ -1887,7 +1891,9 @@ def collect(
_eager,
new_streaming,
)

if background:
issue_unstable_warning("Background mode is considered unstable.")
return InProcessQuery(ldf.collect_concurrently())

# Only for testing purposes atm.
Expand Down
11 changes: 6 additions & 5 deletions py-polars/polars/lazyframe/in_process.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ class InProcessQuery:
"""

def __init__(self, ipq: PyInProcessQuery) -> None:
self.ipq = ipq
self._inner = ipq

def cancel(self) -> None:
"""Cancel the query at earliest convenience."""
self.ipq.cancel()
self._inner.cancel()

def fetch(self) -> DataFrame | None:
"""
Expand All @@ -32,11 +32,12 @@ def fetch(self) -> DataFrame | None:
If it is ready, a materialized DataFrame is returned.
If it is not ready it will return `None`.
"""
out = self.ipq.fetch()
out = self._inner.fetch()
if out is not None:
return wrap_df(out)
return None
else:
return None

def fetch_blocking(self) -> DataFrame:
"""Await the result synchronously."""
return wrap_df(self.ipq.fetch_blocking())
return wrap_df(self._inner.fetch_blocking())