Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't raise if the task already have been cancelled explicitly #237

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -89,4 +89,4 @@ ENV/
.ropeproject

.mypy_cache
.pytest_cache
.pytest_cache
2 changes: 2 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ CHANGES

* Deprecate synchronous context manager usage

* Don't raise `TimeoutError` if the task already have been cancelled explicitly (#229)

3.0.1 (2018-10-09)
------------------

Expand Down
23 changes: 12 additions & 11 deletions async_timeout/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ def reject(self) -> None:
# cancel is maybe better name but
# task.cancel() raises CancelledError in asyncio world.
if self._state not in (_State.INIT, _State.ENTER):
raise RuntimeError("invalid state {}".format(self._state.value))
raise RuntimeError(f"invalid state {self._state.value}")
self._reject()

def _reject(self) -> None:
Expand Down Expand Up @@ -179,30 +179,31 @@ def shift_to(self, deadline: float) -> None:
else:
# state is ENTER
raise asyncio.CancelledError
self._timeout_handler = self._loop.call_at(
deadline, self._on_timeout, self._task
)
self._timeout_handler = self._loop.call_at(deadline, self._on_timeout)

def _do_enter(self) -> None:
if self._state != _State.INIT:
raise RuntimeError("invalid state {}".format(self._state.value))
raise RuntimeError(f"invalid state {self._state.value}")
self._state = _State.ENTER

def _do_exit(self, exc_type: Type[BaseException]) -> None:
self._task = None # Early drop circular reference
if exc_type is asyncio.CancelledError and self._state == _State.TIMEOUT:
self._timeout_handler = None
raise asyncio.TimeoutError
# timeout is not expired
# timeout is not expired, propagate the exception
self._state = _State.EXIT
self._reject()
return None

def _on_timeout(self, task: "asyncio.Task[None]") -> None:
def _on_timeout(self) -> None:
# See Issue #229 and PR #230 for details
if task._fut_waiter and task._fut_waiter.cancelled(): # type: ignore[attr-defined] # noqa: E501
return
task.cancel()
self._state = _State.TIMEOUT
self._loop.call_soon(self._cancel)
Copy link
Member

@Dreamsorcerer Dreamsorcerer Sep 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this much harder to reason about, I think it's more risky than the other solutions.

The reason this somewhat works is that it changes the call order in the before case to:
-> t.cancel()
-> Timeout._on_timeout()
-> Timeout._on_exit()
-> Timeout._cancel() # This is a noop, as it's already exited, so only 1 cancellation happens.

I'm not really sure that provides any guarantees (particularly when you have a more complex project with many tasks in flight), as it's just delaying the cancellation, which happens to be after the exit has already happened.

In the after case, this is not fixed at all (just like all the other attempts), in this case the execution looks like:
-> Timeout._on_timeout()
-> t.cancel()
-> Timeout._cancel() # Again, we've triggered 2 cancellations in a row
-> Timeout._do_exit() # No way to know there were 2 cancellations.


def _cancel(self) -> None:
if self._task is not None:
self._task.cancel()
self._state = _State.TIMEOUT


def _current_task(loop: asyncio.AbstractEventLoop) -> "Optional[asyncio.Task[Any]]":
Expand Down
2 changes: 1 addition & 1 deletion async_timeout/py.typed
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Placeholder
Placeholder
12 changes: 6 additions & 6 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
isort==5.9.3
-e .
black==21.8b0; implementation_name=="cpython"
pytest==6.2.5
pytest-asyncio==0.15.1
pytest-cov==2.12.1
docutils==0.17.1
mypy==0.910; implementation_name=="cpython"
flake8==3.9.2
isort==5.9.3
mypy==0.910; implementation_name=="cpython"
pre-commit==2.15
-e .
pytest==6.2.5
pytest-asyncio==0.15.1
pytest-cov==2.12.1
8 changes: 4 additions & 4 deletions tests/test_timeout.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,9 +373,9 @@ async def test_task(deadline: float, loop: asyncio.AbstractEventLoop) -> None:
await t


@pytest.mark.xfail(
reason="The test is CPU performance sensitive, might fail on slow CI box"
)
# @pytest.mark.xfail(
# reason="The test is CPU performance sensitive, might fail on slow CI box"
# )
@pytest.mark.skipif(sys.version_info < (3, 7), reason="Not supported in 3.6")
@pytest.mark.asyncio
async def test_race_condition_cancel_after() -> None:
Expand All @@ -396,7 +396,7 @@ async def test_task(deadline: float, loop: asyncio.AbstractEventLoop) -> None:
loop = asyncio.get_running_loop()
deadline = loop.time() + 1
t = asyncio.create_task(test_task(deadline, loop))
loop.call_at(deadline + 0.0000000000001, t.cancel)
loop.call_at(deadline + 0.00000000001, t.cancel)
Copy link
Member

@Dreamsorcerer Dreamsorcerer Sep 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
loop.call_at(deadline + 0.00000000001, t.cancel)
loop.call_at(deadline + 0.000001, t.cancel)

I think you made the value so small that you've tricked your system into passing the tests by running the cancel() before the previous task. The value needs to be big enough to ensure the task is scheduled after the timeout task, but small enough that it happens immediately after without a chance to run the __exit__().

This test should be reliably failing. To verify the conditions, add prints to see when this cancel happens in relation to the timeout actions. The execution should be:
-> Timeout._on_timeout()
-> t.cancel()
-> Timeout._cancel()
-> Timeout._do_exit()

I suspect on your machine you've managed to change the execution order so t.cancel() happens first, which is what the previous test is for.

# If we get a TimeoutError, then the code is broken.
with pytest.raises(asyncio.CancelledError):
await t