Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump scrapy-poet #156

Merged
merged 3 commits into from
Jan 5, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions docs/usage/scrapy-poet.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,6 @@ priority of 2000, so when you have page objects producing
``ZyteApiProvider`` if you want these items to come from these page objects,
and lower values if you want them to come from Zyte API.

Currently, when ``ItemProvider`` is used together with ``ZyteApiProvider``,
it may make more requests than is optimal: the normal Scrapy response will be
always requested even when using a :class:`~scrapy_poet.DummyResponse`
annotation, and in some dependency combinations two Zyte API requests will be
made for the same page. We are planning to solve these problems in the future
releases of :doc:`scrapy-poet <scrapy-poet:index>` and scrapy-zyte-api.


Dependency annotations
======================

Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ def get_version():
"zyte-api>=0.4.8",
],
extras_require={
# Sync with [testenv:provider-pinned] @ tox.ini
# Sync with [testenv:pinned-provider] @ tox.ini
"provider": [
"andi>=0.5.0",
"scrapy-poet>=0.18.0",
"andi>=0.6.0",
"scrapy-poet>=0.19.0",
"web-poet>=0.15.1",
"zyte-common-items>=0.7.0",
]
Expand Down
2 changes: 0 additions & 2 deletions tests/test_providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,6 @@ def parse_( # type: ignore[override]
assert "product" in item


# https://github.com/scrapy-plugins/scrapy-zyte-api/issues/91
@pytest.mark.xfail(reason="Not implemented yet", raises=AssertionError, strict=True)
@ensureDeferred
async def test_itemprovider_requests_indirect_dependencies(fresh_mockserver):
class ItemDepSpider(ZyteAPISpider):
Expand Down
3 changes: 2 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,8 @@ extras = provider
deps =
# scrapy-poet >= 0.4.0 depends on scrapy >= 2.6.0
{[testenv:pinned-scrapy-2x6]deps}
scrapy-poet==0.18.0
andi==0.6.0
scrapy-poet==0.19.0
web-poet==0.15.1
zyte-common-items==0.7.0

Expand Down