Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid duplicate requests #105

Merged
merged 117 commits into from
Dec 27, 2023
Merged

Avoid duplicate requests #105

merged 117 commits into from
Dec 27, 2023

Conversation

Gallaecio
Copy link
Contributor

Fixes #91

wRAR and others added 30 commits July 3, 2023 17:44
@wRAR
Move the provider deps into an optional feature.
5908b6c
@wRAR
Run the provider tests on CI.
eaa3409
@kmike
Merge pull request scrapy-plugins#98 from scrapy-plugins/scrapy-poet-…
a3f3e15 
…optional

Move the provider deps into an optional feature.
@kmike
ZyteApiProvider: support more data types
5ad4d79
@kmike
pre-commit fixes
914d599
@kmike
document requirements for scrapy-poet integration
3c81296
@kmike
cleanup tools configuration
be25b79 
* move to pyproject.toml when possible
* use pre-commit in tox, don't duplicate configuration
* don't run mypy in pre-commit
* update flake8, mypy and isort
* update github actions
@kmike
fix black configuration
edd2ed2
@kmike
document new dependencies which scrapy-zyte-api can provide
44c3a5f
@kmike
proper mypy config
8fd2138
@kmike @Gallaecio
Update README.rst
7ac7f12 
Co-authored-by: Adrián Chaves <[email protected]>
@kmike
make SCRAPY_POET_PROVIDERS configuration copy-pasteable (scrapy-plugi…
aeca861 
…ns#102)
@kmike
Merge pull request scrapy-plugins#101 from scrapy-plugins/cleanup-ci
fd91fbf 
Cleanup tools configuration
@kmike
Merge pull request scrapy-plugins#100 from scrapy-plugins/more-data-t…
bd4bd6b 
…ypes

Support more data types in the provider
@kmike
add build/ folder to gitignore (scrapy-plugins#103)
6d549b7
@kmike
add a quick start section to readme
226620c
@kmike
fix rst syntax in readme
139f55e
@kmike
Merge pull request scrapy-plugins#104 from scrapy-plugins/quickstart
267d689 
add a quick start section to readme
@Gallaecio
WIP
23c4940
@Gallaecio
Support driving Zyte API requests through a proxy
1c7db87
@wRAR
Add release notes for 0.10.0.
4e4c829
@wRAR
Merge pull request scrapy-plugins#109 from scrapy-plugins/relnotes-0.…
0c8b648 
…10.0

Add release notes for 0.10.0.
@Gallaecio
Reword the behind-a-proxy readme entry to minimize confusion
46ae308
@Gallaecio
Merge remote-tracking branch 'scrapy-plugins/main' into trust-env
24a4914
@Gallaecio
CHANGES:rst: cover ZYTE_API_USE_ENV_PROXY
4361ca7
@kmike
Merge pull request scrapy-plugins#106 from Gallaecio/trust-env
28927c0 
Support driving Zyte API requests through a proxy
@wRAR
Bump version: 0.9.0 → 0.10.0
23762b6
@BurnzZ
introduce ZYTE_API_MAX_REQUESTS
febe99f
@BurnzZ
update tests to include non-ZAPI requests
5a8ce24
@BurnzZ
refactor conditional check on max request reached
561d9ff
Gallaecio and others added 29 commits November 14, 2023 10:46
@Gallaecio
Set a close reason for bad key and suspended account scenarios
e55e99c
@wRAR
Merge remote-tracking branch 'origin/main' into annotated-support
87bd23a
@wRAR
Fix removing *Options.
4b5c088
@wRAR
Update the pinned zyte-api version in CI (scrapy-plugins#149)
aa3f77b
@wRAR
Fixes and improvements.
b63ad66
@wRAR
Fix CI issues.
15fc06e
@wRAR
More fixes.
08a5f48
@wRAR
Install more deps for the mypy test.
e0ba154
@kmike
Merge pull request scrapy-plugins#148 from Gallaecio/bad-auth
37e3517 
Set a close reason for bad key and suspended account scenarios
@wRAR
Fix an old typing issue.
2d936fd
@Gallaecio
Use Sphinx and ReadTheDocs for the documentation (scrapy-plugins#150)
b892453
@Gallaecio
Drop the Set-Cookie header (scrapy-plugins#132)
493a48c
@Gallaecio
Improve forbidden domain handling (scrapy-plugins#147)
dc09ac3
@Gallaecio
Implement a parameter map (scrapy-plugins#151)
702cc63
@Gallaecio
Fail on SPM + transparent mode (scrapy-plugins#152)
d701f48
@wRAR
Merge remote-tracking branch 'origin/main' into annotated-support
cad9c3a
@wRAR
Bump the andi version.
e9be1a3
@wRAR
Fix and improve extractFrom tests.
3ee24f0
@wRAR
Roll back capture_exceptions.
2b0d659
@wRAR
Bump the scrapy-poet version.
de01fc2
@wRAR
Bump the web-poet version.
412475e
@wRAR
Move ExtractFrom into scrapy_zyte_api/_annotations.py.
5305b46
@wRAR
Add docs for ExtractFrom.
f5bb894
@wRAR
Merge pull request scrapy-plugins#141 from scrapy-plugins/annotated-s…
11a9b03 
…upport

Initial support for typing.Annotated, support for extractFrom
@wRAR
Release notes for 0.13.0.
a80fc0b
@wRAR
Merge pull request scrapy-plugins#153 from scrapy-plugins/relnotes-0.…
fab5e58 
…13.0

Release notes for 0.13.0.
@wRAR
Bump version: 0.12.2 → 0.13.0
0536f42
@Gallaecio
Merge remote-tracking branch 'scrapy-plugins/main' into provider-fix-…
a5f909c 
…indirect-duplication
@Gallaecio
Require scrapy-poet >= 0.19.0
857b51c
@Gallaecio Gallaecio merged commit 857b51c into scrapy-plugins:provider-fix-indirect-duplication Dec 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Gallaecio @wRAR @kmike @BurnzZ @PyExplorer