v2.6.0
This release brings many improvements for deploying custom images to Scrapy Cloud, particularly for non-Scrapy crawlers.
Enhancements:
- Extend onboarding wizard to include custom image configuration
- Use
images.scrapinghub.com
as default registry for custom images - Unify workflow for deploying Scrapy projects and custom image projects (deploy custom image on
shub deploy
if configured) - Improve progress reporting during image building, pushing, and deploying
- Test custom images before pushing them
- Add
-S/--skip-tests
parameter toshub image push
andshub image upload
- Rewrite custom image docs
- Improve documentation of environment variables available on Scrapy Cloud
- Add singleton configuration (
project:
as shortcut toprojects: default
and similar forstacks
) - Rewrite configuration docs
- Unify how subcommands are called within
shub
- Add parsing for new job URL format, adjust printed job URLs
- Create
setup.py
onshub image init
instead ofshub image build
Deprecations:
- Deprecate
images
section inscrapinghub.yml
in favor of global defaultimage
and optional per-project configuration, and fail on ambiguous configurations - Deprecate
--async
parameter forshub image deploy
- Deprecate
--base-deps
parameter forshub image init
Bugfixes:
- Fix requirements and trove classifier in
setup.py
- Various smaller fixes throughout the docs
- Improve error output on
shub image init
without Scrapy project - Improve error output on failed repository checkout