Skip to content

v2.6.0

Compare
Choose a tag to compare
@jdemaeyer jdemaeyer released this 22 May 10:47
· 206 commits to master since this release

This release brings many improvements for deploying custom images to Scrapy Cloud, particularly for non-Scrapy crawlers.

Enhancements:

  • Extend onboarding wizard to include custom image configuration
  • Use images.scrapinghub.com as default registry for custom images
  • Unify workflow for deploying Scrapy projects and custom image projects (deploy custom image on shub deploy if configured)
  • Improve progress reporting during image building, pushing, and deploying
  • Test custom images before pushing them
  • Add -S/--skip-tests parameter to shub image push and shub image upload
  • Rewrite custom image docs
  • Improve documentation of environment variables available on Scrapy Cloud
  • Add singleton configuration (project: as shortcut to projects: default and similar for stacks)
  • Rewrite configuration docs
  • Unify how subcommands are called within shub
  • Add parsing for new job URL format, adjust printed job URLs
  • Create setup.py on shub image init instead of shub image build

Deprecations:

  • Deprecate images section in scrapinghub.yml in favor of global default image and optional per-project configuration, and fail on ambiguous configurations
  • Deprecate --async parameter for shub image deploy
  • Deprecate --base-deps parameter for shub image init

Bugfixes:

  • Fix requirements and trove classifier in setup.py
  • Various smaller fixes throughout the docs
  • Improve error output on shub image init without Scrapy project
  • Improve error output on failed repository checkout