v2.12.0

The minor release adds --build-arg option to shub image build/upload commands to provide build-time variables when building a custom Docker image.

v2.11.0

This release is mainly focused on maintenance (updating support for different Python versions, upgrading dependencies).

One of the key changes which you should be aware of is switching to newer versions of docker-py library with a different Python package name (docker-py -> docker) and different internal API. Please ensure you installed docker python package if you work with Docker images (and uninstalled docker-py).

New features:

• added --json mode for shub log (disabled by default)
• add sources support when creating requirements file from Pipfile

Enhancements:

• officially support newer Python versions (3.7, 3.8)
• upgrade docker-py package (docker-py<2.0.0 not supported)
• upgrade other (dev/test) dependencies where appropriate
• update .gitignore content
• address a DST-related issue in unittests

Deprecations:

• deprecate support of old Python versions (3.3, 3.4)

v2.10.0

Changes:

• add a new shub cancel command to cancel multiple jobs at once (#359)
• add --no-cache flag to ignore cache when building an image locally (#351)
• add --reauth flag to re-authenticate when pushing an image to a registry (#365)
• ensure that --insecure can't be used along with user credential parameters (#321)
• fix the last character of the stream log when building an image (#361)
• remove intermediate containers after a successful image build (#367)
• update a few dependencies to address known security vulnerabilities (#353)
• drop official support of Python 3.4
• development improvements and minor fixes

v2.9.0

Changes:

• add shub image run command to run custom images locally
• add Poetry support via pyproject.toml as a requirements file
• documentation updates, CI configuration fixes

v2.8.2

Changes:

• add an option to schedule jobs with custom environment variables
• replace legacy Connection client class with ScrapinghubClient
• check for a custom image size before pushing it to a Docker registry
• describe environment variables to setup/parametrize shub
• use DOCKER_API_VERSION env var instead of DOCKER_VERSION

v2.8.1

Changes:

• Remove use of async reserved word
• Respect per-project/per-spider units settings
• Improve Docker image registry parsing logic
• Drop official support for Python 3.3

v2.8.0

New features:

• Add Pipfile support when deployng with stacks

Enhancements:

• Add support for priority/units/tags for schedule command
• Allow custom Dockerfile name when using a custom image
• Check if project is too big to be deployed with stacks

Bugfixes:

• Fix issue with missing scripts when using a custom image
• Fix compatibility with a new version of docker-py
• Fix Travis/AppVeyor tests
• Clarify Custom Images contract
• Update outdated documentation links

v2.7.0

New features:

• Add shub bootstrap for quick-starting custom image projects

Enhancements:

• Deprecate list-spiders in custom images in favour of shub-image-info

v2.6.1

Bugfixes:

• Fix deploying "Scrapy" projects that have no settings section in their scrapy.cfg (because they contain only Python scripts and no Scrapy crawlers) #284

v2.6.0

This release brings many improvements for deploying custom images to Scrapy Cloud, particularly for non-Scrapy crawlers.

Enhancements:

• Extend onboarding wizard to include custom image configuration
• Use images.scrapinghub.com as default registry for custom images
• Unify workflow for deploying Scrapy projects and custom image projects (deploy custom image on shub deploy if configured)
• Improve progress reporting during image building, pushing, and deploying
• Test custom images before pushing them
• Add -S/--skip-tests parameter to shub image push and shub image upload
• Rewrite custom image docs
• Improve documentation of environment variables available on Scrapy Cloud
• Add singleton configuration (project: as shortcut to projects: default and similar for stacks)
• Rewrite configuration docs
• Unify how subcommands are called within shub
• Add parsing for new job URL format, adjust printed job URLs
• Create setup.py on shub image init instead of shub image build

Deprecations:

• Deprecate images section in scrapinghub.yml in favor of global default image and optional per-project configuration, and fail on ambiguous configurations
• Deprecate --async parameter for shub image deploy
• Deprecate --base-deps parameter for shub image init

Bugfixes:

• Fix requirements and trove classifier in setup.py
• Various smaller fixes throughout the docs
• Improve error output on shub image init without Scrapy project
• Improve error output on failed repository checkout