Refactor how Dagster and user pipelines are built and deployed

[blarghmatey]

User Story

• As a data platform owner I would like to easily build and deploy new releases of the core Dagster services for patches and upgrades
• As a data engineer I would like to easily build and deploy new code and pipelines to a running Dagster instance and have visibility of when my changes are running

Description/Context

We currently have a script that uses a project called vdist for building our whole Dagster project as a debian package that is installed on an EC2 box via Salt. This is functional, but very cumbersome and coarse-grained. As we scale the number of pipelines and contributors that we would like to manage, we need a more streamlined way of packaging individual or groupings of pipelines and deploying them to a running Dagster environment.

Because we are using a mono-repo approach for our data platform we would like to use Pants to bundle the pipelines up with their relevant dependencies to create more streamlined bundles. It can also easily handle loading those packaged pipelines into Docker images that we can deploy to the running Dagster environment.

Dagster has a number of supported options for running user pipelines as containers. The most relevant for our environment is to either run them with Docker on an EC2 box or investigate using the ECS integration.

Acceptance Criteria

• I can package pipeline code into a Python archive for deployment using Pants
• I can build Docker images that contain my pipeline code and associated dependencies for deploying to my Dagster environment
• Document steps involved in adding a pipeline to Dagster deployment #276
• I have a Concourse pipeline that packages pipeline code and builds associated Docker images
• I have a Concourse pipeline that deploys updated images to the target environment (ECS or EC2)