Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Option to deploy single workflow per project #111

Closed
nachatz opened this issue May 8, 2024 · 1 comment
Closed

[FEATURE] Option to deploy single workflow per project #111

nachatz opened this issue May 8, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@nachatz
Copy link

nachatz commented May 8, 2024

Is your feature request related to a problem? Please describe.

Currently, when deploying a given project in Brickflow, it bundles and deploys all workflows under the project. Ideally, for versioned individual workflows we'd like to have the option to deploy just 1 workflow at a time. Ex.

Projects:
...Project1
......workflow_a.py
......workflow_b.py
...Project2
......workflow_a.py
......workflow_b.py

brickflow projects deploy --project Project1 -e local deploys workflow_a and workflow_b, a nice addition would be to have the option to only deploy workflow_a or workflow_b.

   
Describe the solution you'd like

brickflow projects deploy --project Project1 --workflow workflow_a -e local with --workflow as optional

Where the bundled path could then possibly include the workflow name

...user/project/workflow/env

   
Describe alternatives you've considered
The alternative is creating a new project per workflow. In our case where we want a ETL pipeline's stages very carefully bundled, versioned, and released this yields 3 new projects at minimum per pipeline (with 3+ pipelines this yields 9+ minimum total projects increasing codebase size & complexity).

@nachatz nachatz added the enhancement New feature or request label May 8, 2024
@pariksheet
Copy link
Collaborator

This is indeed very handy feature. However, we should enable this feature only for local development.

for higher environments, it does not make sense to deploy only set of workflows and should be part of version control.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants