Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add support for run time parameters while deploying workflows #65

Open
4 tasks
pariksheet opened this issue Nov 19, 2023 · 1 comment
Open
4 tasks
Labels
enhancement New feature or request
Milestone

Comments

@pariksheet
Copy link
Collaborator

Currently user can specify custom params as hardcoded value

wf = Workflow(
    "my-workflow",
    default_cluster=cluster,
    common_task_parameters={'my_key': 'my_value'}
)

And there is no way to pass params at deployment time
poetry run brickflow projects deploy -e local

Cloud Information

  • AWS
  • Azure
  • GCP
  • Other

Describe the solution you'd like
User will pass the params (max 10) at deployment time
poetry run brickflow projects deploy -e local -p key1=value1 -p key2=value2

These params will be available as string in task parameters as brickflow_runtime_params.

For above example, brickflow_runtime_params='{'key1': 'value1', 'key2': 'value2'}'

Describe alternatives you've considered
User can get it working using ENVIRONMENT variable, not very ideal.

Additional context
Add any other context or screenshots about the feature request here.

@pariksheet pariksheet added the enhancement New feature or request label Nov 19, 2023
@asingamaneni asingamaneni added this to the v1.0.0 milestone Nov 21, 2023
@asingamaneni
Copy link
Collaborator

Disccussed with @pariksheet and @stikkireddy - We will be going with BRICKFLOW_PROJECT_PARAMS and use cli to pass the params. The parameters would be available through ctx and a helper function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants