Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/airflow] Consistency on Airflow configuration & init improvements #30349

Merged
merged 8 commits into from
Nov 11, 2024

Conversation

juan131
Copy link
Contributor

@juan131 juan131 commented Nov 8, 2024

Description of the change

This PR changes how Airflow configuration is adapted based on users parameters: instead of setting a bunch of env. vars and relying on main containers' init logic to prepare the config file(s), everything (except config properties containing credentials) is prepared by the chart in the ConfigMap, then init containers are used to complete configuration with config properties containing credentials.

The goals is to ensure the airflow.cfg file is the same on every Airflow component (scheduler, worker, web server, etc.) given currently there are small discrepancies. Also, similar to #30246, applying changes related to credentials on init-containers has important benefits on bootstrapping, issues isolation and resource usage efficiency.

Benefits

  • Configuration consistency
  • Improved bootstrapping

Possible drawbacks

None

Applicable issues

Additional information

Other changes included:

  • "wait-for-web-server` init container is replaced by "wait-for-db-migrations" which is more accurate.
  • pod_template.yaml is moved to its own ConfigMap.
  • Simplifications on helper libraries.

Checklist

  • Chart version bumped in Chart.yaml according to semver. This is not necessary when the changes only affect README.md files.
  • Variables are documented in the values.yaml and added to the README.md using readme-generator-for-helm
  • Title of the pull request follows this pattern [bitnami/<name_of_the_chart>] Descriptive title
  • All commits signed off and in agreement of Developer Certificate of Origin (DCO)

@github-actions github-actions bot added airflow verify Execute verification workflow for these changes bitnami labels Nov 8, 2024
@github-actions github-actions bot requested a review from fmulero November 8, 2024 13:49
@juan131 juan131 marked this pull request as ready for review November 8, 2024 16:46
Signed-off-by: Bitnami Containers <[email protected]>
Copy link
Collaborator

@fmulero fmulero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great @juan131

I've just left a suggestion. Could you give it a look?

bitnami/airflow/templates/_helpers.tpl Show resolved Hide resolved
bitnami/airflow/templates/web/configmap.yaml Outdated Show resolved Hide resolved
Co-authored-by: Fran Mulero <[email protected]>
Signed-off-by: Juan Ariza Toledano <[email protected]>
Copy link
Collaborator

@fmulero fmulero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! LGTM

@juan131 juan131 merged commit 6074846 into bitnami:main Nov 11, 2024
12 checks passed
@juan131 juan131 deleted the feat/airflow-configmap branch November 11, 2024 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airflow bitnami solved verify Execute verification workflow for these changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Airflow LDAP variables are checked incorrectly
3 participants