Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stabilize charts #78

Merged
merged 7 commits into from
Jan 23, 2024
Merged

Stabilize charts #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
97f0dfb
TODO list
chaen Dec 7, 2023
d04576a
Add a diracx-demo-cli deployment and create demo CS content from there
chaen Jan 17, 2024
1fb85e8
Better chart docs rendering
chaen Jan 17, 2024
e30a442
factorize mysql config
chaen Jan 18, 2024
8cdca9f
init job for opensearch
chaen Jan 19, 2024
7b20037
cleanup
chaen Jan 19, 2024
19d9c8b
allow to specify different internal name for DBs
chaen Jan 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 47 additions & 36 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,17 @@ When running the demo some an `values.yaml` file is created as `.demo/values.yam
This file can be used with helm to interact with the running demo to make changes to what is running in the cluster.

```bash

# Show what will be changed by running "helm upgrade"
helm diff upgrade diracx-demo ./diracx --values .demo/values.yaml
# Note: helm diff has a ``bug`` that requires you to specify the existing password
# https://github.com/databus23/helm-diff/issues/460

export RABBITMQ_PASSWORD=$(kubectl get secret --namespace "default" rabbitmq-secret -o jsonpath="{.data.rabbitmq-password}" | base64 -d)
export MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace "default" mysql-secret -o jsonpath="{.data.mysql-root-password}" | base64 -d)
export MYSQL_PASSWORD=$(kubectl get secret --namespace "default" mysql-secret -o jsonpath="{.data.mysql-password}" | base64 -d)

helm diff upgrade diracx-demo ./diracx --values .demo/values.yaml --set rabbitmq.auth.password=$RABBITMQ_PASSWORD --set mysql.auth.rootPassword=$MYSQL_ROOT_PASSWORD --set mysql.auth.password=$MYSQL_PASSWORD

# Actually run "helm upgrade" to apply changes
helm upgrade diracx-demo ./diracx --values .demo/values.yaml
```
Expand Down Expand Up @@ -67,6 +76,16 @@ diracx:

TODO

## Workflow

4 types of installations:
* demo/dev: we install everything and configure everything with pre-configured values
* prod: you already have a DIRAC installation with it's own DBs and everything, so you want to create a cluster, but bridge on existing external resources (like DBs)
* New: you start from absolutely nothing (no DIRAC), and you want to install all the dependencies
* New without dependencies: you start with nothing, but you want to use externally managed resources (like DB provided by your IT service)

Depending on the installation you perform, some tasks may be necessary or not. The bottom line is that to simplify the various cases, we want to be able to always run the initialization steps (like DB initialization, or CS initialization) but they should be adiabatic and non destructive.

## Requirements

| Repository | Name | Version |
Expand All @@ -84,25 +103,21 @@ TODO
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| affinity | object | `{}` | |
| autoscaling.enabled | bool | `false` | |
| autoscaling.maxReplicas | int | `100` | |
| autoscaling.minReplicas | int | `1` | |
| autoscaling.targetCPUUtilizationPercentage | int | `80` | |
| cert-manager-issuer.enabled | bool | `true` | |
| cert-manager.enabled | bool | `true` | |
| cert-manager.installCRDs | bool | `true` | |
| developer.autoReload | bool | `true` | |
| developer.editableMountedPythonModules | bool | `true` | |
| developer.enableCoverage | bool | `false` | |
| developer.autoReload | bool | `true` | Enable automatic reloading inside uvicorn when the sources change Used by the integration tests for running closer to prod setup |
| developer.editableMountedPythonModules | bool | `true` | Use pip install -e for mountedPythonModulesToInstall This is used by the integration tests because editable install might behave differently |
| developer.enableCoverage | bool | `false` | Enable collection of coverage reports (intended for CI usage only) |
| developer.enabled | bool | `true` | |
| developer.ipAlias | string | `nil` | |
| developer.localCSPath | string | `"/local_cs_store"` | |
| developer.mountedPythonModulesToInstall | list | `[]` | |
| developer.nodeImage | string | `"node:16-alpine"` | |
| developer.nodeModuleToInstall | string | `nil` | |
| developer.offline | bool | `false` | |
| developer.sourcePath | string | `"/diracx_source"` | |
| developer.urls | object | `{}` | |
| developer.ipAlias | string | `nil` | The IP that the demo is running at |
| developer.localCSPath | string | `"/local_cs_store"` | If set, mount the CS stored localy instead of initializing a default one |
| developer.mountedPythonModulesToInstall | list | `[]` | List of packages which are mounted into developer.sourcePath and should be installed with pip install SOURCEPATH/... |
| developer.nodeImage | string | `"node:16-alpine"` | Image to use for the webapp if nodeModuleToInstall is set |
| developer.nodeModuleToInstall | string | `nil` | List of node modules to install |
| developer.offline | bool | `false` | Make it possible to launch the demo without having an internet connection |
| developer.sourcePath | string | `"/diracx_source"` | Path from which to mount source of DIRACX |
| developer.urls | object | `{}` | URLs which can be used to access various components of the demo (diracx, minio, dex, etc). They are used by the diracx tests |
| dex."https.enabled" | bool | `false` | |
| dex.config.enablePasswordDB | bool | `true` | |
| dex.config.expiry.authRequests | string | `"24h"` | |
Expand All @@ -121,28 +136,24 @@ TODO
| dex.config.storage.type | string | `"sqlite3"` | |
| dex.config.web.http | int | `8000` | |
| dex.enabled | bool | `true` | |
| dex.image.tag | string | `"v2.37.0"` | |
| dex.ingress.enabled | bool | `false` | |
| dex.service.ports.http.nodePort | int | `32002` | |
| dex.service.ports.http.port | int | `8000` | |
| dex.service.type | string | `"NodePort"` | |
| diracx.manageOSIndices | bool | `true` | |
| diracx.mysqlDatabases[0] | string | `"AuthDB"` | |
| diracx.mysqlDatabases[1] | string | `"JobDB"` | |
| diracx.mysqlDatabases[2] | string | `"JobLoggingDB"` | |
| diracx.mysqlDatabases[3] | string | `"SandboxMetadataDB"` | |
| diracx.mysqlDatabases[4] | string | `"TaskQueueDB"` | |
| diracx.osDatabases[0] | string | `"JobParametersDB"` | |
| diracx.pythonModulesToInstall | list | `[]` | |
| diracx.hostname | string | `""` | Required: The hostname where the webapp/API is running |
| diracx.osDbs.dbs | string | `nil` | Which DiracX OpenSearch DBs are used? |
| diracx.osDbs.default | string | `nil` | |
| diracx.pythonModulesToInstall | list | `[]` | List of install specifications to pass to pip before launching each container |
| diracx.service.port | int | `8000` | |
| diracx.service.type | string | `"ClusterIP"` | |
| diracx.settings.DIRACX_CONFIG_BACKEND_URL | string | `"git+file:///cs_store/initialRepo"` | |
| diracx.settings.DIRACX_SERVICE_AUTH_ALLOWED_REDIRECTS | string | `"[\"http://anything:8000/docs/oauth2-redirect\"]"` | |
| diracx.settings.DIRACX_SERVICE_AUTH_TOKEN_KEY | string | `"file:///signing-key/rsa256.key"` | |
| diracx.settings | object | "e.g. DIRACX_CONFIG_BACKEND_URL=..." | Settings to inject into the API container via environment variables |
| diracx.settings.DIRACX_CONFIG_BACKEND_URL | string | `"git+file:///cs_store/initialRepo"` | This corresponds to the basic dirac.cfg which must be present on all the servers TODO: autogenerate all of these |
| diracx.sqlDbs.dbs | string | `nil` | Which DiracX MySQL DBs are used? |
| diracx.sqlDbs.default | string | `nil` | |
| diracxWeb.service.port | int | `8080` | |
| diracxWeb.service.type | string | `"ClusterIP"` | |
| fullnameOverride | string | `""` | |
| global.activeDeadlineSeconds | int | `900` | |
| global.batchJobTTL | int | `600` | |
| global.activeDeadlineSeconds | int | `900` | timeout for job deadlines |
| global.batchJobTTL | int | `600` | How long should batch jobs be retained after completing? |
| global.imagePullPolicy | string | `"Always"` | |
| global.images.client | string | `"ghcr.io/diracgrid/diracx/client"` | |
| global.images.services | string | `"ghcr.io/diracgrid/diracx/services"` | |
Expand All @@ -153,7 +164,6 @@ TODO
| ingress.className | string | `"nginx"` | |
| ingress.enabled | bool | `true` | |
| ingress.tlsSecretName | string | `"myingress-cert"` | |
| init-cs.defaultUsers | list | `[]` | |
| init-cs.enabled | bool | `true` | |
| init-secrets.enabled | bool | `true` | |
| init-secrets.rbac.create | bool | `true` | |
Expand All @@ -162,6 +172,7 @@ TODO
| init-secrets.serviceAccount.name | string | `nil` | |
| init-sql.enabled | bool | `true` | |
| init-sql.env | object | `{}` | |
| initOs.enabled | bool | `true` | |
| minio.consoleIngress.enabled | bool | `false` | |
| minio.consoleService.type | string | `"NodePort"` | |
| minio.enabled | bool | `true` | |
Expand All @@ -181,7 +192,7 @@ TODO
| mysql.initdbScriptsConfigMap | string | `"mysql-init-diracx-dbs"` | |
| nameOverride | string | `""` | type=kubernetes.io/dockerconfigjson imagePullSecrets: - name: regcred |
| nodeSelector | object | `{}` | |
| opensearch.config."opensearch.yml" | string | `"cluster.name: opensearch-cluster\n\n# Bind to all interfaces because we don't know what IP address Docker will assign to us.\nnetwork.host: 0.0.0.0\n\n# Setting network.host to a non-loopback address enables the annoying bootstrap checks. \"Single-node\" mode disables them again.\n# Implicitly done if \".singleNode\" is set to \"true\".\n# discovery.type: single-node\n\n# Start OpenSearch Security Demo Configuration\n# WARNING: revise all the lines below before you go into production\nplugins:\n security:\n ssl:\n transport:\n pemcert_filepath: esnode.pem\n pemkey_filepath: esnode-key.pem\n pemtrustedcas_filepath: root-ca.pem\n enforce_hostname_verification: false\n http:\n enabled: true\n pemcert_filepath: esnode.pem\n pemkey_filepath: esnode-key.pem\n pemtrustedcas_filepath: root-ca.pem\n allow_unsafe_democertificates: true\n allow_default_init_securityindex: true\n authcz:\n admin_dn:\n - CN=kirk,OU=client,O=client,L=test,C=de\n audit.type: internal_opensearch\n enable_snapshot_restore_privilege: true\n check_snapshot_restore_write_privileges: true\n restapi:\n roles_enabled: [\"all_access\", \"security_rest_api_access\"]\n system_indices:\n enabled: true\n indices:\n [\n \".opendistro-alerting-config\",\n \".opendistro-alerting-alert*\",\n \".opendistro-anomaly-results*\",\n \".opendistro-anomaly-detector*\",\n \".opendistro-anomaly-checkpoints\",\n \".opendistro-anomaly-detection-state\",\n \".opendistro-reports-*\",\n \".opendistro-notifications-*\",\n \".opendistro-notebooks\",\n \".opendistro-asynchronous-search-response*\",\n ]\n######## End OpenSearch Security Demo Configuration ########\ncluster:\n routing:\n allocation:\n disk:\n threshold_enabled: \"true\"\n watermark:\n flood_stage: 200mb\n low: 500mb\n high: 300mb\n"` | |
| opensearch.config | object | `{}` | |
| opensearch.enabled | bool | `true` | |
| opensearch.opensearchJavaOpts | string | `"-Xms256m -Xmx256m"` | |
| opensearch.resources.requests.cpu | string | `"100m"` | |
Expand All @@ -197,9 +208,9 @@ TODO
| replicaCount | int | `1` | |
| resources | object | `{}` | |
| securityContext | object | `{}` | |
| serviceAccount.annotations | object | `{}` | |
| serviceAccount.create | bool | `true` | |
| serviceAccount.name | string | `""` | |
| serviceAccount.annotations | object | `{}` | Annotations to add to the service account |
| serviceAccount.create | bool | `true` | Specifies whether a service account should be created |
| serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template |
| tolerations | list | `[]` | |

----------------------------------------------
Expand Down
27 changes: 26 additions & 1 deletion README.md.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,25 @@ When running the demo some an `values.yaml` file is created as `.demo/values.yam
This file can be used with helm to interact with the running demo to make changes to what is running in the cluster.

```bash


# Show what will be changed by running "helm upgrade"
helm diff upgrade diracx-demo ./diracx --values .demo/values.yaml
# Note: helm diff has a ``bug`` that requires you to specify the existing password
# https://github.com/databus23/helm-diff/issues/460

export RABBITMQ_PASSWORD=$(kubectl get secret --namespace "default" rabbitmq-secret -o jsonpath="{.data.rabbitmq-password}" | base64 -d)
export MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace "default" mysql-secret -o jsonpath="{.data.mysql-root-password}" | base64 -d)
export MYSQL_PASSWORD=$(kubectl get secret --namespace "default" mysql-secret -o jsonpath="{.data.mysql-password}" | base64 -d)

helm diff upgrade diracx-demo ./diracx --values .demo/values.yaml --set rabbitmq.auth.password=$RABBITMQ_PASSWORD --set mysql.auth.rootPassword=$MYSQL_ROOT_PASSWORD --set mysql.auth.password=$MYSQL_PASSWORD

# Actually run "helm upgrade" to apply changes
helm upgrade diracx-demo ./diracx --values .demo/values.yaml
```




## Deploying a custom branch to DIRAC certification

Apply the following on top of the standard `values.yaml` file, replacing `USERNAME` and `BRANCH_NAME` with the appropriate values.
Expand All @@ -64,6 +77,18 @@ diracx:

TODO


## Workflow

4 types of installations:
* demo/dev: we install everything and configure everything with pre-configured values
* prod: you already have a DIRAC installation with it's own DBs and everything, so you want to create a cluster, but bridge on existing external resources (like DBs)
* New: you start from absolutely nothing (no DIRAC), and you want to install all the dependencies
* New without dependencies: you start with nothing, but you want to use externally managed resources (like DB provided by your IT service)

Depending on the installation you perform, some tasks may be necessary or not. The bottom line is that to simplify the various cases, we want to be able to always run the initialization steps (like DB initialization, or CS initialization) but they should be adiabatic and non destructive.


{{ template "chart.requirementsSection" . }}

{{ template "chart.valuesSection" . }}
Expand Down
10 changes: 10 additions & 0 deletions demo/values.tpl.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@ diracx:
DIRACX_SANDBOX_STORE_BUCKET_NAME: demo-sandboxes
DIRACX_SANDBOX_STORE_S3_CLIENT_KWARGS: '{"endpoint_url": "http://{{ hostname }}:32000", "aws_access_key_id": "console", "aws_secret_access_key": "console123"}'
DIRACX_SANDBOX_STORE_AUTO_CREATE_BUCKET: "true"
sqlDbs:
dbs:
AuthDB:
JobDB:
JobLoggingDB:
SandboxMetadataDB:
TaskQueueDB:
osDbs:
dbs:
JobParametersDB:

minio:
environment:
Expand Down
25 changes: 25 additions & 0 deletions diracx/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/name: {{ include "diracx.name" . }}-web
app.kubernetes.io/instance: {{ .Release.Name }}-web
{{- end }}
{{- define "diracxCli.selectorLabels" -}}
app.kubernetes.io/name: {{ include "diracx.name" . }}-cli
app.kubernetes.io/instance: {{ .Release.Name }}-cli
{{- end }}

{{/*
Create the name of the service account to use
Expand Down Expand Up @@ -144,3 +148,24 @@ reduce collisions.
{{- $rand := randAlphaNum 3 | lower }}
{{- printf "%s-%d-%s" $name .Release.Revision $rand | trunc 63 | trimSuffix "-" -}}
{{- end -}}


{{/*
Return the fullname template for the init-os job.
*/}}
{{- define "init-os.fullname" -}}
{{- printf "%s-init-os" .Release.Name -}}
{{- end -}}

{{/*
Create a default fully qualified job name for init-os.
Due to the job only being allowed to run once, we add the chart revision so helm
upgrades don't cause errors trying to create the already ran job.
Due to the helm delete not cleaning up these jobs, we add a random value to
reduce collisions.
*/}}
{{- define "init-os.jobname" -}}
{{- $name := include "init-os.fullname" . | trunc 55 | trimSuffix "-" -}}
{{- $rand := randAlphaNum 3 | lower }}
{{- printf "%s-%d-%s" $name .Release.Revision $rand | trunc 63 | trimSuffix "-" -}}
{{- end -}}
Loading