diff --git a/docs/content-development/guide.md b/docs/content-development/guide.md index f04b4d0..04b7dc3 100644 --- a/docs/content-development/guide.md +++ b/docs/content-development/guide.md @@ -18,7 +18,7 @@ To develop and test the created content, you will need to have a Capact environm * [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [Capact CLI](../cli/getting-started.mdx) -* [populator](https://github.com/capactio/capact/tree/main/cmd/populator/docs/populator_register-ocf-manifests.md) - For now, you need to compile it from source +* [Populator](https://github.com/capactio/capact/blob/main/cmd/populator/docs/populator.md) - download the binary from the [latest Capact release](https://github.com/capactio/capact/releases/latest) Also, clone the repository with the Capact manifests: @@ -56,6 +56,8 @@ Let's try to create manifests required to define a capability to install [Matter mattermost.install(mattermost.install-input) -> mattermost.config ``` +> **NOTE:** To simplify and speed up the process of creating the manifests, you can use Capact Manifest Generator in the Capact CLI. You can read more about it in [this document](generating.md). + ### Create the Interface Group manifest First, we need to create an **InterfaceGroup** manifest, which groups **Interfaces** corresponding to some application. @@ -2003,7 +2005,8 @@ Let's take a look on the **Implementation** YAML. **Implementation** has the fol | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `appVersion` | Application versions, which this **Implementation** supports. | | `outputTypeInstanceRelations` | Specifies all output TypeInstances to upload to Hub with theirs relationships between them. Only the TypeInstances created in this Implementation have to be mentioned here. If a TypeInstances in created in another action and brought into the context with `capact-outputTypeInstances`, then it should not be defined here. | -| `additionalInput` | Additional input for the **Implementation**, compared to the **Interface**. In our case, here we define the `postgresql.config`, as our **Implementation** uses a PostgreSQL instance for Mattermost. The additional parameter `helm.install-input` is used to specify parameters for Helm Chart used by this implementation. | +| `additionalInput` | Additional input for the **Implementation**, compared to the **Interface**. In our case, here we define the `postgresql.config`, as our **Implementation** uses a PostgreSQL instance for Mattermost. The additional parameter `helm.install-input` is used to specify optional overrides for Helm Chart values used by this implementation. | +| `additionalOutput` | This section defines any additional **TypeInstances**, which are created in the **Implementation**, compared to the **Interface**. We don't make use of that in our example. | | `implements` | Defines which **Interfaces** are implemented by this **Implementation**. | | `requires` | List of system prerequisites that need to be present in the environment managed by Capact to use this **Implementation**. In our example, we will deploy Mattermost as a Helm chart on Kubernetes, which means we need a Kubernetes cluster. Requirement items can specify `alias` and be used inside workflow under `{{workflow.outputs.artifacts.{alias}}}`, where `{alias-name}` is the alias. A TypeInstance with alias is injected into the workflow based on Policy configuration. To learn more, see the [TypeInstance Injection](../feature/policies/overview.md#typeinstance-injection) paragraph in Policy Configuration document. | | `imports` | Here we define all other **Interfaces**, we use in our **Implementation**. We can then refer to them as `'.'`. | @@ -2018,8 +2021,8 @@ The workflow syntax is based on [Argo Workflows](https://argoproj.github.io/work | `.templates.steps[][].capact-when` | Allows for conditional execution of a step, based on an expression with an input workflow artifacts arguments. You can make assertions on artifacts defined under `inputs.arguments.artifacts` for a given template. It supports the syntax defined here: [antonmedv/expr](https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md). | | `.templates.steps[][].capact-action` | Allows to import another **Interface**. In our example, we use this to provision PostgreSQL with `postgresql.install` **Interface**. | | `.templates.steps[][].capact-policy` | Allows defining Workflow step policy. | | -| `.templates.steps[][].capact-outputTypeInstance` | A list of **TypeInstances**, from the called action, which are brought into the context of this **Implementations**. The `from` property must match the name of the output from the called Action. You can then use it in the Implementations `outputTypeInstanceRelations`, when defining relations between TypeInstances. The `backend` property you can choose where to store the data of TypeInstance. If it is not provided, then the default storage backend is used. | -| `.templates.steps[][].capact-updateTypeInstance` | A list of **TypeInstances**, from the called action, which are brought into the context of this **Implementations** and will be used to update existing TypeInstance. The `from` property must match the name of the output from the called Action. The `backend` property you can choose where to store the data of TypeInstance. If it is not provided, then the default storage backend is used. | +| `.templates.steps[][].capact-outputTypeInstance` | A list of **TypeInstances**, from the called action, which are brought into the context of this **Implementations**. The `from` property must match the name of the output from the called Action. You can then use it in the Implementations `outputTypeInstanceRelations`, when defining relations between TypeInstances. The optional `backend` property specifies where to store the TypeInstance data. Read more about storage backend in the [using custom backend storage](#using-custom-backend-storage). | +| `.templates.steps[][].capact-updateTypeInstance` | A list of **TypeInstances**, from the called action, which are brought into the context of this **Implementations** and will be used to update existing TypeInstance. The `from` property must match the name of the output from the called Action. The optional `backend` property specifies where to store the TypeInstance data. Read more about storage backend in the [using custom backend storage](#using-custom-backend-storage). | Let's go through the **Implementation** and try to understand, what is happening in each step of the action. Our Mattermost installation uses a PostgreSQL database. We defined an additional input `postgresql` of type `cap.type.database.postgresql.config`. Additional inputs are optional, so we need to handle the scenario, where no **TypeInstance** for `postgresql` was provided. The first workflow step `install-db` is conditionally using the `postgresql.install` **Interface** to create an PostgreSQL instance. @@ -2041,14 +2044,12 @@ Let's go through the **Implementation** and try to understand, what is happening > ``` > You can read more about policies on the [Policy overview](../feature/policies/overview.md) page. -> In this example, the default backend storage is used. If you would like to use for instance AWS Secrets Manager, you can read how to do this [here](../feature/storage-backends/aws-secrets-manager.md#use-the-storage-backend). - -In the next step we are creating a database for the Mattermost server. If you look at the **Interface** definition of [`cap.interface.database.postgresql.create-db`](https://github.com/capactio/hub-manifests/tree/main/manifests/interface/database/postgresql/create-db.yaml), you will see, that it requires a `postgresql` **TypeInstance** of **Type** [`cap.type.database.postgresql.config`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/config.yaml) and input parameters [`cap.type.database.postgresql.database-input`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/database-input.yaml), and outputs a `database` **TypeInstance** of **Type** [`cap.type.database.postgresql.database`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/database.yaml). In the step, we are providing the inputs to the **Interface** via the `.arguments.artifacts` field. We also have to map the output of this step to our output definitions in `additionalOutput` and the implemented **Interface** in the `capact-outputTypeInstances` field. + In the next step we are creating a database for the Mattermost server. If you look at the **Interface** definition of [`cap.interface.database.postgresql.create-db`](https://github.com/capactio/hub-manifests/tree/main/manifests/interface/database/postgresql/create-db.yaml), you will see, that it requires a `postgresql` **TypeInstance** of **Type** [`cap.type.database.postgresql.config`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/config.yaml) and input parameters [`cap.type.database.postgresql.database-input`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/database-input.yaml), and outputs a `database` **TypeInstance** of **Type** [`cap.type.database.postgresql.database`](https://github.com/capactio/hub-manifests/tree/main/manifests/type/database/postgresql/database.yaml). The `render-create-db-args` renders input parameters for `postgresql.instal` **Interface**. In the `create-db` step, we are providing the inputs to the **Interface** via the `.arguments.artifacts` field. We also have to map the output of this step to our output definitions in `additionalOutput` and the implemented **Interface** in the `capact-outputTypeInstances` field. -The `render-create-db-args`, `create-helm-args` steps are used to prepare the input parameters for the `helm.install` **Interface**. Jinja template engine is used here to render the Helm runner arguments with the required data from the `postgresql` and `database` **TypeInstances**. Those steps don't create any **TypeInstances** and serve only the purpose of creating the input parameters for the Helm runner. +The `create-helm-args` step is used to prepare the input parameters for the `helm.install` **Interface**. Jinja template engine is used here to render the Helm runner arguments with the required data from the `postgresql` and `database` **TypeInstances**. This step doesn't create any **TypeInstances** and serves only the purpose of creating the input parameters for the Helm runner. You can check the schema of the Helm runner args in the [Type manifest](https://github.com/capactio/hub-manifests/blob/main/manifests/type/runner/helm/install-input.yaml). -> To create the input parameters for `helm.install` we have to use data from two artifacts. As the current `jinja.template` **Interface** consumes only a template and a single variables input, we add merger step `prepare-parameters` that merge various inputs to a single output. You can read more about merger [here](https://github.com/capactio/capact/blob/main/hack/images/merger/README.md). +> **NOTE:** To create the input parameters for `helm.install` we have to use data from two artifacts. As the `jinja.template` **Interface** consumes a Jinja2 template and a single variable input, we introduced merger container. that merge multiple inputs to a single artifact. It is used in the `prepare-parameters` step. You can read more about merger [here](https://github.com/capactio/capact/blob/main/hack/images/merger/README.md). The last step launches the Helm runner, deploys the Mattermost server and creates the `mattermost-config` and `mattermost-helm-release` **TypeInstances**. The `mattermost-config` **TypeInstance** data was provided by the Helm runner in the `additional` output artifacts from this step. Check the Helm runner documentation, on how the `additional` output is created. @@ -2061,17 +2062,73 @@ arguments: ``` To verify, if a runner needs the context, check the **Interface** of the runner (e.g. [Interface for Helm runner](https://github.com/capactio/hub-manifests/blob/main/manifests/interface/runner/helm/install.yaml)). +## Using custom storage backend + +In the Mattermost installation example, we used the default storage backend but you can use a custom backend. The list of available storage backends is [here](../feature/storage-backends/introduction.md#available-storage-backends). + +In order to add storage backend as a prerequisite in the **Implementation**, you can use the `requires` section. + +```yaml +requires: + cap.type.aws.secrets-manager: + allOf: + - typeRef: + path: storage + revision: 0.1.0 + alias: aws-storage +``` + +The **Implementation** which use that section cannot be run unless the `cap.type.aws.secrets-manager.storage` is installed, where the `aws-storage` is an alias for that backend storage. + +Then, you can store the TypeInstance data using the `backend` property in the `capact-outputTypeInstance` or `capact-updateTypeInstances`: + +```yaml +capact-outputTypeInstances: + - name: example-artifact + from: example-artifact + backend: aws-storage +``` + + +You can read more about the storage backends feature [here](../feature/storage-backends/introduction.md). + +## Look into the artifact content + +Defining the artifact, you can have multiple options of how you want to store the data. Let's consider them: + +1. To store a given value on default storage backend or backend without any required additional parameters, which also accepts TypeInstance value: + + ```yaml + value: foo + ``` + +1. To save a specific value with additional parameters in a given storage backend: + + ```yaml + value: foo + backend: + context: + key: bar + value: baz + ``` + + The `context` is backend-specific properties that allow the backend to obtain additional information about the saved value. + +> **NOTE:** The syntax has been changed since the backend storage support. Before, the whole artifact content was treated as a value. + ## Validate the manifests using Capact CLI -You can use the Capact CLI to validate the manifests you created. The `capact manifests validate` command checks the manifests against JSON schemas and can tell you, if your manifests are correct. +You can use the Capact CLI to validate the manifests you created. The `capact manifest validate` command checks the manifests against JSON schemas and can tell you if your manifests are valid. > For now the Capact CLI does not verify the content of the `action` property in **Implementations**. It will not verify, that your workflow is correct and will execute properly. To verify all your manifests in `manifests` directory, execute: ``` -capact manifests validate -r manifests +capact manifest validate -r manifests ``` +You can use add an optional flag `--server-side` flag which will execute additional manifests checks against Capact Hub. As this flag requires connection to the Hub (Gateway) server, ensure that you are [logged in](../cli/commands/capact_login.md). + You can read more about the Capact CLI [here](https://github.com/capactio/capact/tree/main/cmd/cli/README.md). ## Populate the manifests into Hub @@ -2141,7 +2198,7 @@ Use the Capact CLI to run your Action. capact action get mattermost-install -ojson | yq e '.Actions[0].output.typeInstances' - ``` -In the output, there are visible the ids of created TypeInstances and their [storage backend](../feature/storage-backends/introduction.md). +Every output TypeInstance contains the ID of the [storage backend](../feature/storage-backends/introduction.md) used to store its value. You can query the storage backend details with the command `capact typeinstance get {id} -oyaml`. ### View the Action workflow in Argo UI