Handle multiple backends for the TypeInstance

[mkuziemko]

Description

Changes proposed in this pull request:

• Argo action in case of the artifact:

value: 
    key: test
backend:
    context:
        (...)

will create an appropriate TypeInstance object and create a mutation.

• modify Helm and Terraform runner to add value to output,
• modify Jinja with the new configuration field: strip-value: true,
• modify merger to remove value if exists in the artifacts.

Testing

Mattermost installation

1. Check out this branch.
2. Create a cluster with make dev-cluster
3. Set manifest location to my fork

kubectl set env deploy/capact-hub-public -n capact-system -c hub-public-populator MANIFESTS_SOURCES="github.com/mkuziemko/hub-manifests?ref=context_handling"

4. Wait until the manifests are populated
5. Use port-foward for capact-gateway :

kubectl port-forward svc/capact-gateway 8080:80 -n capt-system

6. Create parameters for Mattermost installation:

cat <<EOF > /tmp/mattermost-install.yaml
input-parameters:
  host: mattermost.capact.local
EOF

7. Create action

capact action create --name mattermost-install cap.interface.productivity.mattermost.install --parameters-from-file /tmp/mattermost-install.yaml

8. Use capact act run mattermost-install and capact act watch mattermost-install to observe the status of the running action.
9. When it is finished, you can delete the action using capact act delete mattermost-install and helm charts.

PostgreSQL installation

10. Prepare inputs for PostgreSQL:

cat <<EOF > /tmp/postgresql-install.yaml
input-parameters:
   superuser:
       username: "username"
       password: "password"
   defaultDBName: "default_db"
EOF

11. Create the action:

capact action create --name test-1 cap.interface.database.postgresql.install --parameters-from-file /tmp/postgresql-install.yaml

12. Run and watch the action:

capact act run test-1
capact act watch test-1

Run integration tests

Currently, the tests are failing due to a lack of storage backend integration. It will be fixed by: #655

Note: No longer valid. The PR was merged and this branch was updated. We can run tests by simple: make test-integration

1. Install secret storage backend:

helm install dotenv -n capact-system ./deploy/kubernetes/charts/secret-storage-backend --set=global.containerRegistry.path="ghcr.io/capactio/pr" --set=global.containerRegistry.overrideTag="PR-653" --set=supportedProviders={aws_secretsmanager,dotenv} --wait

2. Comment out line from e2e_suite_test.go:

waitTillServiceEndpointsAreReady()

3. Create envs for test:

export STATUS_ENDPOINTS="not-needed"
export GATEWAY_USERNAME="graphql"
export GATEWAY_PASSWORD="t0p_s3cr3t"
export GATEWAY_ENDPOINT="https://gateway.capact.local/graphql"

4. Run new scenario:

ginkgo -tags="integration" -v --focus="should propagate context provider to storage backend" ./test/e2e/...
ginkgo -tags="integration" -v --focus="should fail due to incorrect storage provider" ./test/e2e/...

Related issue(s)

• Handle multiple backends for the TypeInstance upload from workflow #634

[mkuziemko]

it is mainly for the case like here

[mkuziemko]

will be removed before merge

[mszostok]

In general, our contract is not implemented. So I didn't run the test as it shouldn't work that way.

[mszostok]

Overall it LGTM 👍 But please address my comments and additionally find in our code base all TODO(https://github.com/capactio/capact/issues/634) and fix them. Thanks!

[mszostok]

I saw the WIP label, so it's probably not for review, but I already had some comments so I posted them but I understand that you may already addressed them locally :)

Please also fix e2e tests (dotenv storage backend definition). Thanks!

[mszostok]

nit: IMO we should return an error when typeInstance.Backend.ID is not set. As setting context without backend ID means that it will be stored in built-in storage and this storage doesn't allow context by definition.

[mszostok]

For consistency reason, it would be good to also have unpackValue conf option here. When I went through our manifests, it was hard to understand why it works in this way, as this is a "hidden" functionality.

But let's do that on a new follow-up PR. It's not a blocker for merging this one 👍