Install AWS Secrets Manager storage backend with Capact Action

[pkosiec]

Description

Changes proposed in this pull request:

• Add manifests for AWS Secrets Manager storage installation

Testing

1. Install Capact (any method you want)

2. Set manifest location to my fork

   kubectl set env deploy/capact-hub-public -n capact-system -c hub-public-populator MANIFESTS_SOURCES="github.com/pkosiec/hub-manifests?ref=secret-storage-backend"

3. Wait until the manifests are populated

4. Create AWS security credentials with SecretsManagerReadWrite policy and export the environment variables:

   export AWS_ACCESS_KEY_ID=''
   export AWS_SECRET_ACCESS_KEY=''

5. Create AWS credentials TypeInstance: https://capact.io/docs/next/example/typeinstances#aws-credentials

   Export it as TI_ID.

6. Update Global policy (once the Add common TypeInstance injection capact#646 is merged):

   cat > /tmp/aws-storage-policy.yaml << ENDOFFILE
   interface:
     rules:
         - interface:
             path: cap.*
           oneOf:
           - implementationConstraints:
               requires:
               - path: cap.core.type.platform.kubernetes
           - implementationConstraints: {}
     default:
       inject:
         requiredTypeInstances:
         - id: "${TI_ID}"
           description: "AWS credentials"
   typeInstance:
     rules: [] 
   ENDOFFILE
   capact policy apply -f /tmp/aws-storage-policy.yaml

7. Create parameters:

   cat > /tmp/aws-storage-input-params.yaml << ENDOFFILE
   input-parameters: {}
   ENDOFFILE

8. Create Action:

   capact act create cap.interface.aws.secrets-manager.storage.install --name aws-storage -n capact-system --parameters-from-file /tmp/aws-storage-input-params.yaml

9. Run Action:

   capact act run aws-storage -n capact-system

10. Watch the Action for completion:

    capact act watch aws-storage -n capact-system

    Optionally, watch the logs with:

    argo logs @latest -n capact-system -f

11. Get output TypeInstances:

    capact act get aws-storage -n capact-system -ojson | jq '.Actions[0].output.typeInstances'

    See the details of the installed AWS storage backend:

    AWS_SM_STORAGE_ID=$(capact act get aws-storage -n capact-system -ojson | jq '.Actions[0].output.typeInstances | map(select(.typeRef.path == "cap.type.aws.secrets-manager.storage"))[0].id' -r)
    capact ti get $AWS_SM_STORAGE_ID -oyaml

12. Configure it as a default backend for all TypeInstances:

    cat > /tmp/aws-storage-policy2.yaml << ENDOFFILE
    interface:
      rules:
          - interface:
              path: cap.*
            oneOf:
            - implementationConstraints:
                requires:
                - path: cap.core.type.platform.kubernetes
            - implementationConstraints: {}
    typeInstance:
       rules:
         - typeRef:
             path: "cap.*"
           backend:
             id: "${AWS_SM_STORAGE_ID}"
             description: "AWS Secrets Manager"
    ENDOFFILE
    capact policy apply -f /tmp/aws-storage-policy2.yaml

13. Create and run Action to test the new default storage:

    1. Create TypeInstances:

       cat > /tmp/download-ti.yaml << ENDOFFILE
       typeInstances:
       - alias: "download"
         typeRef:
           path: cap.type.capactio.capact.validation.download
           revision: 0.1.0
         value:
           key: "true"
       ENDOFFILE
       export DOWNLOAD_TI=$(capact ti create -f /tmp/download-ti.yaml -ojson | jq -r '.[] | select(.alias == "download") | .id')
       cat > /tmp/update-ti.yaml << ENDOFFILE
       typeInstances:
       - alias: "update"
         typeRef:
           path: cap.type.capactio.capact.validation.update
           revision: 0.1.0
         value:
           key: "true"
       ENDOFFILE
       export UPDATE_TI=$(capact ti create -f /tmp/update-ti.yaml -ojson | jq -r '.[] | select(.alias == "update") | .id')

    2. Create Action input:

       cat > /tmp/act-input-ti.yaml << ENDOFFILE
       typeInstances:
         - name: "testUpdate"
           id: ${UPDATE_TI}
         - name: "testInput"
           id: ${DOWNLOAD_TI}
       ENDOFFILE

    3. Run Action

       capact act create cap.interface.capactio.capact.validation.action.passing --name test --type-instances-from-file /tmp/act-input-ti.yaml

    4. Run capact act run test and capact act watch test

    5. Get Action output TypeInstances:

       capact act get test -ojson | jq '.Actions[0].output.typeInstances'

       Observe the Backend ID near the cap.type.capactio.capact.validation.upload TypeInstance.

    6. Use capact ti get {id} -oyaml to see the details.

       SAMPLE_TI_ID=$(capact act get test -ojson | jq '.Actions[0].output.typeInstances | map(select(.typeRef.path == "cap.type.capactio.capact.validation.upload"))[0].id' -r)
       capact ti get $SAMPLE_TI_ID -oyaml

    7. See the AWS Secrets Manager UI to double confirm the TypeInstance value has been stored externally.

    8. Clean up the secret from AWS UI (as the delete functionality is not implemented yet).

Test storage schema

After following all above steps, you can switch to the test mode of Secret Storage Backend:

# enable dotenv provider
DEPLOYMENT_NAME="secret-storage-backend-1646664197" # set proper deployment name, as it has random suffix
kubectl set env "deploy/${DEPLOYMENT_NAME}" -n capact-system -c secret-storage-backend APP_SUPPORTED_PROVIDERS="aws_secretsmanager,dotenv"

# Create TI
cat > /tmp/validation-storage-ti.yaml << ENDOFFILE
typeInstances:
- alias: "storage"
  typeRef:
    path: cap.type.capactio.capact.validation.storage
    revision: 0.1.0
  value:
    url: "${DEPLOYMENT_NAME}.capact-system:50051"
    acceptValue: true
    contextSchema: {
       "\$schema": "http://json-schema.org/draft-07/schema",
      "type": "object",
      "required": [
        "provider"
      ],
      "properties": {
        "provider": {
          "\$id": "#/properties/context/properties/provider",
          "type": "string",
          "enum": [
            "aws_secretsmanager",
            "dotenv"
          ]
        }
      },
      "additionalProperties": false
    }
ENDOFFILE
capact ti create -f /tmp/validation-storage-ti.yaml -ojson

You can use this TypeInstance for further testing (capactio/capact#634)

Related issue(s)

capactio/capact#647

[mkuziemko]

👍