Add support for external storages on lock, unlock, update, get, and delete

[mszostok]

Description

Changes proposed in this pull request:

• Support external storage backends on:
  • update + add option to update backend.context field
  • get
  • lock
  • unlock
  • delete
• Store data in built-in storage only if required

New scope:

• Fix: Prevent TypeInstance deletion when it's used by others #632

Testing

The test that I wrote, proves that it works e2e (Go client -> GraphQL -> Local Hub -> External Storage).

1. Run neo4j:

   docker run -d \
     -p 7687:7687 -p 7474:7474 \
     -e "NEO4J_AUTH=neo4j/okon" \
     -e "NEO4JLABS_PLUGINS=[\"apoc\"]" \
     --name hub-neo4j-instance \
     ghcr.io/capactio/neo4j:4.2.13-apoc

2. Run modified storage server:

   APP_LOGGER_LEVEL="debug" APP_LOGGER_DEV_MODE=true APP_SUPPORTED_PROVIDERS="dotenv" go run ./cmd/secret-storage-backend/main.go

3. Run Local Hub:

   cd hub-js; APP_NEO4J_ENDPOINT=bolt://localhost:7687 APP_NEO4J_PASSWORD=okon APP_HUB_MODE=local npm run dev; cd ..

4. Check that no data is stored: ls /tmp/capact

5. Run test:

   GRPC_SECRET_STORAGE_BACKEND_ADDR="0.0.0.0:50051" go test ./pkg/hub/client/local/ -v -count 1

   You should output that describe Local Hub state after each operation.

6. Check that no data is stored: ls /tmp/capactas the above test also cleans up all created TypeInstances and gRPC call is also executed against external backends.

Related issue(s)

• Implement gRPC delegation in Local Hub #645

Notes

Sometimes I got unpredictable result from dotenv:

#1: graphql: /storage_backend.StorageBackend/GetValue NOT_FOUND: TypeInstance "85e25a0e-76ef-4c26-a4ea-110873aea215" in revision 1 was not found

and in file:

$ /bin/cat /tmp/capact/85e25a0e-76ef-4c26-a4ea-110873aea215
2="{\"updated-value\":\"context 0\"}"
locked_by="demo/testing"%

I'm not sure how this can happen, as it's not possible via exposed API. Probably there is some "race" directly in used library with Put implementation. However, we decided that backends don't need to be thread safe. To resolve that issue I used the mutex lib to prevent concurrent execution of value field resolver.

[pkosiec]

While the provided test works, I wasn't able to test it manually, as I run into some problems:

When I wanted to create TI with Backend ID created during tests (I commented cleanup so I made sure there is such TI), all my mutations failed:

1. Create TI without context

query:

mutation CreateTypeInstances {
  createTypeInstances(
    in: {
     typeInstances: [
      {
        alias: "helm-release"
        typeRef: { path: "cap.type.helm.chart.release", revision: "0.1.0" }
        value: {
          key: "test" # same as it was
        }
        backend: {
          id: "1cd21f38-49f0-42d9-b2f5-c3e69e00c896" # it's like that already.
        }
      }  
      {
        alias: "helm-release2"
        typeRef: { path: "cap.type.helm.chart.release", revision: "0.1.0" }
        value: {
          key: "test" # same as it was
        }
        # backend: {
        #   id: "123" # it's like that already.
        #   context: { # new property of type `Any!`
        #     name: "release-name",
        #     namespace: "release-namespace",
        #   }
        # }
      }    
    ]
    usesRelations: [ # All the TIs exist
      {from: "helm-release", to: "helm-release2"}
      {from: "54f39b4d-280b-406f-afa9-5645328fae70", to: "helm-release2"}
      {from: "61d48278-3c5c-4eb5-8b6b-de4ce4582128", to: "helm-release2"}
    ]
  }
  ) {
    id
    alias
  }
}

logs:

2022-03-09T12:44:58.298+0100	INFO	secret-storage-backend	secret-storage-backend/server.go:314	getting whole secret	{"path": "/tmp/capact/f2863a93-ccc6-46c8-a535-d55836f348e5", "provider": "dotenv"}

output:

{
  "errors": [
    {
      "message": "/storage_backend.StorageBackend/OnDelete NOT_FOUND: path \"/tmp/capact/f2863a93-ccc6-46c8-a535-d55836f348e5\" in provider \"dotenv\" not found",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "createTypeInstances"
      ],
      "extensions": {
        "code": "INTERNAL_SERVER_ERROR",
        "exception": {
          "path": "/storage_backend.StorageBackend/OnDelete",
          "code": 5,
          "details": "path \"/tmp/capact/f2863a93-ccc6-46c8-a535-d55836f348e5\" in provider \"dotenv\" not found",
          "name": "ClientError",
          "stacktrace": [
            "ClientError: /storage_backend.StorageBackend/OnDelete NOT_FOUND: path \"/tmp/capact/f2863a93-ccc6-46c8-a535-d55836f348e5\" in provider \"dotenv\" not found",
            "    at wrapClientError (/Users/pkosiec/repositories/capact/hub-js/node_modules/nice-grpc/lib/client/wrapClientError.js:9:16)",
            "    at Object.callback (/Users/pkosiec/repositories/capact/hub-js/node_modules/nice-grpc/lib/client/createUnaryMethod.js:31:66)",
            "    at Object.onReceiveStatus (/Users/pkosiec/repositories/capact/hub-js/node_modules/@grpc/grpc-js/build/src/client.js:180:36)",
            "    at Object.onReceiveStatus (/Users/pkosiec/repositories/capact/hub-js/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:365:141)",
            "    at Object.onReceiveStatus (/Users/pkosiec/repositories/capact/hub-js/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)",
            "    at /Users/pkosiec/repositories/capact/hub-js/node_modules/@grpc/grpc-js/build/src/call-stream.js:182:78",
            "    at processTicksAndRejections (node:internal/process/task_queues:78:11)"
          ]
        }
      }
    }
  ],
  "data": null
}

I believe this is "OnDelete" because the transaction tried to do rollback, but it would be good to know the root cause.

2. Create TI with context

This returns a different error:

mutation CreateTypeInstances {
  createTypeInstances(
    in: {
     typeInstances: [
      {
        alias: "helm-release"
        typeRef: { path: "cap.type.helm.chart.release", revision: "0.1.0" }
        value: {
          key: "test" # same as it was
        }
        backend: {
          id: "1cd21f38-49f0-42d9-b2f5-c3e69e00c896"
          context: {
            provider: "mock-me" # same with dotenv
          }
        }
      }  

    ]
    usesRelations: []
  }
  ) {
    id
    alias
  }
}

result:

{
  "errors": [
    {
      "message": "Cannot convert object to primitive value",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "createTypeInstances"
      ],
      "extensions": {
        "code": "INTERNAL_SERVER_ERROR",
        "exception": {
          "stacktrace": [
            "TypeError: Cannot convert object to primitive value",
            "    at TextEncoder.encode (node:internal/encoding:333:32)",
            "    at DelegatedStorageService.encode (/Users/pkosiec/repositories/capact/hub-js/dist/local/storage/service.js:261:34)",
            "    at DelegatedStorageService.Delete (/Users/pkosiec/repositories/capact/hub-js/dist/local/storage/service.js:147:31)",
            "    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
            "    at async createTypeInstances (/Users/pkosiec/repositories/capact/hub-js/dist/local/mutation/create-type-instances.js:44:9)"
          ]
        }
      }
    }
  ],
  "data": null
}

The /tmp/capact is empty in both cases after running the mutation.

Additional data - Get query for backend TI:

query Get($typeInstanceId: ID!) {
  typeInstance(id: $typeInstanceId) {
    resourceVersions {
       spec {
         value
       }
    }
    id
    backend {
       abstract
       id
    }
    usedBy {
      id
    }
  }
}

output:

{
  "data": {
    "typeInstance": {
      "resourceVersions": [
        {
          "spec": {
            "value": {
              "acceptValue": true,
              "contextSchema": "{\n\t\"$id\": \"#/properties/contextSchema\",\n\t\"type\": \"object\",\n\t\"properties\": {\n\t\t\"provider\": {\n\t\t\t\"$id\": \"#/properties/contextSchema/properties/name\",\n\t\t\t\"type\": \"string\",\n\t\t\t\"const\": \"dotenv\"\n\t\t}\n\t},\n\t\"additionalProperties\": false\n}",
              "url": "0.0.0.0:50051"
            }
          }
        }
      ],
      "id": "1cd21f38-49f0-42d9-b2f5-c3e69e00c896",
      "backend": {
        "abstract": true,
        "id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
      },
      "usedBy": []
    }
  }
}

@mszostok could you please try by your own and see whether you also encounter such issues? Thanks!

[pkosiec]

As this is done in #634, let's make sure @mkuziemko is aware of that (that's why I'm mentioning Mati) 🙂

[pkosiec]

During manual testing I spotted two bugs. Once they are resolved, the PR is ready to be merged 🙂

[pkosiec]

LGTM, great work! 🚀