feat: pushes airbyte to Glide GCP Artifact registry

The readme contains instructions for how to use this with Airbyte and how to push new versions.
There were also some fixes to our airbyte config to make the API path and host optional and they default to prod.
This is probably getting close to closing glideapps/glide#27833 once it is reviewed

airbyte-integrations/connectors/destination-glide/README.md

@@ -47,6 +47,12 @@ There are a set of simple integration tests that Airbyte provides that can be tr
 
 These simply call commands that Airbyte provides in their connector template. The dev-write one appears to be the most comprehensive, but I've struggled to get that one to consistently run (see TODO).
 
+There are additional tests you can run against the live API in `integration_tests/` that you can run with:
+
+```sh
+./scripts/test-integration.sh
+```
+
 ### Build & Deployment
 
 The Airbyte destination is packed as Docker image. This script uses Airbyte-provided tooling named `airbyte-ci` that leverages the same tooling they use in their CI pipeline to build the container.
@@ -57,7 +63,22 @@ To install the tooling see [`airbyte-ci` README in this repo](https://github.com
 ./scripts/build-docker-image.sh
 ```
 
-We are currently deploying this to a public repository for ease of access from an Airbyte OSS instance. To deploy it to a docker container registry use the script at:
+We are deploying this to Glide's docker repository in Google Artifact Registry (GAR) for ease of access from an Airbyte OSS instance. To deploy it to a docker container registry you need to authenticate docker to GAR and push it. The steps are:
+
+#### Glide Google Artifact Registry Docker Repo:
+
+Our repo is at https://console.cloud.google.com/artifacts/docker/glide-connectors/us-central1/airbyte-glide-destination?project=glide-connectors
+
+#### Pushing a Docker Image to Google Artifact Registry
+
+Read access is available for all, but to push you have to authenticate. Authenticate docker by running the following command which adds credential helper to `~/.docker/config.json`:
+
+```
+gcloud auth configure-docker \
+    us-central1-docker.pkg.dev
+```
+
+Then you can use gcloud and docker commands as normal. To push a new image version run:
 
 ```sh
 ./scripts/push-docker-image.sh
@@ -69,13 +90,48 @@ To install Airbyte follow the guide at https://docs.airbyte.com/deploying-airbyt
 
 Once install it should be available at http://localhost:8000/. You should have been prompted for username/pw during install.
 
-### Installing Glide Destination in Airbyte OSS
+### Installing Glide Destination in Airbyte OSS (on Kubernetes)
+
+Install kind (macOS):
+
+NOTE: abctl installs kind, but it doesn't install the kind CLI, so to work with the airbyte kubernetes cluster follow these steps:
+
+```sh
+brew install kind
+```
+
+Then you can list the clusters and you should see one named `airbyte-abctl`:
+
+```sh
+$ kind get clusters
+airbyte-abctl
+```
+
+To use it with kubectl (or k9s) use:
+
+```sh
+# this updates ~/.kube/config to add the cluster
+kind export kubeconfig --name airbyte-abctl
+
+# set the context to the cluster context that kind added:
+kubectl config set-context kind-airbyte-abctl
+
+# now kubectl works!
+kubectl get namespaces
+kubectl get -n airbyte-abctl pods
+```
+
+Now we follow the course guidance at https://docs.airbyte.com/operator-guides/using-custom-connectors/#for-kubernetes-airbyte-deployments ...
+
+We made our docker registry public so you don't have to authenticate airbyte's kubernetes deployment.
 
 To install the destination into Airbyte OSS follow these steps:
 
 1. Click on **Settings** on the far left then select **Destinations** in the sub-panel. You should see a list of **Available destination connectors**.
 2. At the top click the **+ New Connector** button fill in the fields. The **Docker repository name** and **Docker image tag** are the important bits.
 
+For repository name use `us-central1-docker.pkg.dev/glide-connectors/airbyte-glide-destination/destination-glide` and for tag, run `gcloud artifacts docker tags list us-central1-docker.pkg.dev/glide-connectors/airbyte-glide-destination/destination-glide` to get the available tags and choose the latest.
+
 Once installed, you can upgrade it to a new version by visiting the same settings page and changing the tag in the **Change to** box and clicking the **Change** button.
 
 ---

airbyte-integrations/connectors/destination-glide/destination_glide/spec.json

@@ -6,16 +6,16 @@
     "$schema": "http://json-schema.org/draft-07/schema#",
     "title": "Destination Glide",
     "type": "object",
-    "required": ["api_host", "api_path_root", "api_key"],
+    "required": ["api_key"],
     "additionalProperties": false,
     "properties": {
       "api_host": {
         "type": "string",
-        "description": "The protocol and host name (e.g. 'https://api.glideapp.io'). of the Glide API destination"
+        "description": "The protocol and host name of the Glide API destination. Defaults to 'https://api.glideapps.com'."
       },
       "api_path_root": {
         "type": "string",
-        "description": "The path root of the Glide API destination"
+        "description": "The path root of the Glide API destination. Defaults to empty."
       },
       "api_key": {
         "type": "string",

airbyte-integrations/connectors/destination-glide/scripts/build-docker-image.sh

@@ -2,5 +2,6 @@
 this_dir=$(cd $(dirname "$0"); pwd) # this script's directory
 this_script=$(basename $0)
 
-airbyte-ci connectors --name=destination-glide build
+# segment sometimes fails, so AIRBYTE_CI_DISABLE_TELEMETRY=true to disable it per airbyte-ci/connectors/pipelines/README.md
+AIRBYTE_CI_DISABLE_TELEMETRY=true airbyte-ci connectors --name=destination-glide build
 

airbyte-integrations/connectors/destination-glide/scripts/push-docker-image.sh

@@ -3,15 +3,22 @@ this_dir=$(cd $(dirname "$0"); pwd) # this script's directory
 this_script=$(basename $0)
 
 #!/usr/bin/env bash
+LOCAL_BUILT_IMAGE_NAME=airbyte/destination-glide:dev
+GLIDE_DOCKER_IMAGE_NAME=us-central1-docker.pkg.dev/glide-connectors/airbyte-glide-destination/destination-glide
+
+echo "Pushing to remote image repository: $GLIDE_DOCKER_IMAGE_NAME"
+docker inspect --format='Pushing the local image "{{index .RepoTags 0}}" created at "{{.Created}}"' $LOCAL_BUILT_IMAGE_NAME
+
 
 # Fetch the list of tags from Docker Hub
-tags=$(wget -q -O - "https://hub.docker.com/v2/namespaces/activescott/repositories/destination-glide/tags?page_size=10" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$' | grep -E '([0-9]+\.)+[0-9]+' )
-``
-echo "Found tags: $tags"
+#tags=$(wget -q -O - "https://hub.docker.com/v2/namespaces/activescott/repositories/destination-glide/tags?page_size=10" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$' | grep -E '([0-9]+\.)+[0-9]+' )
+# Get the list of tags using gcloud with the appropriate format
+tags=$(gcloud artifacts docker tags list $GLIDE_DOCKER_IMAGE_NAME --format="get(tag)")
+
 # Sort the tags and get the highest one
-highest_tag=$(echo "$tags" | sort -V | tail -n 1)
+highest_tag=$(echo "$tags" | awk -F'/' '{print $NF}' | sort -V | tail -n 1)
 
-echo "found highest tag: $highest_tag"
+echo "found highest tag on remote: $highest_tag"
 
 # Increment the version
 IFS='.' read -ra ADDR <<< "$highest_tag"
@@ -28,12 +35,10 @@ else
     exit 1
 fi
 
-
 # Tag the local image with the new version
-# TODO: airbyte/destination-glide is the local docker image that airbyte's CI builds it and names it as locally. Can't we change this ??
-docker image tag airbyte/destination-glide:dev activescott/destination-glide:$new_version
-
-# Push the image with the new tag
-docker push activescott/destination-glide:$new_version
+# NOTE: airbyte/destination-glide is the local docker image that airbyte's CI builds and names locally.
 
+docker image tag $LOCAL_BUILT_IMAGE_NAME $GLIDE_DOCKER_IMAGE_NAME:$new_version
 
+# Push the image with the new tag
+docker push $GLIDE_DOCKER_IMAGE_NAME:$new_version