Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] Functional tests v2 #949

Merged
merged 15 commits into from
Oct 10, 2023
20 changes: 20 additions & 0 deletions .github/workflows/configs/kind-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
- |
kind: KubeletConfiguration
serverTLSBootstrap: true
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
66 changes: 66 additions & 0 deletions .github/workflows/functional_test_v2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: functional-tests


on:
pull_request:
push:
branches: [ main ]

env:
# Make sure to exit early if cache segment download times out after 2 minutes.
# We limit cache download as a whole to 5 minutes.
SEGMENT_DOWNLOAD_TIMEOUT_MINS: 2
jobs:
kubernetes-test:
env:
KUBECONFIG: /tmp/kube-config-splunk-otel-collector-chart-functional-testing
strategy:
fail-fast: false
matrix:
k8s-version:
- v1.28.0 # Support: 28 Aug 2024 - 28 Oct 2024
- v1.27.3 # Support: 28 Apr 2024 - 28 Jun 2024
- v1.26.6 # Support: 28 Dec 2023 - 28 Feb 2024
- v1.25.11 # Support: 27 Aug 2023 - 27 Oct 2023
- v1.24.15 # Support: 28 May 2023 - 28 Jul 2023
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
with:
go-version: ~1.20.8
cache: false
- name: Cache Go
id: go-cache
timeout-minutes: 5
uses: actions/cache@v3
with:
path: |
~/go/bin
~/go/pkg/mod
key: go-cache-${{ runner.os }}-${{ hashFiles('**/go.sum') }}
- name: Create kind cluster
uses: helm/[email protected]
with:
node_image: kindest/node:${{ matrix.k8s-version }}
kubectl_version: ${{ matrix.k8s-version }}
cluster_name: kind
config: ./.github/workflows/configs/kind-config.yaml
- name: Fix kubelet TLS server certificates
run: |
kubectl get csr -o=jsonpath='{range.items[?(@.spec.signerName=="kubernetes.io/kubelet-serving")]}{.metadata.name}{" "}{end}' | xargs kubectl certificate approve
- name: Update dependencies
run: |
make repo-update dep-build
- name: Deploy cert-manager
run: |
make cert-manager
- name: Build Node.js docker image
run: |
cd functional_tests/testdata/nodejs
docker build -t nodejs_test:latest .
kind load docker-image nodejs_test:latest --name kind
- name: run functional tests
run: |
cd functional_tests
go test -v
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@
helm-charts/splunk-otel-collector/Chart.lock

**/__pycache__/*
bin/
25 changes: 25 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,28 @@ chlog-preview: chlog-validate ## Provide a preview of the generated CHANGELOG.md
chlog-update: chlog-validate ## Creates an update to CHANGELOG.md for a release entry from content in .chloggen
$(CHLOGGEN) update --version "[$(VERSION)] - $$(date +'%Y-%m-%d')" || exit 1; \
ci_scripts/chloggen-update.sh || exit 1

CERTMANAGER_VERSION ?= $(shell yq eval ".dependencies[] | select(.name == \"cert-manager\") | .version" helm-charts/splunk-otel-collector/Chart.yaml)

.PHONY: cert-manager
cert-manager: cmctl
# Consider using cmctl to install the cert-manager once install command is not experimental
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/${CERTMANAGER_VERSION}/cert-manager.yaml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is cert-manager installation for the operator webhook? if so, then can we just enable the cert-manager dependency when installing helm chart?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that didn't work. I copied the cert-manager testing code from the operator project instead.

Copy link
Contributor

@jvoravong jvoravong Oct 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jina's suggestion should theoretically be plausible, I wonder if this a limitation of using kind in a GH workflow. This solution is acceptable for now.

The operator project runs tests in parallel using 1 kind cluster with the kuttl test framework which results in each test case getting a unique k8s namespace to run the test case in. Since you can only have 1 cert-manager per k8s cluster, the operator project runs 'make cert-manager' once when setting up the kind cluster before any tests are run.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really seemed like a weird ordering issue, as in the operator would be failing to get a cert and just hang there. It is probably because of the nature of the deployment, if you install cert-manager at the same time as the operator.

The good news is that we can patch this to use the cert-manager in the helm chart with a smaller delta.

$(CMCTL) check api --wait=5m

PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
CMCTL = $(shell pwd)/bin/cmctl
.PHONY: cmctl
cmctl:
@{ \
set -e ;\
if (`pwd`/bin/cmctl version | grep ${CERTMANAGER_VERSION}) > /dev/null 2>&1 ; then \
exit 0; \
fi ;\
TMP_DIR=$$(mktemp -d) ;\
curl -L -o $$TMP_DIR/cmctl.tar.gz https://github.com/jetstack/cert-manager/releases/download/$(CERTMANAGER_VERSION)/cmctl-`go env GOOS`-`go env GOARCH`.tar.gz ;\
tar xzf $$TMP_DIR/cmctl.tar.gz -C $$TMP_DIR ;\
[ -d bin ] || mkdir bin ;\
mv $$TMP_DIR/cmctl $(CMCTL) ;\
rm -rf $$TMP_DIR ;\
}
Loading