-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI integration of networkservicemesh integration tests (NSMNSE-79) #23
base: vl3_latest
Are you sure you want to change the base?
Changes from all commits
25366f8
5159511
f9c29c9
b5756fa
7656f1c
3215b2d
8377e22
2f1e4f9
8e3fa80
aeef96b
e6d59f9
51afdd1
93ccd37
8239b95
9ca800f
38cd1d4
f2520e4
b8edfc1
0365ef6
2f52c5b
a355cf9
4ccfbae
95553fa
aedaf82
ddfbba0
04d69f4
d7007c4
f2eba70
01ffb5b
cbbe6b9
b92b097
a5906a0
e155706
d0ca32f
a5e53c3
9ebeca5
42e0393
4a842fa
02a03e9
5addda7
e74f800
8140068
1978eda
68931b1
59817d6
b8fded3
c188e63
10f96b8
1c68a87
815833b
bcac0b7
6912e0f
58debbe
cd31d11
8fb9d7a
8064ba6
cbb7aa4
a15c242
23b4804
3c7e2ac
6d85130
a3c03fb
34146a0
1f6abff
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
--- | ||
version: 1.0 | ||
root: "./.tests/cloud_test/bench/" | ||
timeout: 7200 # 2 hour total total timeout | ||
shuffle-enabled: false | ||
statistics: | ||
enabled: true | ||
interval: 30 # 30 seconds for statistics | ||
providers: | ||
- name: "kind" # A name of our provider | ||
kind: "shell" # A type of provider, shell indicate we need to specify all operations with shell commands. | ||
instances: 1 # We need to have 1 instance of cloud. | ||
node-count: 3 # With 3 nodes, CloudTest will validate if cloud has 3 nodes alive. | ||
retry: 1 # A retry count if cloud is failed to start tool will try again. | ||
enabled: true # Indicate Provider is active with config by default, if set to false, values could be overriden using command line arguments. | ||
timeout: 300 # 5 minutes to start cluster | ||
stop-delay: 10 | ||
env: | ||
- KIND_CLUSTER_NAME=cloudtest-kind-$(rands10) # Generate a uniq cluster name | ||
- CONFIG_LOCATION=$(tempdir)/config # Put Kubernetes configuration file here, $(tempdir) a executable variable name pointing to temporary folder. | ||
scripts: | ||
start: make kind-start | ||
config: make kind-export-kubeconfig # Use make file to generate kind config and put it into a file specified with CONFIG_LOCATION environment variable. | ||
stop: make kind-stop | ||
prepare: | | ||
make k8s-load-images | ||
make spire-install | ||
make k8s-config | ||
executions: | ||
- name: "Benchmark tests" | ||
env: | ||
- ARTIFACTS_ARCHIVE=true | ||
source: | ||
tags: | ||
- bench | ||
root: ./test/integration | ||
timeout: 600 | ||
cluster-count: 1 | ||
cluster-env: | ||
- KUBECONFIG | ||
on-fail: | | ||
make k8s-reset | ||
reporting: | ||
junit-report: "results/junit-benchmark-tests.xml" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
--- | ||
version: 1.0 | ||
root: "./.tests/cloud_test/interdomain/" | ||
timeout: 7200 # 2 hour total total timeout | ||
shuffle-enabled: false | ||
statistics: | ||
enabled: true | ||
interval: 30 # 30 seconds for statistics | ||
providers: | ||
- name: "kind" # A name of our provider | ||
kind: "shell" # A type of provider, shell indicate we need to specify all operations with shell commands. | ||
instances: 2 | ||
node-count: 3 # With 3 nodes, CloudTest will validate if cloud has 3 nodes alive. | ||
retry: 1 # A retry count if cloud is failed to start tool will try again. | ||
enabled: true # Indicate Provider is active with config by default, if set to false, values could be overriden using command line arguments. | ||
timeout: 300 # 5 minutes to start cluster | ||
stop-delay: 10 | ||
env: | ||
- KIND_CLUSTER_NAME=cloudtest-kind-$(rands10) # Generate a uniq cluster name | ||
- CONFIG_LOCATION=$(tempdir)/config # Put Kubernetes configuration file here, $(tempdir) a executable variable name pointing to temporary folder. | ||
scripts: | ||
start: make kind-start | ||
config: make kind-export-kubeconfig # Use make file to generate kind config and put it into a file specified with CONFIG_LOCATION environment variable. | ||
stop: make kind-stop | ||
prepare: | | ||
make k8s-load-images | ||
make spire-install | ||
make k8s-config | ||
executions: | ||
- name: "Interdomain tests" | ||
env: | ||
- ARTIFACTS_ARCHIVE=true | ||
- USE_JAEGER_SERVICE=true | ||
source: | ||
tags: | ||
- interdomain | ||
root: ./test/integration | ||
timeout: 600 | ||
cluster-count: 2 | ||
cluster-selector: | ||
- kind | ||
- kind | ||
cluster-env: | ||
- KUBECONFIG_CLUSTER_1 | ||
- KUBECONFIG_CLUSTER_2 | ||
on-fail: | | ||
make k8s-reset | ||
reporting: | ||
junit-report: "results/junit-interdomain-tests.xml" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
version: 1.0 | ||
root: "./.tests/cloud_test/single-cluster/" | ||
timeout: 7200 # 2 hour total total timeout | ||
shuffle-enabled: false | ||
statistics: | ||
enabled: true | ||
interval: 30 # 30 seconds for statistics | ||
providers: | ||
- name: "kind" # A name of our provider | ||
kind: "shell" # A type of provider, shell indicate we need to specify all operations with shell commands. | ||
instances: 1 # We need to have 1 instance of cloud. | ||
node-count: 3 # With 3 nodes, CloudTest will validate if cloud has 3 nodes alive. | ||
retry: 1 # A retry count if cloud is failed to start tool will try again. | ||
enabled: true # Indicate Provider is active with config by default, if set to false, values could be overriden using command line arguments. | ||
timeout: 300 # 5 minutes to start cluster | ||
stop-delay: 10 | ||
env: | ||
- KIND_CLUSTER_NAME=cloudtest-kind-$(rands10) # Generate a uniq cluster name | ||
- CONFIG_LOCATION=$(tempdir)/config # Put Kubernetes configuration file here, $(tempdir) a executable variable name pointing to temporary folder. | ||
scripts: | ||
start: make kind-start | ||
config: make kind-export-kubeconfig # Use make file to generate kind config and put it into a file specified with CONFIG_LOCATION environment variable. | ||
stop: make kind-stop | ||
prepare: | | ||
make k8s-load-images | ||
make spire-install | ||
make k8s-config | ||
executions: | ||
- name: "Single cluster tests" | ||
env: | ||
- ARTIFACTS_ARCHIVE=true | ||
source: | ||
tags: | ||
- basic | ||
- recover | ||
- usecase | ||
root: ./test/integration | ||
timeout: 300 | ||
cluster-count: 1 | ||
cluster-env: | ||
- KUBECONFIG | ||
on-fail: | | ||
make k8s-reset | ||
reporting: | ||
junit-report: "results/junit-single-cluster-tests.xml" |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,23 +2,27 @@ | |
* | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you comment on how these changes came about? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I basically rearranged them in alphabetical order, and then added some folders and files that were needed by cloudtest. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was using a dockerfile for local testing on my mac before I started using the cisco-hosted-jenkins environment. |
||
# ... except these | ||
!.cloudtest | ||
!.env | ||
!.git | ||
!.mk | ||
!applications | ||
!build | ||
!controlplane | ||
!forwarder | ||
!deployments | ||
!docker | ||
!test/applications | ||
!test/go.mod | ||
!test/go.sum | ||
!side-cars | ||
!go.mod | ||
!go.sum | ||
!k8s/api | ||
!k8s/cmd | ||
!k8s/pkg | ||
!k8s/go.mod | ||
!k8s/go.sum | ||
!forwarder | ||
!k8s | ||
!pkg | ||
!scripts | ||
!sdk | ||
!side-cars | ||
!test | ||
!utils | ||
!.cloudtest-bench.yaml | ||
!.cloudtest-single-cluster.yaml | ||
!.cloudtest-interdomain.yaml | ||
!go.mod | ||
!go.sum | ||
!Makefile | ||
!vendor | ||
!scripts/go-mod-download.sh |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
//The Jenkinsfile runs entire build in a golang container and mounts the /var/run/docker.sock file to allow access to the host docker within the container | ||
node('wcm') { | ||
def build_ok = true | ||
docker.image('golang:1.15-buster').inside('-u root --net=host -v /var/run/docker.sock:/var/run/docker.sock') { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we'll possibly have an issue with using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, but I can leave the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you could try without
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, I can give that a go and see what happens. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is what it looks like without There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally, we would have access to an ephemeral docker host, or the build agent would be ephemeral, so that we can easily guarantee that our environments are consistent. There's too many hands touching those build agents to have any sort of guarantee that they're going to be in a consistent state. |
||
environment{ | ||
CGO_ENABLED = 0 | ||
GO111MODULE = on | ||
} | ||
stage ('Install Docker Client') { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I refactored what was in nsm-nse to use the same base for our runner container we use in circleci to avoid having to install all these type of dependencies--docker client, kubectl, helm, kind, go, git, etc. I'm looking to get a more standard set of base container images curated by the SRE team that we can use as our jenkins agent setting. We don't have to change this now but it'll be something we want to follow-up on. See here: https://github.com/cisco-app-networking/nsm-nse/blob/devop_jenkins/.jenkins/Jenkinsfile#L14 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok thanks, good to know. |
||
sh "apt-get update" | ||
sh ''' | ||
apt-get install -y \ | ||
apt-transport-https \ | ||
ca-certificates \ | ||
curl \ | ||
gnupg-agent \ | ||
software-properties-common | ||
''' | ||
sh "curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -" | ||
sh ''' | ||
add-apt-repository \ | ||
"deb [arch=amd64] https://download.docker.com/linux/debian \ | ||
$(lsb_release -cs) \ | ||
stable" | ||
''' | ||
sh "apt-get update && apt-get install -y docker-ce-cli" | ||
} | ||
stage ('Install yq') { | ||
sh "wget -O /usr/bin/yq https://github.com/mikefarah/yq/releases/download/2.1.2/yq_linux_amd64" | ||
sh "chmod 555 /usr/bin/yq" | ||
} | ||
stage ('Install kubectl') { | ||
sh "curl -LO https://storage.googleapis.com/kubernetes-release/release/\"\$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)\"/bin/linux/amd64/kubectl" | ||
sh "chmod +x kubectl && mv kubectl /usr/local/bin/" | ||
} | ||
stage ('Install Helm') { | ||
sh "curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3" | ||
sh "chmod 700 get_helm.sh" | ||
sh "./get_helm.sh" | ||
} | ||
stage ('Checkout code') { | ||
checkout scm | ||
} | ||
stage ('Install kind') { | ||
sh "go get -u sigs.k8s.io/[email protected]" | ||
} | ||
stage ('Install cloudtest version 0.2.0') { | ||
sh "go get github.com/networkservicemesh/[email protected]" | ||
} | ||
|
||
try { | ||
stage ('Run single cluster tests') { | ||
sh "cloudtest --config .cloudtest-single-cluster.yaml" | ||
} | ||
} catch (e) { | ||
build_ok = false | ||
} finally { | ||
junit '.tests/cloud_test/single-cluster/results/junit-single-cluster-tests.xml' | ||
} | ||
|
||
try { | ||
stage ('Run interdomain tests') { | ||
sh "cloudtest --config .cloudtest-interdomain.yaml" | ||
} | ||
} catch (e) { | ||
build_ok = false | ||
} finally { | ||
junit '.tests/cloud_test/interdomain/results/junit-interdomain-tests.xml' | ||
} | ||
|
||
try { | ||
stage ('Run benchmark tests') { | ||
sh "cloudtest --config .cloudtest-bench.yaml" | ||
} | ||
} catch (e) { | ||
build_ok = false | ||
} finally { | ||
junit '.tests/cloud_test/bench/results/junit-benchmark-tests.xml' | ||
} | ||
|
||
stage ('Archive all test results and logs') { | ||
dir('.tests/cloud_test') { | ||
archiveArtifacts artifacts: '**', fingerprint: true, onlyIfSuccessful: false | ||
} | ||
} | ||
|
||
if (build_ok) { | ||
currentBuild.result = "SUCCESS" | ||
} else { | ||
currentBuild.result = "FAILURE" | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does the injection of the
$(rands10)
actually work? Does cloudtest process the yaml file content in a way that a bash shell would exec that?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh I see that a makefile target is wrapping usage of the env var with
$()
exec... interestingThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cloudtest does do extra processing.
https://github.com/networkservicemesh/cloudtest/blob/master/docs/README.md#environment-variables-processing
you get environment variables and a handful of other useful commands, like rands10. I'm not sure if this is executed by bash or not, they may have just borrowed the syntax.