Skip to content

Commit

Permalink
Starting to play with URLs in e2e tests. (#75)
Browse files Browse the repository at this point in the history
* Starting to play with URLs in e2e tests.

Signed-off-by: Ville Aikas <[email protected]>

* bump hoping to kick the actions into gear.

Signed-off-by: Ville Aikas <[email protected]>

* moar

Signed-off-by: Ville Aikas <[email protected]>

* Retrying push due to git outage.

Signed-off-by: Ville Aikas <[email protected]>

* Add a simple knative service for fetch oidc tokens off the cluster
for testing external access with fulcio as well.

Signed-off-by: Ville Aikas <[email protected]>

* Test with k8s version: 1.21, 1.22, and 1.23

Signed-off-by: Ville Aikas <[email protected]>

* install cosign by checking out the main for it and go install.

Signed-off-by: Ville Aikas <[email protected]>

* Bump job retry count to 12.

Signed-off-by: Ville Aikas <[email protected]>

* Put it in the wrong place :)

Signed-off-by: Ville Aikas <[email protected]>

* Comment out 1.23 for now due to wierd job backoff behaviour.

Signed-off-by: Ville Aikas <[email protected]>
  • Loading branch information
vaikas authored Mar 28, 2022
1 parent ea8a7ef commit ec1f975
Show file tree
Hide file tree
Showing 14 changed files with 212 additions and 153 deletions.
115 changes: 79 additions & 36 deletions .github/workflows/fulcio-rekor-kind.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ on:
pull_request:
branches: [ main ]

permissions: read-all

defaults:
run:
shell: bash
working-directory: ./src/github.com/vaikas/sigstore-scaffolding
working-directory: ./src/github.com/sigstore/scaffolding

concurrency:
group: fulcio-rekor-kind-${{ github.head_ref }}
Expand All @@ -17,11 +19,18 @@ jobs:
fulcio-rekor-ctlog-tests:
name: e2e tests
runs-on: ubuntu-latest

strategy:
fail-fast: false # Keep running if one leg fails.
matrix:
k8s-version:
- v1.21.x
- v1.22.x
# 1.23 has wierd behaviour with job backoffs, there's not exponential
# backoff. If you set it to say, 12, they all completed in about 30s.
# Once that gets sorted we can test with it. I'll dig for an upstream
# issue.
#- v1.23.x

leg:
- fulcio rekor ctlog e2e
Expand All @@ -33,6 +42,7 @@ jobs:
GOFLAGS: -ldflags=-s -ldflags=-w
KO_DOCKER_REPO: registry.local:5000/knative
KOCACHE: ~/ko
COSIGN_EXPERIMENTAL: true

steps:
- name: Configure DockerHub mirror
Expand Down Expand Up @@ -70,19 +80,46 @@ jobs:
- name: Check out our repo
uses: actions/checkout@v3
with:
path: ./src/github.com/vaikas/sigstore-scaffolding
path: ./src/github.com/sigstore/scaffolding

- name: Check out cosign repo so we get the head of the main cosign
uses: actions/checkout@v3
with:
repository: sigstore/cosign
path: ./src/github.com/sigstore/cosign

- name: Install cosign
working-directory: ./src/github.com/sigstore/cosign
run: |
go install ./cmd/cosign
- name: Setup Cluster
working-directory: ./src/github.com/vaikas/sigstore-scaffolding
working-directory: ./src/github.com/sigstore/scaffolding
run: |
./hack/setup-kind.sh \
--registry-url $(echo ${KO_DOCKER_REPO} | cut -d'/' -f 1) \
--cluster-suffix cluster.local \
--k8s-version ${{ matrix.k8s-version }} \
--knative-version ${KNATIVE_VERSION}
- name: Create sample image
run: |
pushd $(mktemp -d)
go mod init example.com/demo
cat <<EOF > main.go
package main
import "fmt"
func main() {
fmt.Println("hello world")
}
EOF
demoimage=`ko publish -B example.com/demo`
echo "demoimage=$demoimage" >> $GITHUB_ENV
echo Created image $demoimage
popd
- name: Install all the everythings
working-directory: ./src/github.com//vaikas/sigstore-scaffolding
working-directory: ./src/github.com/sigstore/scaffolding
timeout-minutes: 20
run: |
ko apply -BRf ./config/
Expand All @@ -102,8 +139,8 @@ jobs:
kubectl wait --timeout 10m -n ${ns} --for=condition=Ready pod --all
done
- name: Run signing job
working-directory: ./src/github.com/vaikas/sigstore-scaffolding
- name: Run signing job in k8s using kubernetes tokens in the cluster
working-directory: ./src/github.com/sigstore/scaffolding
run: |
# Grab the secret from the ctlog-system namespace and make a copy
# in our namespace so we can get access to the CT Log public key
Expand All @@ -115,14 +152,14 @@ jobs:
kubectl wait --for=condition=Complete --timeout=90s job/sign-job
- name: Check that an entry was created in Rekor
working-directory: ./src/github.com/vaikas/sigstore-scaffolding
working-directory: ./src/github.com/sigstore/scaffolding
run: |
ko apply -f ./testdata/config/checktree
kubectl wait --for=condition=Complete --timeout=90s job/checktree
- name: Verify the image with cosign
working-directory: ./src/github.com/vaikas/sigstore-scaffolding
- name: Verify the image with cosign using kubernetes tokens in the cluster
working-directory: ./src/github.com/sigstore/scaffolding
run: |
# Grab the secret from the fulcio-system namespace and make a copy
# in our namespace so we can get access to the Fulcio public key
Expand All @@ -132,37 +169,43 @@ jobs:
kubectl wait --for=condition=Complete --timeout=180s job/verify-job
- name: Collect node diagnostics
if: ${{ failure() }}
# TODO(vaikas): There should be a fake issuer on the cluster
- name: Install a Knative service for fetch tokens off the cluster
run: |
for x in $(kubectl get nodes -oname); do
echo "::group:: describe $x"
kubectl describe $x
echo '::endgroup::'
done
ko apply -f ./testdata/config/gettoken
sleep 2
kubectl wait --for=condition=Ready --timeout=15s ksvc gettoken
- name: Collect pod diagnostics
if: ${{ failure() }}
- name: Set the endpoints on the cluster and grab secrets
run: |
for ns in fulcio-system rekor-system trillian-system ctlog-system; do
kubectl get pods -n${ns}
for x in $(kubectl get pods -n${ns} -oname); do
echo "::group:: describe $x"
kubectl describe -n${ns} $x
echo '::endgroup::'
done
done
REKOR_URL=`kubectl -n rekor-system get --no-headers ksvc rekor | cut -d ' ' -f 4`
echo "REKOR_URL=$REKOR_URL" >> $GITHUB_ENV
curl -s $REKOR_URL/api/v1/log/publicKey > ./rekor-public.pem
- name: Collect logs
if: ${{ failure() }}
FULCIO_URL=`kubectl -n fulcio-system get --no-headers ksvc fulcio | cut -d ' ' -f 4`
echo "FULCIO_URL=$FULCIO_URL" >> $GITHUB_ENV
CTLOG_URL=`kubectl -n ctlog-system get --no-headers ksvc ctlog | cut -d ' ' -f 4`
echo "CTLOG_URL=$CTLOG_URL" >> $GITHUB_ENV
ISSUER_URL=`kubectl get --no-headers ksvc gettoken | cut -d ' ' -f 4`
echo "ISSUER_URL=$ISSUER_URL" >> $GITHUB_ENV
OIDC_TOKEN=`curl -s $ISSUER_URL`
echo "OIDC_TOKEN=$OIDC_TOKEN" >> $GITHUB_ENV
kubectl -n ctlog-system get secrets ctlog-public-key -o=jsonpath='{.data.public}' | base64 -d > ./ctlog-public.pem
echo "SIGSTORE_CT_LOG_PUBLIC_KEY_FILE=./ctlog-public.pem" >> $GITHUB_ENV
kubectl -n fulcio-system get secrets fulcio-secret -ojsonpath='{.data.cert}' | base64 -d > ./fulcio-root.pem
echo "SIGSTORE_ROOT_FILE=./fulcio-root.pem" >> $GITHUB_ENV
- name: Sign with cosign from the action using k8s token
run: |
mkdir -p /tmp/logs
kind export logs /tmp/logs
cosign sign --rekor-url ${{ env.REKOR_URL }} --fulcio-url ${{ env.FULCIO_URL }} --force --allow-insecure-registry ${{ env.demoimage }} --identity-token ${{ env.OIDC_TOKEN }}
- name: Upload artifacts
- name: Verify with cosign from the action using k8s token
run: |
SIGSTORE_REKOR_PUBLIC_KEY=./rekor-public.pem SIGSTORE_TRUST_REKOR_API_PUBLIC_KEY=1 cosign verify --rekor-url ${{ env.REKOR_URL }} --allow-insecure-registry ${{ env.demoimage }}
- name: Collect diagnostics
if: ${{ failure() }}
uses: actions/upload-artifact@v2
with:
name: logs
path: /tmp/logs
uses: chainguard-dev/actions/kind-diag@84c993eaf02da1c325854fb272a4df9184bd80fc # main
11 changes: 11 additions & 0 deletions .github/workflows/test-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,17 @@ jobs:
# Wait for all the ksvc to be up.
kubectl wait --timeout 10m -A --for=condition=Ready ksvc --all
- name: Set the endpoints on the cluster
run: |
kubectl -n rekor-system get --no-headers ksvc rekor | cut -d ' ' -f 4
REKOR_URL=`kubectl -n rekor-system get --no-headers ksvc rekor | cut -d ' ' -f 4`
echo "REKOR_URL=$REKOR_URL" >> $GITHUB_ENV
- name: Run curl against Rekor
run: |
echo ${{ env.REKOR_URL }}
curl ${{ env.REKOR_URL }}/api/v1/log
- name: Run Tests
run: |
# Grab the secret from the ctlog-system namespace and make a copy
Expand Down
62 changes: 62 additions & 0 deletions cmd/getoidctoken/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
// Copyright 2022 The Sigstore Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"fmt"
"io/ioutil"
"log"
"net/http"

"github.com/kelseyhightower/envconfig"
)

type envConfig struct {
FileName string `envconfig:"OIDC_FILE" default:"/var/run/sigstore/cosign/oidc-token" required:"true"`
}

// tokenFile is where we mount the oidc token from k8s.
//const tokenFile = "/var/run/sigstore/cosign/oidc-token"
const tokenFile = "/Users/vaikas/projects/go/src/github.com/sigstore/scaffolding/ctlog-public.pem"

func tokenWriter(filename string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, req *http.Request) {
getToken(filename, w, req)
}
}
func getToken(tokenFile string, w http.ResponseWriter, req *http.Request) {
content, err := ioutil.ReadFile(tokenFile)
if err != nil {
log.Print("failed to read token file", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
_, err = fmt.Fprint(w, string(content))
if err != nil {
log.Print("failed to write token file to response", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}

func main() {
var env envConfig
if err := envconfig.Process("", &env); err != nil {
log.Fatalf("failed to process env var: %s", err)
}
http.HandleFunc("/", tokenWriter(env.FileName))
if err := http.ListenAndServe(":8080", nil); err != nil {
panic(err)
}
}
1 change: 1 addition & 0 deletions config/ctlog/certs/300-createconfig.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
name: createctconfig
namespace: ctlog-system
spec:
backoffLimit: 12
template:
spec:
serviceAccountName: createctconfig
Expand Down
5 changes: 5 additions & 0 deletions config/fulcio/fulcio/200-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ data:
"ClientID": "sigstore",
"Type": "email",
"IssuerClaim": "$.federated_claims.connector_id"
},
"https://token.actions.githubusercontent.com": {
"IssuerURL": "https://token.actions.githubusercontent.com",
"ClientID": "sigstore",
"Type": "github-workflow"
}
},
"MetaIssuers": {
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
github.com/google/uuid v1.3.0
github.com/hashicorp/golang-lru v0.5.4
github.com/hashicorp/hcl v1.0.0
github.com/kelseyhightower/envconfig v1.4.0
github.com/pkg/errors v0.9.1
github.com/sigstore/fulcio v0.1.2-0.20220110181937-d890471d8047
github.com/sigstore/rekor v0.5.0
Expand Down
1 change: 1 addition & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1118,6 +1118,7 @@ github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7V
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/karrick/godirwalk v1.8.0/go.mod h1:H5KPZjojv4lE+QYImBI8xVtrBRgYrIVsaRPx4tDPEn4=
github.com/karrick/godirwalk v1.10.3/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
github.com/kelseyhightower/envconfig v1.4.0 h1:Im6hONhd3pLkfDFsbRgu68RDNkGF1r3dvMUtDTo2cv8=
github.com/kelseyhightower/envconfig v1.4.0/go.mod h1:cccZRl6mQpaq41TPp5QxidR+Sa3axMbJDNb//FQX6Gg=
github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
Expand Down
18 changes: 9 additions & 9 deletions hack/setup-kind.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,19 +61,19 @@ done
# The version map correlated with this version of KinD
KIND_VERSION="v0.11.1"
case ${K8S_VERSION} in
v1.19.x)
K8S_VERSION="1.19.11"
KIND_IMAGE_SHA="sha256:07db187ae84b4b7de440a73886f008cf903fcf5764ba8106a9fd5243d6f32729"
v1.21.x)
K8S_VERSION="1.21.2"
KIND_IMAGE_SHA="sha256:0fda882e43d425622f045b492f8bd83c2e0b4984fc03e2e05ec101ca1a685fb7"
KIND_IMAGE="kindest/node:${K8S_VERSION}@${KIND_IMAGE_SHA}"
;;
v1.20.x)
K8S_VERSION="1.20.7"
KIND_IMAGE_SHA="sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9"
v1.22.x)
K8S_VERSION="1.22.4"
KIND_IMAGE_SHA="sha256:f240c00ffb1d82a2a2225ca0f5c85d1c45aa2b97921327cb3f6da4eee7eae5c3"
KIND_IMAGE="kindest/node:${K8S_VERSION}@${KIND_IMAGE_SHA}"
;;
v1.21.x)
K8S_VERSION="1.21.1"
KIND_IMAGE_SHA="sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6"
v1.23.x)
K8S_VERSION="1.23.1"
KIND_IMAGE_SHA="sha256:377370274d04f4976a40f0498e47ebbdec672b951cb7b8212a1897465ddd1e9a"
KIND_IMAGE="kindest/node:${K8S_VERSION}@${KIND_IMAGE_SHA}"
;;
*) echo "Unsupported version: ${K8S_VERSION}"; exit 1 ;;
Expand Down
24 changes: 24 additions & 0 deletions testdata/config/gettoken/gettoken.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: gettoken
spec:
template:
spec:
containers:
- name: gettoken
image: ko://github.com/sigstore/scaffolding/cmd/getoidctoken
env:
- name: OIDC_FILE
value: "/var/run/sigstore/cosign/oidc-token"
volumeMounts:
- name: oidc-info
mountPath: /var/run/sigstore/cosign
volumes:
- name: oidc-info
projected:
sources:
- serviceAccountToken:
path: oidc-token
expirationSeconds: 600
audience: sigstore
27 changes: 0 additions & 27 deletions third_party/VENDOR-LICENSE/LICENSE

This file was deleted.

Loading

0 comments on commit ec1f975

Please sign in to comment.