Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NMS-13163: enable Docker Content Trust #21

Draft
wants to merge 41 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
b626212
NMS-13163: enable Docker Content Trust
swachter Apr 21, 2021
f4bb1d0
fix decoding statement
swachter Apr 21, 2021
ce94f8b
fix option name
swachter Apr 21, 2021
0c57e15
switch to opennmsdcttest repo; set build context for maven-jdk* jobs
swachter May 7, 2021
595e273
contexts for all!
swachter May 7, 2021
85513cf
include only some jobs
swachter May 7, 2021
fc81a36
change base of openjdk-11 image
swachter May 7, 2021
3fd7998
disable change detection
swachter May 7, 2021
09e4134
adapt centos image version
swachter May 7, 2021
fe7c1a8
use the same build numbers everywhere
swachter May 7, 2021
dd08d32
tag centos image for use as base image
swachter May 7, 2021
090ee82
push always
swachter May 7, 2021
242edf3
rely on common tagging logic; try other java version detail
swachter May 7, 2021
344af99
try another openjdk version
swachter May 7, 2021
853782d
adjust base image version of maven-jdk11 image
swachter May 7, 2021
65befe9
use CONTAINER_REGISTRY_REPO to address base images
swachter May 7, 2021
e9ecc3c
replace CONTAINER_REGISTRY_REPO by constants (because it is yet defined)
swachter May 7, 2021
fcc8a47
replace try with other root key
swachter May 10, 2021
f4a5f4a
use swachter repo
swachter May 10, 2021
b306c99
set registry/repo in common build.sh
swachter May 10, 2021
7a3ae45
enable dct for pushing
swachter May 11, 2021
49960eb
use BASH_EVN
swachter May 11, 2021
0e7b727
use BASH_EVN
swachter May 11, 2021
81eb479
use BASH_EVN
swachter May 11, 2021
8119204
test BASH_ENV
swachter May 11, 2021
c9b805d
test BASH_ENV
swachter May 11, 2021
d780d12
test BASH_ENV
swachter May 11, 2021
b6c90bc
test BASH_ENV
swachter May 11, 2021
c3da1b7
test BASH_ENV
swachter May 11, 2021
f53daa3
test BASH_ENV
swachter May 11, 2021
2fb78cc
test BASH_ENV
swachter May 11, 2021
35549b3
test BASH_ENV
swachter May 11, 2021
9df90f0
test BASH_ENV
swachter May 11, 2021
6efd3eb
test BASH_ENV
swachter May 11, 2021
516ead5
disable DCT for docker build
swachter May 11, 2021
310a057
disable DCT for docker build
swachter May 11, 2021
4895247
disable DCT for docker build
swachter May 11, 2021
bde135f
check tagging
swachter May 11, 2021
2952806
opennmsdcttest repo
swachter May 12, 2021
46e20d8
fix build.sh
swachter May 12, 2021
854447d
build openjdk and maven images
swachter May 12, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions .circleci/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,17 @@ found_changes() {
return 1
}

echo "Detect changes in projects"
if found_changes; then
echo "BASH_ENV: $BASH_ENV"
cat "$BASH_ENV"
env | grep CONTAINER
env | grep DOCKER

#echo "Detect changes in projects"
# if found_changes; then
./build_container_image.sh && \
~/opennms-container/.circleci/tag.sh && \
~/opennms-container/.circleci/publish.sh
else
echo "No changes detected"
exit 0
fi
#else
# echo "No changes detected"
# exit 0
#fi
196 changes: 123 additions & 73 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,30 @@ aliases:
command: ~/opennms-container/.circleci/build.sh
- &registry_login
name: Login Container Registry
command: docker login -u ${CONTAINER_REGISTRY_LOGIN} -p ${CONTAINER_REGISTRY_PASS}
command: |
docker login -u ${CONTAINER_REGISTRY_LOGIN} -p ${CONTAINER_REGISTRY_PASS}
KEY_FOLDER=~/.docker/trust/private
mkdir -p $KEY_FOLDER
echo "$DELEGATE_PRIVATE_KEY" | base64 -d > $KEY_FOLDER/$DELEGATE_PRIVATE_KEY_NAME.key
chmod 600 $KEY_FOLDER/*
env | grep DOCKER
echo "BASH_ENV: $BASH_ENV"
cat $BASH_ENV
echo "SHELL: $SHELL"
echo $0
docker trust key load $KEY_FOLDER/$DELEGATE_PRIVATE_KEY_NAME.key
- &bash_env
name: Setup Environment Variables
command: |
echo "export CONTAINER_REGISTRY=docker.io" >> $BASH_ENV
echo "export CONTAINER_REGISTRY_REPO=opennmsdcttest" >> $BASH_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CONTAINER_REGISTRY (docker.io) and CONTAINER_REGISTRY_REPO/ORGANISATION (opennmsdcttest) are now given by CircleCI config instead of the config in the repository. What I see right now are three options:

  • Container registry and organization directly in the repo in a config
  • Container registry and organization directly in the CI/CD script
  • Container registry and organization as an environment injected by a CircleCI context

Just double checking here if this is the best place to define the target publishing images with just in mind we might want to switch to a different registry structure. It's not super critical, read this here more like an informal comment not as a change request :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the registry / organization should be settable from the outside then we should rethink how we handle the "no build args in FROM" limitation of docker builds with content trust. Some base images have to be addressed by the registry / organization.

echo "export DOCKER_CONTENT_TRUST=1" >> $BASH_ENV
echo "export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=\"$DELEGATE_PRIVATE_KEY_PASSPHRASE\"" >> $BASH_ENV
- &common_steps
- checkout: *checkout
- setup_remote_docker
- run: apk add --no-cache bash
- run: *bash_env
- run: *registry_login
- attach_workspace:
at: ~/
Expand All @@ -30,6 +49,10 @@ aliases:
- &common_environment
docker: *docker_environment
steps: *common_steps
shell: /bin/sh -leo pipefail
environment:
- BASH_ENV: /etc/profile


jobs:
shellcheck:
Expand Down Expand Up @@ -156,84 +179,111 @@ workflows:
container_build:
jobs:
- shellcheck
- tomcat:
requires:
- shellcheck
# - tomcat:
# context: "build-context"
# requires:
# - shellcheck
- centos:
context: "build-context"
requires:
- shellcheck
- ubuntu:
requires:
- shellcheck
- antora:
requires:
- shellcheck
- confd:
requires:
- centos
- net-snmp:
requires:
- shellcheck
- isc-dhcp:
requires:
- shellcheck
- mini-mail:
requires:
- shellcheck
- gobgp:
requires:
- centos
- openjdk-8:
requires:
- centos
# - ubuntu:
# context: "build-context"
# requires:
# - shellcheck
# - antora:
# context: "build-context"
# requires:
# - shellcheck
# - confd:
# context: "build-context"
# requires:
# - centos
# - net-snmp:
# context: "build-context"
# requires:
# - shellcheck
# - isc-dhcp:
# context: "build-context"
# requires:
# - shellcheck
# - mini-mail:
# context: "build-context"
# requires:
# - shellcheck
# - gobgp:
# context: "build-context"
# requires:
# - centos
# - openjdk-8:
# context: "build-context"
# requires:
# - centos
- openjdk-11:
context: "build-context"
requires:
- centos
- maven-jdk8:
requires:
- openjdk-8
# - maven-jdk8:
# context: "build-context"
# requires:
# - openjdk-8
- maven-jdk11:
context: "build-context"
requires:
- openjdk-11
- netlify-cli:
requires:
- shellcheck
- ghr:
requires:
- shellcheck
- gulp:
requires:
- shellcheck
- build-env-jdk8:
requires:
- maven-jdk8
- build-env-jdk11:
requires:
- maven-jdk11
- debian:
requires:
- shellcheck
- debian-openjdk-8:
requires:
- debian
- debian-openjdk-11:
requires:
- debian
- debian-build-env-jdk8:
requires:
- debian-openjdk-8
- debian-build-env-jdk11:
requires:
- debian-openjdk-11
- snapcraft:
requires:
- shellcheck
- yum-repo:
requires:
- shellcheck
- gns3-horizon:
requires:
- shellcheck
- gns3-minion:
requires:
- shellcheck
# - netlify-cli:
# context: "build-context"
# requires:
# - shellcheck
# - ghr:
# context: "build-context"
# requires:
# - shellcheck
# - gulp:
# context: "build-context"
# requires:
# - shellcheck
# - build-env-jdk8:
# context: "build-context"
# requires:
# - maven-jdk8
# - build-env-jdk11:
# context: "build-context"
# requires:
# - maven-jdk11
# - debian:
# context: "build-context"
# requires:
# - shellcheck
# - debian-openjdk-8:
# context: "build-context"
# requires:
# - debian
# - debian-openjdk-11:
# context: "build-context"
# requires:
# - debian
# - debian-build-env-jdk8:
# context: "build-context"
# requires:
# - debian-openjdk-8
# - debian-build-env-jdk11:
# context: "build-context"
# requires:
# - debian-openjdk-11
# - snapcraft:
# context: "build-context"
# requires:
# - shellcheck
# - yum-repo:
# context: "build-context"
# requires:
# - shellcheck
# - gns3-horizon:
# context: "build-context"
# requires:
# - shellcheck
# - gns3-minion:
# context: "build-context"
# requires:
# - shellcheck
12 changes: 6 additions & 6 deletions .circleci/publish.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ source ./config.sh
# shellcheck disable=SC1091
source ../registry-config.sh

if [ "${CIRCLE_BRANCH}" == "master" ]; then
echo "Publish images for master branch ..."
#if [ "${CIRCLE_BRANCH}" == "master" ]; then
# echo "Publish images for master branch ..."
for TAG in ${CONTAINER_VERSION_TAGS[*]}; do
docker push "${CONTAINER_REGISTRY}/${CONTAINER_REGISTRY_REPO}/${CONTAINER_PROJECT}:${TAG}"
done
else
echo "Skip publishing for working branches other than master."
echo "Build images for branches are available in the CircleCI build artifacts."
fi
#else
# echo "Skip publishing for working branches other than master."
# echo "Build images for branches are available in the CircleCI build artifacts."
#fi
2 changes: 2 additions & 0 deletions .circleci/tag.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ source ./config.sh
# shellcheck disable=SC1091
source ../registry-config.sh

echo "tag for registry/repo"

for TAG in ${CONTAINER_VERSION_TAGS[*]}; do
docker tag "${CONTAINER_PROJECT}:${IMAGE_VERSION}" "${CONTAINER_REGISTRY}/${CONTAINER_REGISTRY_REPO}/${CONTAINER_PROJECT}:${TAG}"
done
Expand Down
5 changes: 2 additions & 3 deletions projects/centos/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ ARG BASE_IMAGE="centos"
ARG BASE_IMAGE_VERSION="8"
ARG VERSION=${BASE_IMAGE_VERSION}

FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}

ARG VERSION=${BASE_IMAGE_VERSION}
FROM centos:8
# FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}

RUN dnf -y --nodocs update && \
dnf clean all && \
Expand Down
2 changes: 1 addition & 1 deletion projects/centos/build_container_image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ docker build -t "${CONTAINER_PROJECT}:${IMAGE_VERSION[0]}" \
--build-arg BUILD_BRANCH="${CIRCLE_BRANCH}" \
.

docker image save "${CONTAINER_PROJECT}:${IMAGE_VERSION[0]}" -o "${CONTAINER_IMAGE}"
docker image save "${CONTAINER_PROJECT}:${IMAGE_VERSION[0]}" -o "${CONTAINER_IMAGE}"
2 changes: 2 additions & 0 deletions projects/centos/config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

# shellcheck disable=SC2034

CONTAINER_PROJECT="centos"

# Configure base image dependency
BASE_IMAGE="centos"
VERSION="8"
Expand Down
5 changes: 3 additions & 2 deletions projects/maven-jdk11/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
ARG BASE_IMAGE=opennms/openjdk
ARG BASE_IMAGE="opennms/openjdk"
ARG BASE_IMAGE_VERSION="11.0.4.11"

FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}
FROM opennmsdcttest/openjdk:11.0.11.0.9
# FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}

ARG MAVEN_VERSION="3.6.1"

Expand Down
6 changes: 3 additions & 3 deletions projects/maven-jdk11/config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
CONTAINER_PROJECT="maven"

# Base Image Dependency
BASE_IMAGE="opennms/openjdk"
JDK_VERSION="11.0.9.11"
BASE_IMAGE="$CONTAINER_REGISTRY_REPO/openjdk"
JDK_VERSION="11.0.11.0.9"
MAVEN_VERSION="3.6.3"
BASE_IMAGE_VERSION="${JDK_VERSION}-b5706"
BASE_IMAGE_VERSION="${JDK_VERSION}-b${CIRCLE_BUILD_NUM}"
BUILD_DATE="$(date -u +"%Y-%m-%dT%H:%M:%S%z")"

# inherit $BASE_IMAGE_VERSION
Expand Down
3 changes: 2 additions & 1 deletion projects/openjdk-11/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
ARG BASE_IMAGE="centos"
ARG BASE_IMAGE_VERSION="8"

FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}
FROM opennmsdcttest/centos:8
#FROM ${BASE_IMAGE}:${BASE_IMAGE_VERSION}

ARG JDK_MAJOR_VERSION=11
ARG JDK_VERSION_DETAIL=${MAJOR_VERSION}.0.4.11
Expand Down
6 changes: 3 additions & 3 deletions projects/openjdk-11/config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@
CONTAINER_PROJECT="openjdk"

# Base Image Dependency
BASE_IMAGE="opennms/centos"
BASE_IMAGE_VERSION="8-b5649"
BASE_IMAGE="$CONTAINER_REGISTRY_REPO/centos"
BASE_IMAGE_VERSION="8-b${CIRCLE_BUILD_NUM}"
BUILD_DATE="$(date -u +"%Y-%m-%dT%H:%M:%S%z")"

# inherit $BASE_IMAGE_VERSION
PARENT_PROJECT="centos"

# Specific container config
JDK_MAJOR_VERSION="11"
JDK_VERSION_DETAIL="${JDK_MAJOR_VERSION}.0.9.11"
JDK_VERSION_DETAIL="${JDK_MAJOR_VERSION}.0.11.0.9"
IMAGE_VERSION=("${JDK_VERSION_DETAIL}")

# Most specific tag when it is not build locally and in CircleCI
Expand Down
2 changes: 0 additions & 2 deletions projects/registry-config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ if [ -z "${CONTAINER_PROJECT}" ]; then
fi

# Container registry and tags
CONTAINER_REGISTRY="docker.io"
CONTAINER_REGISTRY_REPO="opennms"
CONTAINER_VERSION_TAGS=${IMAGE_VERSION[*]}

# Container Image Artifact
Expand Down