Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate docker-slim for waaaaaaaay smaller run images #33

Merged
merged 28 commits into from
Oct 26, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
23fef56
remove unused templates
lreiher Sep 4, 2024
db39c16
test-integrate slim in gitlab-ci
lreiher Sep 4, 2024
01e5af7
fix missing deps in slim ci job
lreiher Sep 4, 2024
b049458
fix curl installation in slim job
lreiher Sep 4, 2024
e861964
fix slim arguments
lreiher Sep 4, 2024
e65cd1a
dont let slim pull
lreiher Sep 4, 2024
6ec5a09
try to fix slim in container
lreiher Sep 4, 2024
562ffd8
run containerized slim as suggested for gitlab ci in readme
lreiher Sep 4, 2024
ce568da
try another way of invoking slim in gitlab ci
lreiher Sep 4, 2024
48c071f
remove slim debug flags
lreiher Sep 5, 2024
b6c861c
add slim job for arm64
lreiher Sep 5, 2024
ebecb90
add variable to control slim image name
lreiher Sep 5, 2024
6144f8e
remove irrelevant todo comments
lreiher Oct 23, 2024
d619bdd
add flag for enabling slim
lreiher Oct 23, 2024
0be5da5
add slimmed image to multiarch push stage
lreiher Oct 23, 2024
3ca384d
add env to control slim build arguments
lreiher Oct 23, 2024
6235b2a
implement slimming in github ci
lreiher Oct 23, 2024
3e53048
remove slim report artifacts
lreiher Oct 24, 2024
fa95278
try to fix slim in github action
lreiher Oct 24, 2024
66a76a5
fix inputs in github ci action
lreiher Oct 24, 2024
d5d7533
fix missing tag in slim images in github ci
lreiher Oct 24, 2024
4c78b3a
move slim in github ci to ci.sh script
lreiher Oct 24, 2024
1b33b10
disable slim in ci.sh for gitlab ci as separate stage
lreiher Oct 25, 2024
cb56641
only install slim in ci.sh if enabled
lreiher Oct 25, 2024
e546c9e
explain slim in readme
lreiher Oct 25, 2024
d8901c5
test github ci
lreiher Oct 25, 2024
13b1ab1
bump version to v1.6.0
lreiher Oct 25, 2024
3257ce0
fix github ci push of slim image
lreiher Oct 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 68 additions & 8 deletions .gitlab-ci/docker-ros.yml
lreiher marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ variables:
IMAGE_TAG: latest # Image tag of run image
DEV_IMAGE_NAME: ${IMAGE_NAME} # Image name of dev image
DEV_IMAGE_TAG: ${IMAGE_TAG}-dev # Image tag of dev image
SLIM_IMAGE_NAME: ${IMAGE_NAME} # Image name of slim run image
SLIM_IMAGE_TAG: ${IMAGE_TAG}-slim # Image tag of slim run image
lreiher marked this conversation as resolved.
Show resolved Hide resolved
BUILD_CONTEXT: . # Build context of Docker build process
REGISTRY: ${CI_REGISTRY} # Docker registry to push images to
REGISTRY_USER: ${CI_REGISTRY_USER} # Docker registry username
Expand Down Expand Up @@ -47,19 +49,26 @@ variables:

_RUN_IMAGE: ${IMAGE_NAME}:${IMAGE_TAG}
_DEV_IMAGE: ${DEV_IMAGE_NAME}:${DEV_IMAGE_TAG}
_SLIM_IMAGE: ${SLIM_IMAGE_NAME}:${SLIM_IMAGE_TAG}

_IMAGE_DEV_CI: ${_DEV_IMAGE}_${CI_COMMIT_REF_SLUG}_ci
_IMAGE_RUN_CI: ${_RUN_IMAGE}_${CI_COMMIT_REF_SLUG}_ci
_IMAGE_SLIM_CI: ${_SLIM_IMAGE}_${CI_COMMIT_REF_SLUG}_ci
_IMAGE_DEV_CI_AMD64: ${_IMAGE_DEV_CI}-amd64
_IMAGE_DEV_CI_ARM64: ${_IMAGE_DEV_CI}-arm64
_IMAGE_RUN_CI_AMD64: ${_IMAGE_RUN_CI}-amd64
_IMAGE_RUN_CI_ARM64: ${_IMAGE_RUN_CI}-arm64
_IMAGE_DEV_LATEST: ${DEV_IMAGE_NAME}:latest-dev
_IMAGE_SLIM_CI_AMD64: ${_IMAGE_SLIM_CI}-amd64
_IMAGE_SLIM_CI_ARM64: ${_IMAGE_SLIM_CI}-arm64
_IMAGE_DEV_LATEST: ${DEV_IMAGE_NAME}:latest-dev # TODO: is this a bug here that DEV_IMAGE_TAG is not used?
lreiher marked this conversation as resolved.
Show resolved Hide resolved
_IMAGE_RUN_LATEST: ${IMAGE_NAME}:latest
_IMAGE_SLIM_LATEST: ${SLIM_IMAGE_NAME}:latest-slim
_IMAGE_DEV_TARGET_TAG: ${_DEV_IMAGE}-${CI_COMMIT_TAG}
_IMAGE_RUN_TARGET_TAG: ${_RUN_IMAGE}-${CI_COMMIT_TAG}
_IMAGE_DEV_TAG: ${DEV_IMAGE_NAME}:${CI_COMMIT_TAG}-dev
_IMAGE_SLIM_TARGET_TAG: ${_SLIM_IMAGE}-${CI_COMMIT_TAG}
_IMAGE_DEV_TAG: ${DEV_IMAGE_NAME}:${CI_COMMIT_TAG}-dev # TODO: is this a bug here that DEV_IMAGE_TAG is not used?
lreiher marked this conversation as resolved.
Show resolved Hide resolved
_IMAGE_RUN_TAG: ${IMAGE_NAME}:${CI_COMMIT_TAG}
_IMAGE_SLIM_TAG: ${SLIM_IMAGE_NAME}:${CI_COMMIT_TAG}-slim

GIT_SUBMODULE_STRATEGY: recursive
DOCKER_DRIVER: overlay2
Expand All @@ -71,19 +80,20 @@ stages:
- Build dev Images
- Build run Images
- Test ROS Industrial CI
- Slim Images
- Push Multi-Arch Images


default:
image: docker:20.10.22-git
image: docker:20.10.22-git # TODO: upgrade?
lreiher marked this conversation as resolved.
Show resolved Hide resolved
services:
- docker:20.10.22-dind
tags:
- privileged
- amd64
before_script:
- echo -e "section_start:`date +%s`:setup_section[collapsed=true]\r\e[0K[docker-ros] Setup docker-ros"
- apk add bash
- apk add --update bash
- cd ${BUILD_CONTEXT}
- |-
if [[ ! -d docker/docker-ros ]]; then
Expand Down Expand Up @@ -168,6 +178,7 @@ run-arm64:


.test:
stage: Test ROS Industrial CI
variables:
UPSTREAM_WORKSPACE: ${BUILD_CONTEXT}/.repos
TARGET_WORKSPACE: ${BUILD_CONTEXT}
Expand All @@ -186,7 +197,6 @@ run-arm64:
script: .industrial_ci/gitlab.sh

Test dev-amd64:
stage: Test ROS Industrial CI
extends: .test
needs:
- job: dev-amd64
Expand All @@ -197,7 +207,6 @@ Test dev-amd64:
_PLATFORM: amd64

Test dev-arm64:
stage: Test ROS Industrial CI
extends: .test
tags: [privileged, arm64]
needs:
Expand All @@ -209,7 +218,6 @@ Test dev-arm64:
_PLATFORM: arm64

Test run-amd64:
stage: Test ROS Industrial CI
extends: .test
needs:
- job: run-amd64
Expand All @@ -220,7 +228,6 @@ Test run-amd64:
_PLATFORM: amd64

Test run-arm64:
stage: Test ROS Industrial CI
extends: .test
needs:
- job: run-arm64
Expand All @@ -231,6 +238,55 @@ Test run-arm64:
_PLATFORM: arm64


.slim:
stage: Slim Images
lreiher marked this conversation as resolved.
Show resolved Hide resolved
before_script:
- apk add --update --upgrade curl tar
- curl -L -o ds.tar.gz ${SLIM_DOWNLOAD_URL}
- tar -xvf ds.tar.gz
- cd dist_linux*
- docker login -u ${REGISTRY_USER} -p ${REGISTRY_PASSWORD} ${REGISTRY}
- docker pull ${FAT_IMAGE}
script:
- ./slim build --sensor-ipc-mode proxy --target ${FAT_IMAGE} --tag ${SLIM_IMAGE} --continue-after=10 --show-clogs --http-probe=false
lreiher marked this conversation as resolved.
Show resolved Hide resolved
- docker push ${SLIM_IMAGE}
artifacts:
lreiher marked this conversation as resolved.
Show resolved Hide resolved
expire_in: 1 day
paths:
- slim.report.json

Slim run-amd64:
stage: Slim Images
extends: .slim
needs:
- job: run-amd64
optional: true
- job: Test run-amd64
optional: true
rules:
- if: $PLATFORM =~ /.*amd64.*/ && $TARGET =~ /.*run.*/
variables:
FAT_IMAGE: ${_IMAGE_RUN_CI_AMD64}
SLIM_IMAGE: ${_IMAGE_SLIM_CI_AMD64}
SLIM_DOWNLOAD_URL: "https://github.com/slimtoolkit/slim/releases/download/1.40.11/dist_linux.tar.gz"

Slim run-arm64:
stage: Slim Images
extends: .slim
tags: [privileged, arm64]
needs:
- job: run-arm64
optional: true
- job: Test run-arm64
optional: true
rules:
- if: $PLATFORM =~ /.*arm64.*/ && $TARGET =~ /.*run.*/
variables:
FAT_IMAGE: ${_IMAGE_RUN_CI_ARM64}
SLIM_IMAGE: ${_IMAGE_SLIM_CI_ARM64}
SLIM_DOWNLOAD_URL: "https://github.com/slimtoolkit/slim/releases/download/1.40.11/dist_linux_arm64.tar.gz"


.push:
needs:
- job: dev-amd64
Expand All @@ -249,6 +305,10 @@ Test run-arm64:
optional: true
- job: Test run-arm64
optional: true
- job: Slim run-amd64
optional: true
- job: Slim run-arm64
optional: true
rules:
- if: $PLATFORM == '' || $TARGET == ''
when: never
Expand Down
105 changes: 0 additions & 105 deletions templates/README.template.md

This file was deleted.

43 changes: 0 additions & 43 deletions templates/docker-compose.template.yml

This file was deleted.