Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement dumb cache for images #1102

Merged
merged 1 commit into from
Jul 31, 2023
Merged

Conversation

jonjohnsonjr
Copy link
Collaborator

@jonjohnsonjr jonjohnsonjr commented Jul 27, 2023

This speeds up repeated builds by caching manifests and config files locally under ${KOCACHE}/img.
Before:

ko build .  1.09s user 2.37s system 148% cpu 2.326 total

After:

ko build .  1.05s user 2.29s system 244% cpu 1.366 total

(Most of the time difference comes from not having to do the TLS handshake with DockerHub to fetch the golang stuff.)

This has a nice effect of allowing offline builds with ko if your base images are referenced by digest and you pre-pull the images into ${KOCACHE}/img.

This does not cache base image layers by default because they are often present in the destination registry and would slow things down and take up space.

It might make sense to add a command or flag to completely cache the base image, but we could also just add some docs about how to pull it off with crane, e.g. for ko:

$ crane pull --format oci golang:1.20 $KOCACHE/img

Then modifying the .ko.yaml file to be:

baseImageOverrides:
  github.com/google/ko: golang:1.20@sha256:cfc9d1b07b1ef4f7a4571f0b60a99646a92ef76adb7d9943f4cb7b606c6554e2

If we --push=false, we don't hit the network at all:

time ko build -v . --push=false
2023/07/27 14:37:32 maxprocs: Leaving GOMAXPROCS=12: CPU quota undefined
2023/07/27 14:37:32 cache hit: golang:1.20@sha256:cfc9d1b07b1ef4f7a4571f0b60a99646a92ef76adb7d9943f4cb7b606c6554e2
2023/07/27 14:37:32 Using base golang:1.20@sha256:cfc9d1b07b1ef4f7a4571f0b60a99646a92ef76adb7d9943f4cb7b606c6554e2 for github.com/google/ko
2023/07/27 14:37:32 cache hit: index.docker.io/library/golang@sha256:832f2f74baa3e2b00ace688cb2fa934dffeade39f5b4c0cc8b1cff8d3fb084a0
2023/07/27 14:37:32 Using build config ko for github.com/google/ko
2023/07/27 14:37:32 Building github.com/google/ko for linux/amd64
us-central1-docker.pkg.dev/jonjohnson-chainguard/ko/ko-98b8c7facdad74510a7cae0cd368eb4e:latest@sha256:3609cb6560c07a6f2cff054efbf962860e35e89cfa495a85cbf71e71c9e99445
ko build -v . --push=false  0.96s user 2.17s system 573% cpu 0.546 total

Copy link
Member

@cpanato cpanato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow

@jonjohnsonjr jonjohnsonjr merged commit 81c1c70 into ko-build:main Jul 31, 2023
23 checks passed
@jonjohnsonjr jonjohnsonjr deleted the image-cache branch July 31, 2023 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants