Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable parallel image pulls #21

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Enable parallel image pulls #21

wants to merge 2 commits into from

Conversation

yankcrime
Copy link
Member

As of 1.27, Kubernetes added support for parallelizing image pulls but this is not enabled by default. This is especially useful in situations where multiple images need to be downloaded to a host, but one of them is particularly large.

As of 1.27, Kubernetes added support for parallelizing image pulls but
this is not enabled by default.  This is especially useful in situations
where multiple images need to be downloaded to a host, but one of them
is particularly large.
@yankcrime
Copy link
Member Author

@spjmurray Happy to be debate what in this PR should be configurable, and where.

@@ -107,6 +107,16 @@ spec:
path: {{ $file.path }}
permissions: "0600"
{{- end }}
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ew... erm, is there any benefit in having this on the CP nodes too? May make start up marginally faster

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And that answers my next question....

[Tue  5 Nov 12:35:12 GMT 2024] simon@symphony ~/src/github.com/kubernetes/kubernetes make all WHAT=cmd/kubelet
+++ [1105 12:35:36] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kubelet (non-static)
[Tue  5 Nov 12:35:38 GMT 2024] simon@symphony ~/src/github.com/kubernetes/kubernetes ./_output/bin/kubelet --help | grep serialize
      --serialize-image-pulls                                    Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given we pre-pull images necessary for bootstrap, I'd say probably not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cilium, occm, blah? I don't see the harm.

@spjmurray
Copy link
Member

owner: "root:root"
permissions: "0644"
content: |
{
Copy link
Member

@spjmurray spjmurray Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know what, I think it'll make life easier for the whole world if we just say...

values.yaml

workloadPools:
  kubelet:
    serializeImagePulls: "false"
    foo:
      bar: sheep
      baz: ball

Then go something like...

{{- with $config := .Values.workloadPools.kubelet }}
{{- $_ := set $config "apiVersion" "kubelet.config.k8s.io/v1beta1" -}}
{{- $_ := set $config "kind" "KubeletConfiguration" -}}
...
content: {{ toJson $config }}
{{- end }}

Then you can inject anything ever without another modification just using --set

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd check with multiple pools first though in case the config is mutated in place and you get multiple instances of the apiVersion and kind. There's probably a deep copy/clone somewhere in sprig.

- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
owner: "root:root"
permissions: "0644"
{{- with $config := $pool.kubelet }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't actually work when you test it...

Needs to be

{{- with ... -}}
{{- $_ := ... -}}
{{- $_ := ... }}

please note the trailing whitespace chomping, otherwise you get

      - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
        owner: "root:root"
        permissions: "0644"content: {"apiVersion":"kubelet.config.k8s.io/v1beta1","foo":"bar","kind":"KubeletConfiguration"}

@@ -115,4 +123,6 @@ spec:
node-labels: {{- include "openstack.nodelabels.workload" $context | nindent 14 }}
taints:
{{- include "openstack.taints.workload" $ | nindent 10 }}
patches:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when you provision something with no content? I can see...

        // Content is the actual content of the file.
        // +optional
        Content string `json:"content,omitempty"`

So the content itself is optional, I just dunno if kubeadm is gonna crap out with an empty patch file. Talk to me Goose!

@yankcrime yankcrime force-pushed the parallel_image_pulls branch 2 times, most recently from 98b321e to 1f6d718 Compare November 7, 2024 16:24
@yankcrime yankcrime marked this pull request as draft November 7, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants