-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable parallel image pulls #21
base: main
Are you sure you want to change the base?
Conversation
As of 1.27, Kubernetes added support for parallelizing image pulls but this is not enabled by default. This is especially useful in situations where multiple images need to be downloaded to a host, but one of them is particularly large.
@spjmurray Happy to be debate what in this PR should be configurable, and where. |
@@ -107,6 +107,16 @@ spec: | |||
path: {{ $file.path }} | |||
permissions: "0600" | |||
{{- end }} | |||
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ew... erm, is there any benefit in having this on the CP nodes too? May make start up marginally faster
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And that answers my next question....
[Tue 5 Nov 12:35:12 GMT 2024] simon@symphony ~/src/github.com/kubernetes/kubernetes make all WHAT=cmd/kubelet
+++ [1105 12:35:36] Building go targets for linux/amd64
k8s.io/kubernetes/cmd/kubelet (non-static)
[Tue 5 Nov 12:35:38 GMT 2024] simon@symphony ~/src/github.com/kubernetes/kubernetes ./_output/bin/kubelet --help | grep serialize
--serialize-image-pulls Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given we pre-pull images necessary for bootstrap, I'd say probably not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cilium, occm, blah? I don't see the harm.
For future reference see here: https://github.com/kubernetes/kubernetes/blob/release-1.31/staging/src/k8s.io/kubelet/config/v1beta1/types.go#L89 |
owner: "root:root" | ||
permissions: "0644" | ||
content: | | ||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You know what, I think it'll make life easier for the whole world if we just say...
values.yaml
workloadPools:
kubelet:
serializeImagePulls: "false"
foo:
bar: sheep
baz: ball
Then go something like...
{{- with $config := .Values.workloadPools.kubelet }}
{{- $_ := set $config "apiVersion" "kubelet.config.k8s.io/v1beta1" -}}
{{- $_ := set $config "kind" "KubeletConfiguration" -}}
...
content: {{ toJson $config }}
{{- end }}
Then you can inject anything ever without another modification just using --set
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd check with multiple pools first though in case the config is mutated in place and you get multiple instances of the apiVersion
and kind
. There's probably a deep copy/clone somewhere in sprig.
cfbcb8a
to
278fcf8
Compare
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json | ||
owner: "root:root" | ||
permissions: "0644" | ||
{{- with $config := $pool.kubelet }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't actually work when you test it...
Needs to be
{{- with ... -}}
{{- $_ := ... -}}
{{- $_ := ... }}
please note the trailing whitespace chomping, otherwise you get
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
owner: "root:root"
permissions: "0644"content: {"apiVersion":"kubelet.config.k8s.io/v1beta1","foo":"bar","kind":"KubeletConfiguration"}
@@ -115,4 +123,6 @@ spec: | |||
node-labels: {{- include "openstack.nodelabels.workload" $context | nindent 14 }} | |||
taints: | |||
{{- include "openstack.taints.workload" $ | nindent 10 }} | |||
patches: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens when you provision something with no content? I can see...
// Content is the actual content of the file.
// +optional
Content string `json:"content,omitempty"`
So the content itself is optional, I just dunno if kubeadm is gonna crap out with an empty patch file. Talk to me Goose!
98b321e
to
1f6d718
Compare
1f6d718
to
d537c28
Compare
As of 1.27, Kubernetes added support for parallelizing image pulls but this is not enabled by default. This is especially useful in situations where multiple images need to be downloaded to a host, but one of them is particularly large.