Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force using /etc/containerd/certs.d for registry config. #3601

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Romain-Geissler-1A
Copy link

This is a breaking change, announced in release v0.20. See https://kind.sigs.k8s.io/docs/user/local-registry/ how to setup a local registry.

Note: users who used to patch the containerd config to set explicitly:

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

should now remove this patch as it is now kind's default configuration.

This is a more brutal change than #3574 in which it was suggested to go the hard way and switch the default now.

This is a breaking change, announced in release v0.20. See
https://kind.sigs.k8s.io/docs/user/local-registry/ how to setup a local
registry.

Note: users who used to patch the containerd config to set explicitly:

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

should now remove this patch as it is now kind's default configuration.
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 3, 2024
@k8s-ci-robot k8s-ci-robot requested a review from aojea May 3, 2024 09:27
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Romain-Geissler-1A
Once this PR has been reviewed and has the lgtm label, please assign aojea for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 3, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @Romain-Geissler-1A. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 3, 2024
@Romain-Geissler-1A
Copy link
Author

Hi,

Since you have just released, is there any chance to consider something like this before the next v0.24.0 release ?

@BenTheElder
Copy link
Member

BenTheElder commented May 16, 2024

I'm not sure I understand the motivation, versus using containerdConfigPatches for those willing and able to get ahead of this.

Otherwise, it seems like we'd be best to hold off on breaking changes until we're forced to ship [edit: containerd] 2.x?

@Romain-Geissler-1A
Copy link
Author

Romain-Geissler-1A commented May 16, 2024

The motivation is that I also maintain software on my own, and I find a bit strange that the default is the deprecated (but historical) behavior. In my own software when I introduce an breaking change, after a reasonable period of time I switch the default to the non-deprecated state, then eventually drop the old one. I initially tried to propose something less breaking, but then you suggested we should be more breaking, so here it is.

I maintain a kindest/node derivative image for a corporate environment, used by hundreds of developers. My policy is like Fedora: upstream first (when applicable). So I proposed this instead of having to maintain an internal patch on my side. Now if you prefer waiting for the upgrade to containerd 2.0, fine, I can decline this pull request.

@BenTheElder
Copy link
Member

The motivation is that I also maintain software on my own, and I find a bit strange that the default is the deprecated (but historical) behavior. In my own software when I introduce an breaking change, after a reasonable period of time I switch the default to the non-deprecated state, then eventually drop the old one. I initially tried to propose something less breaking, but then you suggested we should be more breaking, so here it is.

I agree with this sentiment, I think the distinction here is that we're not introducing this breaking change, but rather we've been preemptively warning users that we'll be forced to at some point (since forking containerd is infeasible / unreasonable for us and switching runtimes would be even more breaking).

And further that it costs us ~nothing to maintain the current state, and the new API is more difficult for users to use.

I maintain a kindest/node derivative image for a corporate environment, used by hundreds of developers. My policy is like Fedora: upstream first (when applicable).

Appreciated, a fair warning that from our point of view while details of the kind image leak and we don't make breaking changes for fun we also generally only advertise that it can run Kubernetes at a given version and future images might look rather different in ways we don't reasonably expect users to depend on (versus configuring registry mirrors in the runtime, that is unfortunately not really covered by the Kubernetes API but particularly useful for kind clusters).

So I proposed this instead of having to maintain an internal patch on my side. Now if you prefer waiting for the upgrade to containerd 2.0, fine, I can decline this pull request.

I'm not actually sure what is preferable. I would've liked to give users a smoother transition, I'm also not sure how soon we will have to do this, and there are unfortunately a lot of users that just adopt kindest/node images across releases without seeing the release notes.

@Romain-Geissler-1A
Copy link
Author

That's why I initially proposed a smoother version which would try to "infer" from the user provided files if we are in one case or another, but maybe this could be refined to look for hosts.toml file for example.

Note that I just re-checked the upstream containerd documentation, and if I understand this commit containerd/containerd@c514630 correctly, it seems there is yet another breaking change (version 2 --> 3 and a change of plugin name) in containerd 2.0. So in effect 3 possible configuration: historical one with < 2.0, the move the "new" one with < 2.0, and the "real new" one with >= 2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants