Enable setting cgroup v2 mode in std edition

[na0x2c6]

Hello.

I previously mentioned briefly about enabling cgroup v2 in the Alpine image at lima-vm/lima#1878. I have made modifications to allow changing the mode through environment variables during machine startup.

The default mode is hybrid mode, so I believe it does not affect the existing functionality. I would appreciate it if you could consider incorporating this modification if possible.

This modification was made with reference to the comment I received at lima-vm/lima#1878 (reply in thread). Thank you!

The pre-built image can be found at the following location:

https://github.com/na0x2c6/alpine-lima/releases/tag/v0.2.32-std-cgroup-experimental

The lima template that is currently being used is available here.

Thanks.

[jandubois]

I'm happy to make the cgroup mode configurable, but would make some changes to your PR.

[jandubois]

This line is not needed; the line above is exporting all LIMA_* environment variables into the container automatically. The 3 variables below are exceptions because they are renaming variables that don't start with LIMA_* on the host.

[jandubois]

Instead of 2 variables (LIMA_INSTALL_CGROUP_CONF and LIMA_ALPINE_CGROUP_MODE) I would prefer to have just a single variable LIMA_CGROUP_MODE that is set to the mode value, e.g.

Suggested change

	LIMA_INSTALL_CGROUP_CONF=true
	LIMA_CGROUP_MODE=hybrid

But given that this is the default, this setting should not be made in edition/std at all. Instead there should be a line in edition/min that just documents that the variable exists:

export LIMA_CGROUP_MODE=

Note that I removed the ALPINE_ substring from the variable name; it feels redundant to me, as every setting in this repo is about building an Alpine image.

[jandubois]

Suggested change

	if [ "${LIMA_INSTALL_CGROUP_CONF}" == "true" ]; then
	mkdir -p "${tmp}/etc/conf.d"
	echo 'rc_cgroup_mode=${LIMA_ALPINE_CGROUP_MODE:-hybrid}' >> ${tmp}/etc/conf.d/cgroups
	fi
	if [ -n "${LIMA_CGROUP_MODE}" ]; then
	mkdir -p "${tmp}/etc/conf.d"
	echo 'rc_cgroup_mode=${LIMA_CGROUP_MODE}' >> ${tmp}/etc/conf.d/cgroups
	fi

[na0x2c6]

@jandubois

Thank you for your review and suggestion. I apologize for my insufficient explanation.

In this PR, the intention is not to specify the cgroup mode at the time of the Alpine image build, but rather to make it possible to set the cgroup mode at the time of lima boot. Specifically, I intended to start the lima machine as rc_cgroup_mode=unified when the lima template is set as follows:

env:
  LIMA_ALPINE_CGROUP_MODE: unified

In other words, this PR creates a file /etc/conf.d/cgroups with the content:

rc_cgroup_mode=${LIMA_ALPINE_CGROUP_MODE:-hybrid}

So that at the time of lima machine startup:

• If the LIMA_ALPINE_CGROUP_MODE environment variable is NOT set in the lima template, cgroup will be (default value) hybrid
• If the LIMA_ALPINE_CGROUP_MODE environment variable is set in the lima template, cgroup will be set to that value.

With the above, at machine startup, cgroups can be selected using environment variables in a lima template, eliminating the need to prepare separate Alpine images for each cgroup mode.

(I included the string ALPINE_ because it becomes an option at lima startup, but I think it's actually redundant as you said...)

Based on the above intentions, would you say the modification is appropriate? Or is the modification policy inappropriate?

[na0x2c6]

I have uploaded the images, which can be used to test the above behavior, at the following location:

https://github.com/na0x2c6/alpine-lima/releases/tag/v0.2.32-std-cgroup-experimental

[na0x2c6]

I apologize for requesting a re-review without making any modifications. I commented because I failed to adequately explain the intent of the modifications. I would appreciate it if you could check it.

#114 (comment)

[jandubois]

Sorry, this PR somehow dropped from my radar. But looking at it again, don't you get the same effect from:

provision:
- mode: system
  script: |
    echo rc_cgroup_mode=unified >/etc/conf.d/cgroups
    rc-service cgroups restart

In which case this PR wouldn't be needed. Or am I missing something again?

[na0x2c6]

@jandubois

Thank you for your reply.

The scripts defined in the lima template are applied after mounting the cgroup here, so the cgroup v1 will be mounted on the first boot. Once it is mounted, it is not easy to unmount it because some processes are started with cgroup v1:

$ ls /sys/fs/cgroup/*/*/tasks
/sys/fs/cgroup/openrc/acpid/tasks  /sys/fs/cgroup/openrc/lima-guestagent/tasks  /sys/fs/cgroup/openrc/sshd/tasks

# show PIDs in cgroup v1
$ cat /sys/fs/cgroup/*/*/tasks
2418
2594
2596
2602
...

In other words, once cgroup v1 is mounted, rc-service cgroups restart cannot apply cgroup v2, thus necessitating a machine restart.

( lima-vm/lima#1878 is a question about whether there is a way to bypass the above problem. )

Actually, I wanted to use cgroup v2 because I am using rootless podman. Also, I have been using the alpine template because I couldn't start Fedora with the vmType: "vz" configuration. However, thanks to the great fixes in lima v0.19, such as lima-vm/lima#2026, lima-vm/lima#1926, lima-vm/lima#1942, lima-vm/lima#1965, I can now comfortably use the podman template.

So, to be honest, there is currently not much motivation to use cgroup v2 in the alpine image. If this PR seems helpful for future purposes, I would be happy to make the necessary modifications. However, if there is no specific reason to incorporate it, I would like to close it.

I am grateful for the development of this excellent image. Thank you for taking the time to review.

[jandubois]

Thank you for your contribution, but I don't want to apply this PR (anymore).

Alpine 3.19 defaults to using cgroup v2.

It is possible to revert back to v1 by editing /etc/rc.conf and rebooting in a provisioning script. This takes an extra 5-10s because it happens early during boot, and is only needed when you want to switch cgroup modes. I don't expect anyone wanting to switch back to hybrid mode, now that unified mode is working pretty well on Alpine.

I believe this is good enough. Please let me know if you disagree!

[jandubois]

I don't expect anyone wanting to switch back to hybrid mode

Well, actually, that was a bit hasty: In Rancher Desktop we switch back to hybrid mode when the user runs an older Kubernetes version because k3s only started supporting cgroup v2 in 1.20.4. So these are ancient/unsupported k3s versions, but they will still run on Rancher Desktop 1.13 using Alpine 3.19.

We select the cgroup mode with an environment variable in lima.yaml, like

env:
  RC_CGROUP_MODE: hybrid

And use this provisioning script to switch, when necessary: https://github.com/rancher-sandbox/rancher-desktop/blob/62807cbc4cff6c1099b966b6496f9770e050cb93/pkg/rancher-desktop/assets/lima-config.yaml#L41-L53

[na0x2c6]

Alpine 3.19 defaults to using cgroup v2.

I understand. I don't think it's a problem at all either. Thank you for confirming my issue/PR.