RFC: should we make HOME writable in v1.0?

[AkihiroSuda]

v1.0 will change the default mount driver from reverse-sshfs to virtiofs (for vz) and 9p (for qemu),
so mounting will be more stable:

• v1.0 roadmap: change the default mount driver from reverse-sshfs to 9p (for QEMU) and virtiofs (for vz) #971

v1.0 is also a good milestone to have breaking changes, although we should try to avoid having breaking changes.

In v1.0, should we make the host home mount (/Users/foo) writable or not?

My current position is no, as I still want to minimize having breaking changes, and the mount can be easily configured to be writable with limactl start --mount-writable .
But I'm happy to hear opinions.

[afbjorklund]

My thoughts that it was a security concern, that Docker Machine would export all of the home directory to the VM*. Both the privacy part, and the risk of doing damage to the host when doing something wrong in the VM or privileged containers...

* Actually it only worked in VirtualBox and not in qemu/kvm, but that is another story (missing the 9p support).

So I made it not mount the home directory by default in Podman Machine, even though it was available as an option if the user accepted the risks involved. The users complained, since it was "not Docker", and the decision was reverted* by Red Hat for v4.1.0.

* It is now $HOME:$HOME (read-write): https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html

---

For Lima, the privacy concerns are still there. But the read-only mount made it harder to do damage to the host...

There are also some performance implications, with encouraging users to work on a remote file system by default.

So I also think "no" (i.e. as-is/opt-in)

But also know it will confuse some.

[jandubois]

I never considered Lima instances to be a "secure sandbox" to run untrusted code, so I never though of mounting directories read-only as a security concern. I assumed we did this to minimize the risk that we would accidentally overwrite some host data.

With Lima (just like Microsoft with WSL) we are trying to make the boundary between the host and the VM disappear as much as possible. When you run containers, we try to make it look like they run on the host and not in the VM. Not being able to write to the mounted directories feels like it contradicts this goal; it re-introduces friction when the purpose of the tool is to remove friction.

Personally I've always been using an override.yaml file to make the home directory (and my /Volumes) writable, for my personal convenience.

I know that many users expect this as well, which is why Rancher Desktop mounts all directories as writable as well.

But I guess it is slightly different when the VM is purely a container host: the mounted directories are not automatically exposed to containers; the user still has to mount them once more, and that is the boundary to decide if the mount should be writable or not. If you mount directories read-only into the VM, then you don't have the option of making them writable inside specific containers.

I prefer the directories mountable, but I have no strong opinion because you can override the default. So I'm also ok with leaving it read-only, if that's what you prefer.

[afbjorklund]

I think this is the main difference, if the (and only "the") VM is an extension of the host - or if it is just a (of many) VM...
To maintain the illusion, the one that I called the "Wizard of Oz mode", I can see why one wants it read-write by default.

And it is true that when you run the container engine directly on your Linux host, it can also see your home directory.
So if it is not a VM, but an extension of your host, then it makes sense that your other operating system can see it too.

[afbjorklund]

@AkihiroSuda : if we do change the default for 1.0, that still means that it is possible to do the opposite of the above. To create a user preference file, if you want to keep the current behaviour of mounting the home directory as read-only...

Maybe expose some similar boolean, so you don't have to fiddle with the mounts ?

      --mount strings       [limactl create] directories to mount, suffix ':w' for writable (Do not specify directories that overlap with the existing mounts)
      --mount-type string   [limactl create] mount type (reverse-sshfs, 9p, virtiofs)
      --mount-writable      [limactl create] make all mounts writable

mounts:
- location: "~"
  mountPoint: null
  writable: null <--- use new default
  sshfs:
    cache: null
    followSymlinks: null
    sftpDriver: null
  9p:
    securityModel: null
    protocolVersion: null
    msize: null
    cache: null
- location: "/tmp/lima"
  writable: true

mountType: null
mountWritable: true   <--- new config

Then you could put the mountWritable into your override.yaml, to avoid having to remember it for create.

Possible a good idea either way, no matter if the Lima built-in defaults to true or false, make it easier to "see". Or something to that effect, since there are good arguments for either default config (i.e. to be like Docker or Vagrant).

[afbjorklund]

Changed my mind: v1.0

After consideration, it should probably be read-write by default - with an option to go back to read-only (or unmounted).

If nerdctl.lima is the main use case for Lima, and if it wants to be similar to Docker Machine and Podman Machine....
Then it makes sense to have it follow the path of least surprise, and have it more similar to WSL - an extension of host.

There were some similarly surprised Apptainer users, and having to tell them to use /tmp/lima rather than their home.

$ apptainer.lima build alpine.sif docker://alpine
INFO:    Starting build...
Getting image source signatures
Copying blob 96526aa774ef done  
Copying config e175fa54c5 done  
Writing manifest to image destination
Storing signatures
2023/10/02 16:41:33  info unpack layer: sha256:96526aa774ef0126ad0fe9e9a95764c5fc37f409ab9e97021e7b4775d82bf6fa
INFO:    Creating SIF file...
FATAL:   While performing build: while creating SIF: while creating container: open /home/anders/lima/alpine.sif: read-only file system

From https://apptainer.org/docs/user/main/quick_start.html

---

Just that it will be unevitable, to tell them to do cd in the lima shell if they want to use a faster (local) file system.

• https://learn.microsoft.com/en-us/windows/wsl/setup/environment#file-storage

And similarly to use name:/foo mounts (from /var), instead of using $PWD/dir:/bar mounts (i.e. from /home).

• https://docs.docker.com/storage/volumes/ (their use of "bind mount" is confusing)

[nirs]

Exposing your home directory to vms is unexpected and insecure. Uses assumes that vms are isolated from the host and are safe place to run anything.

The best way would be to have a config file with these defaults so user can adapt the default to their need. With such config you can create a lima distribution with different defaults that match the use case. Foe example as a tool to run containers you may want to make the lima vm transparent and integrated into the host. As a tool for managing vms (like virsh for linux), lima vms can be completely isolated by default.

[afbjorklund]

Exposing your home directory to vms is unexpected and insecure. Uses assumes that vms are isolated from the host and are safe place to run anything.

Different users have different expectations. Users of Docker or WSL don't even know they are using a VM, and expect all of $HOME or C:\ to be available...

I call it the "Wizard of Oz" mode.

I agree it would make sense to have it configurable, but then we need to remove it from the copy/paste "mounts" configuration in the templates?

images:
- default

mounts:
- default

---

• Provide standard values "default" and "none" for most options #2458
• PoC: Allow loading images by name from imagestore #2404

[afbjorklund]

Even the user name (and uid) has been questioned by users that are expecting a VM, i.e. they don't want it synched

Especially former users of Vagrant are expecting the default user from the cloud-init, and only $PWD mounted (at most)

[jandubois]

I agree it would make sense to have it configurable, but then we need to remove it from the copy/paste "mounts" configuration in the templates?

images:
- default

mounts:
- default

I think this could be another use-case of the basedOn mechanism I described in #2520 (reply in thread):

We could have a default-mounts.yaml template:

param:
  HOME_RW: false
  TMP_LIMA_RW: true

mounts:
- location: ~
  writable: {{.Param.HOME_RW}}
- location: /tmp/lima
  writable: {{.Param.TMP_LIMA_RW}}

and then instead of cuting-and-pasting the default settings each template would include

basedOn: template://default-mounts

Then users can set HOME_RW to true in their override.yaml file, if they want, or set it on a case-by-case basis:

limactl create --name my-vm --param HOME_RW=true template://ubuntu

I think I will work on a PoC for a (nestable) basedOn implementation in the next month or two because I'm convinced this will make our whole templating system much more powerful.

[AkihiroSuda]

limactl create --name my-vm --param HOME_RW=true template://ubuntu

Isn't --mount-writable enough?

[jandubois]

Isn't --mount-writable enough?

Probably. It was just an example how you can use the param mechanism together with the basedOn mechanism (which does not yet exist) to make things configurable without having to modify Lima itself.

I'm in favour of adding a mountWritable: setting that will be the default value for the writable property of each mount, and then exposing this via a --mount-writable option.

I continue to slightly favour this setting to be true by default, in which case a --mount-readonly option would work better than having to write --mount-writable=false. But I'm fine with either default.

[AkihiroSuda]

Closing as unplanned for now, but we may revisit this again in v2.0+

[AkihiroSuda]

basedOn

Could you create a separate issue?
I also prefer this to be called like extend: