Error: upper fs does not support RENAME_WHITEOUT

[yonz2]

Hi,

I'm trying to copy a modified grub.cfg file to the ISO file of the latest ubuntu distribution: ubuntu-22.10-live-server-amd64.iso

I installed livefs-editor from github (so it's the latest version)

When I run the command livefs-edit ../$ORIG_ISO ../$MODDED_ISO --cp /tmp/grub.cfg I get an error:
subprocess.CalledProcessError: Command '['mount', '-t', 'overlay', 'overlay', '-o', 'lowerdir=/tmp/tmpit56cbff/old/iso,upperdir=/tmp/tmpit56cbff/.tmp/tmp03znjpuj,workdir=/tmp/tmpit56cbff/.tmp/tmpk5en_7du', '/tmp/tmpit56cbff/new/iso']' returned non-zero exit status 32.

looking at the messages output by dmesg I get:

[191790.930557] loop4: detected capacity change from 0 to 3208264
[191790.930802]  loop4: p1 p2 p3
[191790.931877]  loop4: p1 p2 p3
[191791.038643] ISO 9660 Extensions: Microsoft Joliet Level 3
[191791.038692] ISO 9660 Extensions: RRIP_1991A
[191791.045626] ISO 9660 Extensions: Microsoft Joliet Level 3
[191791.045665] ISO 9660 Extensions: RRIP_1991A
[191791.049904] overlayfs: upper fs does not support RENAME_WHITEOUT.
[191791.049926] overlayfs: upper fs missing required features.

I am running:

Distributor ID:	Ubuntu
Description:	Ubuntu 22.10
Release:	22.10
Codename:	kinetic

Here the complete output from livefs-edit

root@# ls ..
grub.cfg  livefs-editor  ubuntu-22.10-live-server-amd64.iso

root@# echo $ORIG_ISO
ubuntu-22.10-live-server-amd64.iso

root@# echo $MODDED_ISO
ubuntu-22.10-live-server-amd64-modded.iso

root@# livefs-edit ../$ORIG_ISO ../$MODDED_ISO --cp /tmp/grub.cfg new/iso/boot/grub/grub.cfg

set up loop device /dev/loop4 backing ../ubuntu-22.10-live-server-amd64.iso
found live iso9660 filesystem on /dev/loop4p1
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/__main__.py", line 62, in main
    ctxt.mount_source()
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/context.py", line 246, in mount_source
    self._source_overlay = self.add_overlay(
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/context.py", line 164, in add_overlay
    mountpoint=self.add_mount(
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/context.py", line 111, in add_mount
    run_capture(cmd)
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/__init__.py", line 11, in run_capture
    return run(
  File "/usr/local/lib/python3.10/dist-packages/livefs_edit/__init__.py", line 7, in run
    return subprocess.run(cmd, check=check, **kw)
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['mount', '-t', 'overlay', 'overlay', '-o', 'lowerdir=/tmp/tmpit56cbff/old/iso,upperdir=/tmp/tmpit56cbff/.tmp/tmp03znjpuj,workdir=/tmp/tmpit56cbff/.tmp/tmpk5en_7du', '/tmp/tmpit56cbff/new/iso']' returned non-zero exit status 32.

Stderr:

mount: /tmp/tmpit56cbff/new/iso: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.
       dmesg(1) may have more information after failed mount system call.

[mwhudson]

Wow. What filesystem is /tmp?

[mwhudson]

My guess: zfs

[yonz2]

Hi,
thanks for the quick response.

I am running "Vanilla Ubuntu 22.10". /tmp is NOT a mounted filesystem. Just a plain old directory under "/".

Running df -Th I get:

Filesystem                                       Type   Size  Used Avail Use% Mounted on
rpool/ROOT/ubuntu_kfivg1                         zfs    883G  5,8G  877G   1% /
...

So yes, you are right /tmp is on an zfs filesystem.

Any suggestions?

BR
Yonz

[yonz2]

Hi,
Moving /tmp to a tmpfs filesystem by adding the following to /etc/fstab:

tmpfs /tmp tmpfs mode=1777,strictatime,nosuid,nodev,size=2G 0 0

... and making sure all files are in the right place, livefs-edit did the job.

Thanks for pointing me in the right direction

BR
Yonz

[mwhudson]

I think livefs-edit could detect this and at least provide a helpful error or maybe even operate in a tmpfs automatically (although it in some cases this might end up using a lot of RAM so maybe should be opt-in)

[Chris-Peterson444]

I also have /tmp on zfs and can confirm a similar issue happens with --inject-snap.

A slightly more cryptic traceback:

sudo ./venv/bin/python3 -m livefs_edit ubuntu-22.04.1-live-server-amd64.iso custom.iso --inject-snap subiquity_test.snap
running inject-snap with arguments {'snap': 'subiquity_test.snap'}
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/cpete/software/subiquity/venv/lib/python3.10/site-packages/livefs_edit/__main__.py", line 75, in <module>
    main(sys.argv[1:])
  File "/home/cpete/software/subiquity/venv/lib/python3.10/site-packages/livefs_edit/__main__.py", line 66, in main
    func(ctxt, **kw)
  File "/home/cpete/software/subiquity/venv/lib/python3.10/site-packages/livefs_edit/actions.py", line 110, in inject_snap
    rootfs = setup_rootfs(ctxt)
  File "/home/cpete/software/subiquity/venv/lib/python3.10/site-packages/livefs_edit/actions.py", line 72, in setup_rootfs
    ctxt.add_sys_mounts(ctxt._rootfs_dir)
  File "/home/cpete/software/subiquity/venv/lib/python3.10/site-packages/livefs_edit/context.py", line 60, in add_sys_mounts
    os.rename(resolv_conf, resolv_conf + '.tmp')
OSError: [Errno 22] Invalid argument: '/tmp/tmppaclacmy/rootfs/etc/resolv.conf' -> '/tmp/tmppaclacmy/rootfs/etc/resolv.conf.tmp'

But almost identical dmesg output:

> [46696.557378] loop30: detected capacity change from 0 to 2880612
> [46696.557608] ISO 9660 Extensions: Microsoft Joliet Level 3
> [46696.557661] ISO 9660 Extensions: RRIP_1991A
> [46696.560302] overlayfs: upper fs does not support RENAME_WHITEOUT.
> [46696.560351] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
> [46703.103971] overlayfs: upper fs does not support RENAME_WHITEOUT.
> [46703.108397] loop31: detected capacity change from 0 to 288592
> [46703.112305] loop32: detected capacity change from 0 to 579640
> [46703.115682] loop33: detected capacity change from 0 to 763488
> [46703.118959] loop34: detected capacity change from 0 to 256936
> [46703.121488] overlayfs: upper fs does not support RENAME_WHITEOUT.
> [46703.121521] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.

An opt-in tmpfs mount behavior would be nice, but in the meantime maybe could we put a warning about filesystem compatibilities with overlayfs in the readme?