Crypttab support testing

[subzero79]

I have created a branch in my repository with basic simplified crypttab support. This means you can now add your luks devices to the crypttab and they will be unlocked at boot.

For testing you need git, make, fakeroot and debhelper

apt install git make fakeroot debhelper
mkdir ~/test/
git clone https://github.com/subzero79/openmediavault-luksencryption/
cd ~/test/openmediavault-luksencryption
git checkout crypttab_test
fakeroot debian/rules binary clean

the package will be left above the directory you can now install with dpkg -i

Usage:

Add the device (has to be unlocked) by pressing crypttab->add, supply a keyfile or leave it empty. the keyfile has to be valid, is not like you are adding a keyfile to the device. If is not valid the backend will throw and exception message

The yellow apply button will come, press it and devices will appear in /etc/crypttab.

If no keyfile is supplied "-" will placed in the third field of crypttab and you'll be prompted at boot for a valid passphrase in the local console.

there is a hidden column in the grid that displays if the device is in crypttab(this relies on omv internal db).

You cannot edit a crypttab device, if you want change the keyfile, delete it from crypttab and add it again, for this you don't have to press the yellow button once is deleted, just can just add the keyfile again and finally press apply.

[subzero79]

To update when a commit is pushed just go to the src directory and run

git checkout crypttab_test
git pull

[nuentes]

I'm interested in testing. Do I need to remove the existing LUKS plugin first? Or does that not matter at all? Also, I see that this has been testing for ~7 months. Are there any plans to move this to the production environment in the near future?

[Herjar]

Moving from Windows with Bitlocker I am really missing pre-boot auth. Are there any news about this? I see there are no new commits the last 11 months. Is it working ok?

[subzero79]

You can see there’s no feedback. Maybe no interest.

[Herjar]

Maybe it was just not noticed here? This repo is only starred 8 times, and I am pretty sure the plugin is used by many more people than that. Maybe posting the information in the forums would get some interest?

Are you using it yourself?

[Herjar]

I read a bit on the forums and I see you already mentioned your branch here: https://forum.openmediavault.org/index.php?thread/18158-luks-keyfile-automount-solved/&pageNo=3

That thread also made me try crypttab with initramfs, which seems to do what I want.

[dmtucker]

Gave it a go and I also get

Failed to execute XPath query '//system/storage/luks'.

Error #0:
OMV\Config\DatabaseException: Failed to execute XPath query '//system/storage/luks'. in /usr/share/php/openmediavault/config/database.inc:262
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/luks.inc(591): OMV\Config\Database->set(Object(OMV\Config\ConfigObject))
#1 [internal function]: OMVRpcServiceLuksMgmt->setCrypttab(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('setCrypttab', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('LuksMgmt', 'setCrypttab', Array, Array, 1)
#5 {main}

[subzero79]

Yeah so is missing the db entry. should be added on package postinst. I need to dig where i left the testing VM tyhis was 11 months ago

[subzero79]

Just pushed a commit to fix that error you need to update from the crypttab_test branch , recreate the package and re-install

[subzero79]

Are you using it yourself?

I am not using, since my omv is a proxmox kvm machine, i don't use the keyfile so i just do

ssh pve qm terminal id

then i am prompted for the disk passphrase. I created this to finish something the original creator of the plugin was intending to do.

[dmtucker]

Just pushed a commit to fix that error

Nice! This seems to have worked. I can get through the "Crypttab > Add" dialog, then it asks to apply the config change (which completes successfully); however, there's still nothing in /etc/crypttab and the volume does not automatically unlock/mount on reboot 🤔

[subzero79]

Did you add a key file or left it empty so that it prompts at boot ?

[subzero79]

So to debug all of this output as root:

omv-confdbadm read conf.system.storage.luks.device

omv-salt deploy run luks

salt-call --log-level debug --local state.orchestrate omv.deploy.luks

[dmtucker]

I added a key file.

$ sudo omv-confdbadm read conf.system.storage.luks.device
[{"uuid": "d41038fa-7847-4b62-9600-2fea6ca86ccd", "luksuuid": "7e393fcd-a7d7-4dcc-a1b6-b7f67db7d9a4", "devicefile": "/dev/sdb", "keyfile": "/etc/openmediavault/luks/LUKS_Mobius_-dev-sdb_WDC-WD40EZRZ-22GXCB0_WD-WCC7K2TCS3YX_7e393fcd-a7d7-4dcc-a1b6-b7f67db7d9a4.key", "devicemappername": "sdb-crypt"}]

$ sudo omv-salt deploy run luks
debian:
----------
          ID: remove_openmediavault_luks_default
    Function: file.absent
        Name: /etc/default/openmediavault-luksencryption
      Result: True
     Comment: File /etc/default/openmediavault-luksencryption is not present
     Started: 01:42:44.661585
    Duration: 0.465 ms
     Changes:   

Summary for debian
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:   0.465 ms

Final command at https://gist.github.com/dmtucker/a4e42690efa346702f5a8fde1613116d

[subzero79]

I think I know what’s going on. This is an issue with saltstack cache. Is not picking the new salt files. I cannot remember exactly how to clear them but I have seen this issue before

[subzero79]

Cache should clear with

salt-call —local saltultil.clear_cache

then try again in the webui and check crypttab has entries those time.

[dmtucker]

Ok, ran that (couple of typo fixes)
sudo salt-call --local saltutil.clear_cache
Hit Crypttab > Remove in the webUI and got this when I clicked Apply:

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; omv-salt deploy run --no-color luks 2>&1' with exit code '1': debian: Data failed to compile: ---------- Rendering SLS 'base:omv.deploy.luks.01crypttab' failed: Jinja variable 'salt.utils.templates.AliasedLoader object' has no attribute 'omv_conf.get'

Error #0:
OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; omv-salt deploy run --no-color luks 2>&1' with exit code '1': debian:
    Data failed to compile:
----------
    Rendering SLS 'base:omv.deploy.luks.01crypttab' failed: Jinja variable 'salt.utils.templates.AliasedLoader object' has no attribute 'omv_conf.get' in /usr/share/php/openmediavault/system/process.inc:195
Stack trace:
#0 /usr/share/php/openmediavault/engine/module/serviceabstract.inc(62): OMV\System\Process->execute()
#1 /usr/share/openmediavault/engined/rpc/config.inc(167): OMV\Engine\Module\ServiceAbstract->deploy()
#2 [internal function]: Engined\Rpc\Config->applyChanges(Array, Array)
#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#4 /usr/share/php/openmediavault/rpc/serviceabstract.inc(149): OMV\Rpc\ServiceAbstract->callMethod('applyChanges', Array, Array)
#5 /usr/share/php/openmediavault/rpc/serviceabstract.inc(588): OMV\Rpc\ServiceAbstract->OMV\Rpc\{closure}('/tmp/bgstatus1J...', '/tmp/bgoutputLf...')
#6 /usr/share/php/openmediavault/rpc/serviceabstract.inc(159): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))
#7 /usr/share/openmediavault/engined/rpc/config.inc(189): OMV\Rpc\ServiceAbstract->callMethodBg('applyChanges', Array, Array)
#8 [internal function]: Engined\Rpc\Config->applyChangesBg(Array, Array)
#9 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#10 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('applyChangesBg', Array, Array)
#11 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Config', 'applyChangesBg', Array, Array, 1)
#12 {main}

Couldn't get around it... I'll try re-installing.

[subzero79]

I haven’t seen that error before. I’ll try later to see if I can replicate by clearing the cache and upgrading omv to latest. Could be an internal api that doesn’t exist then I need to update the salt conf.

[dmtucker]

Sounds good... reinstalling hasn't seemed to help FWIW

[subzero79]

Is the rest of the ui still working when saving and applying ?

[dmtucker]

I ended up having to uninstall fully (i.e. not just taking the stock 5.0.2 update) before reinstalling, and then when I rebooted and tried to unlock manually, I did get an "already mounted" error (it wasn't unlocked/mounted before the error). I can try to repro if you'd like.

[subzero79]

5.0.2 ? Why such old version?

[dmtucker]

The plugin version? That's the latest I'm offered (and same as the custom build).
OMV version is 5.6.4-1

[dmtucker]

Here's that busy error I mentioned

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; mount -v --source '/dev/disk/by-uuid/185b4871-f827-4804-87c4-c1c937d7643d' 2>&1' with exit code '32': mount: /srv/dev-disk-by-label-Mobius: /dev/mapper/sdb-crypt already mounted or mount point busy.

Error #0:
OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; mount -v --source '/dev/disk/by-uuid/185b4871-f827-4804-87c4-c1c937d7643d' 2>&1' with exit code '32': mount: /srv/dev-disk-by-label-Mobius: /dev/mapper/sdb-crypt already mounted or mount point busy. in /usr/share/php/openmediavault/system/process.inc:195
Stack trace:
#0 /usr/share/php/openmediavault/system/filesystem/filesystem.inc(742): OMV\System\Process->execute()
#1 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(949): OMV\System\Filesystem\Filesystem->mount()
#2 [internal function]: Engined\Rpc\OMVRpcServiceFileSystemMgmt->mount(Array, Array)
#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#4 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('mount', Array, Array)
#5 /usr/share/openmediavault/engined/rpc/luks.inc(392): OMV\Rpc\Rpc::call('FileSystemMgmt', 'mount', Array, Array)
#6 /usr/share/openmediavault/engined/rpc/luks.inc(319): OMVRpcServiceLuksMgmt->mountContainerFS('/dev/mapper/sdb...', Array)
#7 [internal function]: OMVRpcServiceLuksMgmt->openContainer(Array, Array)
#8 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#9 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('openContainer', Array, Array)
#10 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('LuksMgmt', 'openContainer', Array, Array, 1)
#11 {main}

[subzero79]

Oh ok. Yes I got confused. I though you mean omv version. I’ll have to look later What’s with that error.

[dmtucker]

Not sure 🤔 Some behind-the-scenes confusion regarding the mount point maybe? I was able to repro pretty easily, but it seems harmless so far.

[subzero79]

That error happens when the you attempt to unlock
The device ? Is calling the rpc service method of luks first then it triggers to mount the associated device but the device is already mounted or has content under somehow. The crypttab is still empty right ?

[subzero79]

So to fix the salt omv_conf.get attributte error you need to run

omv-salt stage run prepare

src: https://forum.openmediavault.org/index.php?thread/28373-samba/

Then you can try again in the webui to add a valid keyfile to a container, check the crypttab again if is populated correctly

[dmtucker]

That worked!

By the end of yesterday, I had re-installed the stock LUKS plugin and rebooted.
Trying again just now, altogether I ran

sudo dpkg -i openmediavault-luksencryption_5.0.2_all.deb
sudo salt-call --local saltutil.clear_cache
sudo omv-salt stage run prepare

Then, in the UI, I clicked on my drive and did Crypttab > Remove (lingering config from before) followed by Crypttab > Add and Apply (which succeeded). At that point /etc/crypttab populated 😊 I rebooted to take it for a spin, and it worked beautifully! 😎

[subzero79]

That’s good to know. Anymore feedback ?

@Herjar ?

I don’t know how to handle the plugin upgrade via apt and the salt issue.

[ryecoaaron]

Because the same version keeps getting installed, I think that causes most of the issues. I would think clearing the cache and doing the stage prepare shouldn't be needed in normal install cases.

[subzero79]

Ok I’ll have to test this in a clean vm with a new version then see how it goes.

[dmtucker]

LMK if there's anything more I can do to help 😊 I'm keen to see this go mainline!

[dmtucker]

@subzero79 Any update on this?

[subzero79]

Sorry I forgot about this. The patch should be ok except for the package postinst stage.
Unfortunately we are not in stage for adding features as omv6 is about to be out. I was trying to port the plug-in to version 6 of omv but I have to stop because core omv front end framework is much more capped than extjs. We cannot upload files the way we use to. I haven’t tested any workarounds so far as I am pretty limited in time at the moment.

If the plug-in goes out if I am not mistaken it could be possible that keyfile handling will not be possible.

[dmtucker]

How's it looking on OMV6? I'm still using this dev build which is the only reason I haven't upgraded yet 😅
Even if it's not expected to work on 6, will OMV overwrite a manually-configured crypttab?

[ryecoaaron]

@dmtucker I had forgotten about this branch and I ended up porting the plugin to OMV 6.x independent of any work from subzero. So, the current 6.x plugin has no crypttab support and no ability to use keyfiles (not possible with omv 6). I will look at the crypttab changes to see if they would be an easy add to what I ported.

[thisisgregm]

@ryecoaaron - Any updates on that please? Would be great to have it implemented. Thanks.

[ryecoaaron]

No. I forgot about it. I will try to look at it later today.

[ryecoaaron]

Just looking at this quickly, it would need a lot of work to port to 6.x since it relies on keyfiles. There is no keyfile support in the 6.x plugin because the functionality doesn't exist in OMV 6.x.