[Bug 1780332] Re: vaultlocker does not ensure that udev is triggered to create /dev/disk/by-uuid/<uuid-in-luks-header> symlink and fails

[Dmitrii Shcherbakov]

asi,

I would rule out the lack of rules because on both xenial and bionic we
have LVM udev rules with the following line that is supposed to create a
LUKS UUID-based symlink:`

ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*",
SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"

https://git.launchpad.net/~usd-import-team/ubuntu/+source/lvm2/tree/udev/13-dm-disk.rules.in?h=ubuntu/xenial
https://git.launchpad.net/~usd-import-team/ubuntu/+source/lvm2/tree/udev/13-dm-disk.rules.in?h=ubuntu/bionic

The above test have shown that the symlink gets created on bionic.

For xenial it seems to be the same but this test is different in terms
of CPU load present on a machine (there is no load in my tests now in
comparison to the situations based on which we filed this bug):

uname -a
Linux maas-vhost6 4.4.0-135-generic #161-Ubuntu SMP Mon Aug 27 10:45:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

grep -RP ID_FS_UUID_ENC /lib/udev/rules.d/
/lib/udev/rules.d/60-persistent-storage-dm.rules:ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
/lib/udev/rules.d/60-persistent-storage.rules:ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
/lib/udev/rules.d/69-bcache.rules:ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
/lib/udev/rules.d/69-lvm-metad.rules:ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-id/lvm-pv-uuid-$env{ID_FS_UUID_ENC}"
/lib/udev/rules.d/63-md-raid-arrays.rules:ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"

lsblk 
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda       8:0    0    64G  0 disk 
`-sda1    8:1    0    64G  0 part /
sdb       8:16   0     8G  0 disk 
sdc       8:32   0 102.4M  0 disk 
sdd       8:48   0 102.4M  0 disk 
sde       8:64   0 102.4M  0 disk 
vda     253:0    0 102.4M  0 disk 
vdb     253:16   0 102.4M  0 disk 
nvme0n1 259:0    0    20G  0 disk 
nvme1n1 259:1    0    20G  0 disk 


tree /dev/disk/by-uuid/
/dev/disk/by-uuid/
└── d26a75c9-15f7-41de-8c0e-20f795ed5729 -> ../../sda1

0 directories, 1 file

sudo cryptsetup luksFormat /dev/sdb

WARNING!
========
This will overwrite data on /dev/sdb irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase: 
Verify passphrase: 


ubuntu at maas-vhost6:~$ tree /dev/disk/by-uuid/
/dev/disk/by-uuid/
├── 42bf3808-9987-454f-be27-5d6c9b9c5c12 -> ../../sdb
└── d26a75c9-15f7-41de-8c0e-20f795ed5729 -> ../../sda1

0 directories, 2 files


sudo cryptsetup luksDump /dev/sdb | grep UUID
UUID:          	42bf3808-9987-454f-be27-5d6c9b9c5c12

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to cryptsetup in Ubuntu.
https://bugs.launchpad.net/bugs/1780332

Title:
  vaultlocker does not ensure that udev is triggered to create /dev/disk
  /by-uuid/<uuid-in-luks-header> symlink and fails

Status in vaultlocker:
  Fix Released
Status in cryptsetup package in Ubuntu:
  New
Status in systemd package in Ubuntu:
  Incomplete

Bug description:
  When an encrypted device is setup up a UUID (osd_fsid) is passed from
  the charm to be used in the cryptsetup command which accepts a UUID to
  place into the LUKS header (shown in cryptsetup luksDump <path-to-
  block-device>).

  https://github.com/openstack/charm-ceph-osd/blob/stable/18.05/lib/ceph/utils.py#L1788-L1804 
  UUID comes from osd_fsid

  https://github.com/openstack-charmers/vaultlocker/blob/8c9cb85dc3ed5dbf18c66a810d189a5230d85c34/vaultlocker/shell.py#L69-L80
  # else statement is used here
       block_uuid = str(uuid.uuid4()) if not args.uuid else args.uuid

       dmcrypt.luks_format(key, block_device, block_uuid) # creates a LUKS header
  # ...
       dmcrypt.luks_open(key, block_uuid) # sets up a device with device mapper decrypting it via dmcrypt

  https://github.com/openstack-
  charmers/vaultlocker/blob/d813233179bdf2eec8ed101c702a8e552a966f44/vaultlocker/dmcrypt.py#L44-L56

  This UUID is visible in blkid output

  /dev/sdc: UUID="<luks-header-uuid>" TYPE="crypto_LUKS"

  and a udev rule exists to create a /dev/disk/by-uuid/<luks-header-
  uuid> symlink (which is normally used for filesystem -> block device
  resolution)

  https://git.launchpad.net/~usd-import-team/ubuntu/+source/lvm2/tree/udev/13-dm-disk.rules.in#n25
  ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"

  
  Where vaultlocker fails is in luks_open command (right after luks_format) 

   # cryptsetup --batch-mode --key-file - open UUID=<luks-header-uuid>
  crypt-<luks-header-uuid> --type luks

  because it tries to access /dev/disk/by-uuid/<luks-header-uuid> which
  does not exist.

  This happens since udev rules are not re-triggered to create this
  symlink after a LUKS device is created.

  Solution: call the command below after luks_format before luks_open

  udevadm settle --exit-if-exists=/dev/disk/by-uuid/<luks-header-uuid-
  equal-to-osd-fsid>

To manage notifications about this bug go to:
https://bugs.launchpad.net/vaultlocker/+bug/1780332/+subscriptions