My Gentoo based Linux laptop which is still running the older kernel 5.3.15. A short while back, I had a failed attempt to upgrade the kernel to 5.4.2 as the disk which encrypted with dm-crypt couldn’t be decrypted on boot. Since the older kernel 5.3.15 was working fine and free time was I what I lacked of, it seemed logical to just not doing anything about it.

Today while updating all the packages on my Gentoo laptop, I notice there was a newer kernel available: 5.6.3. So let’s try to install this newest kernel again and see if the issue with Luks/dm-crypt still persists.

I followed the instruction which I wrote in my previous posts:

The compilation and the installation of the 5.6.3 kernel went smoothly. However, on booting up, when I was prompted for the Luks password to decrypt the/dev/sda3, root partition, it failed again.

Here’s the (error) message:

device-mapper: table: 253:0: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
Enter passphrase for /dev/sda3: _

The above message gave some clues that it’s possible some cipher mode is missing. It’s also possible that I might have used a weak cipher mode when the partition was encrypted during this Gentoo installation.

To find out which cipher mode used for the (root) partition, the laptop was rebooted using an older working kernel. Then issue this command:

# cryptsetup  luksDump /dev/sda3
LUKS header information for /dev/sda3

Version:       	1
Cipher name:   	aes
Cipher mode:   	cbc-essiv:sha256
Hash spec:     	sha256
Payload offset:	4096
MK bits:       	256
MK digest:     	a1 e2 f8 6d c3 71 de 37 0d 45 a8 47 ec 88 67 df 6b 00 1c b2 
MK salt:       	28 f8 04 49 74 63 27 19 ae af 55 c8 86 76 72 0d 
               	0e 5a 50 e2 20 ac e1 20 9f d7 1f 27 bc 1b 57 6a 
MK iterations: 	154021
UUID:          	0c57c200-0167-498b-ab4c-1f756ade9c20

Key Slot 0: ENABLED
	Iterations:         	2464338
	Salt:               	d6 fa 9a a9 71 f8 0b 78 bc 0f 89 1a a5 ec de aa 
	                      	e7 d0 42 11 df 75 44 c9 48 c8 df 32 3d c2 59 41 
	Key material offset:	8
	AF stripes:            	4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

This line Cipher mode: cbc-essiv:sha256 indicates that I need to have CBC and ESSIV flag sets in the kernel. So I looked into /usr/src/linux/.config (currently pointing to the kernel 5.6.3), and found both CONFIG_CRYPTO_CBC=m and CONFIG_CRYPTO_ESSIV=m. m means it’s compiled as a kernel module, not built into the kernel.

I found somewhere in the Gentoo forum (sorry I forgot the link to the post), that it’s required to have the crypto built into the kernel. So I changed both of the variables to: CONFIG_CRYPTO_CBC=y and CONFIG_CRYPTO_ESSIV=y and recompile the kernel. Guess what? It still didn’t work! Fortunately, this problem was easy to fix. I had CLEAN="no" in /etc/genkernel.conf which skip running make clean before compilation. So the last fix was to change it to CLEAN="yes" and recompiled the kernel again.

After the machine rebooted using kernel 5.6.3, everything was working again!

Here is the command that I used to encrypt the /dev/sda3 partition:

root@localhost #cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 /dev/sda3

Since I got kernel 5.6.3 working, I was wondering if I could also go back to compile the older kernel 5.4.2 by forcing the 2 crypto to be built into the kernel. Indeed, kernel 5.4.2 was working as expected to.