[MULTI BOOT] Can someone explain how to use GRUB EFI and BTRFS to multiboot?

[oshunluvr]

History: I have been multibooting a legacy system (non EFI) for years using a BTRFS subvolume dedicate to grub. In the grub subvolume I have a grub.cfg that "points" to the various other BTRFS installs. Works like a charm.

I now have a UEFI laptop and am building a new desktop machine and I'd like to go ahead and finally transition to EFI booting.

I can not for the life of me, figure out how to change the grub.cfg that's located in /boot/efi/EFI/ubuntu. I can manually edit it, but the edit does not take effect and it's driving my batty.

Here's the setup:
Dual boot with Windows and Linux.
The Linux installs all reside in BTRFS subvolumes on a single partition.
The EFI "ubuntu" entry initially pointed to the original Kubuntu install in subvolume "@"

What I'm trying to do:
I made a snapshot of @ as @grub and then renamed it to @kubuntu_2204.
Rename the subvolumes allows me to install other versions of *buntus to the same BTRFS file system like KDEneon because they all default to "@" as the install subvolume.
The goal is to boot to @grub and use it as my grub installation to boot to other installs.

What goes wrong:
I manually edit /boot/efi/EFI/ubuntu/grub.cfg and change "@" to "@grub"
When I boot, grub boots to @kubuntu_2204 which is the original installation formerly named "@".
Basically, the edit I made is ignored.

What I've tried:
I have run update-grub and reinstalled grub-efi to /boot/efi. Nothing has made it work differently.
I have re-created @ and edited all the grub configs and NONE of the three grub.cfgs that now exist are actually being read when I boot.

What I want:
I want to be able to control my dang system.

I don't understand why I can't change my grub menu and I don't want to have yet another thing to manage like the EFI menu.

[oshunluvr]

It actually is boot into @grub but not reading the grub.cfg that's in @grub. It's really odd IMO

[GreyGeek]

I don't under stand why you don't "btrfs su snapshot @ @kubuntu_2204", assuming you don't use the "-r" flag?
I.e., why the renaming step?
Anyway, I'd use the grub terminal line to see what is going on.

[jlittle]

That's what I do at home, but I won't be free there till 8 pm-ish, that's 2022-10-26T07:00:00 UTC.

An unobvious thing is that even if you set the NVRAM boot variable (using efibootmgr) to point to a grubx64.efi other than the one in EFI/ubuntu/grubx64.efi, it still uses the grub.cfg in EFI/ubuntu.

IIRC it's not necessary to snapshot the existing linux root. With the btrfs root mounted on /mnt/top, one can

sudo btrfs subvolume create /mnt/top/@grub
sudo grub-install --boot-directory=/mnt/top/@grub/boot --efi-directory=/boot/efi

[GreyGeek]

John, wouldn't you have to include a device at the end of the second command?

[jlittle]

John, wouldn't you have to include a device at the end of the second command?

No. I've just run it as above, with no device. It wrote a new version of /boot/efi/ubuntu/grubx64.efi, changed the boot order to put an entry for it to the front, and finished with "Installation finished. No error reported.".

However, it did not write a new version of EFI/ubuntu/grub.cfg, which suggests to me that it is no longer used, which would explain oshunluvr's problem.

When I first started doing this, EFI/ubuntu/grub.cfg was written but not used. The path to /boot/grub/grub.cfg was coded into the grubx64.efi binary. Then some time before August 2020, it started being used. Maybe grub has gone back to that practice. Running

$ strings /boot/efi/EFI/ubuntu/grubx64.efi | grep @grub
(,gpt2)/@grub/boot/grub

suggests it might be (@grub is my invention, so the efi binary must have been changed).

[edit] Yes, on a reboot EFI/ubuntu/grub.cfg was ignored, and grub loaded @grub/boot/grub/grub.cfg. This was not good, as that file was generated in 2021, and tried to boot kernels from then, long gone. My john_grub.cfg was still there, so pressing "c" and running configfile $prefix/john_grub.cfg gave me back my old menu. I've copied my file over grub.cfg, so I'm back in control. Running update-grub updates /boot/grub/grub.cfg, not in @grub. Because I've switched my grub package from grub-efi-amd64 to grub-efi-amd64-bin, kernel and grub updates will not attempt to change the ESP or the NVRAM boot entries.

[oshunluvr]

UPDATE: It appears running update-grub does in fact change the menu I'm seeing. However, the added entries in grub.cfg are not appearing. The issue may have nothing to do with EFI, directly at least.

Here's what I did: I edited /etc/default/grub and ran update-grub. The changes I made (default kernel selection and timeout) appeared in the next boot, but the added entries from 40_custom are not.

[jlittle]

... but the added entries from 40_custom are not.

You might check that /etc/grub.d/40_custom is executable.

Alternatively, IMO the custom.cfg mechanism is simpler than the clunky 40_custom approach. Just put the entries in /boot/grub/custom.cfg. It has the advantage of not requiring a run of update-grub, and so easily edited while booted from another system, such as a live USB.

[oshunluvr]

Yes it is executable. I'm saying the 40_custom entries ARE IN /boot/grub/grub.cfg but they DO NOT display at boot time.They are in the file, but not showing in the boot menu.

I've checked the syntax several times to make sure it's correct. I'm flummoxed...

[jlittle]

I'm saying the 40_custom entries ARE IN /boot/grub/grub.cfg but they DO NOT display at boot tim.

That suggests to me that /boot/grub/grub.cfg is not the grub.cfg that is used at boot time.

[edit] since you state in an earlier post "update-grub does in fact change the menu I'm seeing" that would discount my suggestion.

I suggest typing "c" on the grub menu and running

echo $prefix
set pager=1
cat $prefix/grub.cfg

$prefix will show, using grub's own drive and partition scheme, where the running grub is. Mine says (hd0,gpt2)/@grub/grub which means the first SATA drive and the second partition on it, in Linux that would be /dev/sda2 (though in principle one can't rely on that mapping).

The set pager=1 will let you page through grub.cfg, and see whether your 40_custom entries are there. If they are, all I can think of is a misplaced "{" or "}" putting things off. If you can't, maybe there's something wrong with the clunky exec tail -n +3 $0 mechanism it uses.

[oshunluvr]

PARTIALLY SOLVED: I was thinking about this (before jlittle's last post - he was spot on as usual) and decided there must be a typo in the grub.cfg. Nothing else made sense. Sure enough, I used a capital M in "Menuentry" in 40_custom. I eventually saw a message flash saying "Command Menuentry not understood" or something similar and this was my hint.

However, I'm still unsure how to correctly update /boot/efi/EFI/ubuntu/grub.cfg except manually. At this point I've played around so much I'm unsure what command actually changes this entry. After manually editing the it, at least I'm booting correctly at this point.

[jlittle]

However, I'm still unsure how to correctly update /boot/efi/EFI/ubuntu/grub.cfg except manually.

When I ran grub-install --efi-directory=/boot/efi --boot-directory=/mnt/top/@grub/grub, that file (/boot/efi/EFI/ubuntu/grub.cfg) was not written and afterwards is not used. I know because I've edited it to run /boot/grub/john_grub.cfg, and that doesn't happen. I've concluded that the path to /mnt/top/@grub/grub (in grub form) is written into the grub EFI binary, as it used to be about 5 years ago. (I've had to copy john_grub.cfg over grub.cfg, which is ok because update-grub does not know about the @grub subvolume, and updates /boot/grub/grub.cfg.)

If your /boot/efi/EFI/ubuntu/grub.cfg is being used, and your grubx64.efi is up to date, I'm interested because it's more sensible than patching the efi binary. (I suspect the practice of patching the binary is due to the malign influence of secure boot.) Sensible methods tend to work better, and are more fixable when things go wrong.