Announcement

Collapse
No announcement yet.

EFI and BTRFS booting mess. Can I change the GRUB host?

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    EFI and BTRFS booting mess. Can I change the GRUB host?

    I'm not even into EFI ('cause why would I want to be?), but my newish laptop still has windows on it for warranty reasons so it's got EFI on it.

    If it's not yet clear, I'm an EFI noob or maybe even knob.

    ANYway, I had mucked about quite awhile ago and had on my laptop: Windows 10, KDEneon1, KDEneon2, and Manjaro. All of which were in the EFI GRUB menu.

    HISTORY:
    Both KDEneon installs were on a single BTRFS file system/partition
    Manjaro was on an EXT4 partition
    Windows was, well, all over the place <eyes rolling here>

    This lappy has a 1TB NVME drive so I just shrank the main Windows partition and made my 3 Linux partitions; BTRFS, EXT4, SWAP.

    Booting it: Some update broke Manjaro quite awhile ago. Something I had done when the KDEneon install was fresh left it unbootable, so my solution was to install a second neon, which allowed my to repair the first neon. This laptop was still a playground at this point. I didn't care enough to even try an fix Manjaro. Any distro that easy to break I don't need. Windows still works (as well as it can).

    TODAY:
    So last night I decided to finally get around to cleaning up my mess. I reformatted the EXT4 partition and mounted it as my vm_drive for my virtual machines. I deleted it's folder (Manjaro) from /boot/efi/EFI. Then I deleted the second KDEneon install and deleted it's EFI folder as well. One odd thing I noticed was in the EFI folder for the second neon install only had a single file in it - one of the .efi files. I don't remember which one. Then I rebooted...

    ...I had forgotten that the SECOND neon install was the GRUB host so I had a dead box (sort of). Rebooted into Windows via the BIOS boot list, Burned a fresh Kubuntu to a USB stick, and booted it. I decided rather than try and un-screw GRUB I would just install Kubuntu along side neon. I wanted to have to to compare to neon and keep it as a backup OS anyway so seemed like the right time. That all went perfectly and I booted to Kubuntu in less than 20 minutes.

    I was surprised slightly that GRUB did not pick up the Neon install from the EFI folder like it did windows. And it still won't. I don't know if this is related to the GRUB vs. BTRFS thing and I just have to manually enter it into GRUB or if something else is wrong. Since it's EFI I'm just not sure. I decided the "fix" was to boot into neon, run grub-install, and I'd be back to defaulting to neon, but it didn't work. GRUB is still booting from Kubuntu.

    This isn't the end of the world or anything, but I was wondering two things:
    1. Can I get GRUB to detect the second Linux install since the EFI stuff is actually there?
    2. Can I get GRUB to boot from KDEneon again or am I stuck with Kubuntu?


    Also note: The EFI folder for neon is named neon, but the second neon install EFI folder was named ubuntu, as is the new Kubuntu install. I can't remember for sure, but I assume I had renamed the EFI folder so it wouldn't be overwritten by the second neon install. I don't know if that's even necessary but as I said at the outset - EFI noob.

    Any suggestions would be welcome.

    Please Read Me

    #2
    Firstly, if you haven't, install efibootmgr and learn its clunky interface so you can see what the firmware is doing.

    Now, I think having an independent grub would suit you well. If you have multiple installs that come and go grub will always screw up. IMO its "simple, configuration" scripts are not designed to cope with many grubs vying for control. I've posted the details on KFN before, but briefly, as you've got a btrfs, create a subvolume for grub at the btrfs root (which, for *buntus at least, is not the Linux root), grub-install --boot-directory to that, then copy the EFI .efi executable to some directory and name you choose, and use efibootmgr to tell the firmware to boot from that. Or, if the btrfs might come and go, create a small partition just for grub, maybe ext4.

    When things change, you can rerun grub-install, or edit grub.cfg manually. A manual "menuentry" can be much simpler than the ones grub generates. Again, there's posts here on KFN with examples, but ask if you like.
    Regards, John Little

    Comment


      #3
      This is an issue I noticed sometime ago. Neon names it's EFI folder neon, but Mint and Ubuntu name it ubuntu. If grub sees an ubuntu folder, it does not care about any other folder, named neon or otherwise. You can compare the two grub.cfg files from neon and ubuntu. They will have the same root UUID.

      Sometime in 2019, neon went back to naming their efi folder ubuntu. So your efi folder named neon is probably being disregarded. Go into grub.cfg file of teh ubuntu EFI subfolder and set the UUID to the partition you want to edit.
      .
      Last edited by mr_raider; Jun 27, 2020, 05:43 PM.

      Comment


        #4
        Originally posted by oshunluvr View Post
        I'm not even into EFI ('cause why would I want to be?), but my newish laptop still has windows on it for warranty reasons so it's got EFI on it.

        If it's not yet clear, I'm an EFI noob or maybe even knob.
        Let me first enlighten you about the "EFI" term you use:
        • EFI is commonly used to refer to the ESP-partition which is in short a special purpose partition used by the UEFI-BOOT of the BIOS of your motherboard.
        • If the BIOS of your Laptop/Machine does not support UEFI then you won't be needing nor using the ESP.
        • There actually exists an EFI sub-directory inside the ESP and most usages are done below that EFI dir.

        That said you should have no trouble booting multiple Operating Systems/Distros when you use Systemd-boot correctly because it uses different vendor directories per O/S you configure
        If Neon (Which i have never tried) uses "ubuntu" as vendor directory then that's actually a needless critical fault of that distribution.
        (Although i can understand why, it boils down to laziness)

        If you try my script you will notice it will create "<ESP>/EFI/systemd" and "<ESP>/kubuntu", where "<ESP>/kubuntu" will hold your kernels and ramdisk images relating to your Kubuntu install...

        Note that the ESP has nothing to do with the partition your operating system is installed into...
        Last edited by ©TriMoon™; Jul 27, 2020, 03:29 AM.
        Well thats all for now, 3M

        Comment


          #5
          Looks like you haven't read what I posted, but thanks for commenting.

          Please Read Me

          Comment


            #6
            Can I just emphasize that on the ESP the directories and files you refer to are just that. If you don't like the names some installer uses, you can use dolphin to rename them, or copy them. The .efi files are only about 150 kB, so it's not a problem to have some copies. I have a directory EFI/john, with my.efi in it. So long as there's a UEFI non-volatile variable pointing at that, the names various installers or update scripts use are something I don't have to worry about.
            Regards, John Little

            Comment


              #7
              Originally posted by jlittle View Post
              Can I just emphasize that on the ESP the directories and files you refer to are just that. If you don't like the names some installer uses, you can use dolphin to rename them, or copy them. The .efi files are only about 150 kB, so it's not a problem to have some copies. I have a directory EFI/john, with my.efi in it. So long as there's a UEFI non-volatile variable pointing at that, the names various installers or update scripts use are something I don't have to worry about.
              You sure about that? I thought if you have an efi folder called Ubuntu, grub seed nothing else.

              I had an Ubuntu and neon folder, and any attempt to delete the Ubuntu folder would cause a boot fail.

              Sent from my HD1905 using Tapatalk

              Comment


                #8
                I have four folders in /boot/efi/EFI

                Boot
                Microsoft
                neon
                Ubuntu

                The boot folder has three .efi files in it.

                The "Ubuntu" grub.cfg points at Kubuntu, the other two are obvious.

                Please Read Me

                Comment


                  #9
                  This is my laptop so I don't use it much. There is a primary boot selection menu in the BIOS that I didn't know about before so I think that really solves my issue.

                  I think if I want to make it more robust, I'll do what jlittle suggested early on and create a stand-alone grub install as I have on my desktop. Just makes life easier.

                  Honestly though, I've got it cleaned up and it's apparently stable so I will probably leave well enough alone. When it's time to upgrade neon, I'll just snapshot and go for it. I'm keeping Windows on it for the time being, but only for trouble shooting or the occasional software test purposes. I use Windows for work so always good to have access to it.

                  Please Read Me

                  Comment


                    #10
                    How is refind at directly booting Linux on btrfs partition via the kernel stub loader.

                    In theory you coul ditch grub altogether. But last time I tried it it had issues with subvolumes.

                    Sent from my HD1905 using Tapatalk

                    Comment


                      #11
                      Originally posted by oshunluvr View Post
                      Looks like you haven't read what I posted, but thanks for commenting.
                      Not really, i actually did read it...

                      Originally posted by oshunluvr View Post
                      This isn't the end of the world or anything, but I was wondering two things:
                      1. Can I get GRUB to detect the second Linux install since the EFI stuff is actually there?
                      2. Can I get GRUB to boot from KDEneon again or am I stuck with Kubuntu?


                      Also note: The EFI folder for neon is named neon, but the second neon install EFI folder was named ubuntu, as is the new Kubuntu install. I can't remember for sure, but I assume I had renamed the EFI folder so it wouldn't be overwritten by the second neon install. I don't know if that's even necessary but as I said at the outset - EFI noob.

                      Any suggestions would be welcome.
                      Short answer:
                      If grub does it's work correctly it should detect the neon vendor directory inside the ESP. (I have no idea because i use Systemd-boot)

                      Long answer:
                      If each distro uses their own vendor directory inside the ESP there is no problem of them using grub separately from each other, because the ESP is used by the UEFI Firmware of your hardware to boot your machine.
                      Lets take a look at my ESP: (Mounted as /boot/efi)
                      Code:
                      > tree -I "sb-certs|MOK|tools" /boot/efi | xclip -selection clipboard
                      
                      /boot/efi
                      ├── EFI
                      │   ├── BOOT
                      │   │   ├── BOOTX64.EFI
                      │   │   ├── fbx64.efi
                      │   │   └── mmx64.efi
                      │   ├── systemd
                      │   │   ├── BOOTX64.CSV
                      │   │   ├── KeyTool.efi
                      │   │   ├── grubx64.efi
                      │   │   ├── mmx64.efi
                      │   │   ├── shimx64.efi
                      │   │   └── systemd-bootx64.efi
                      │   └── ubuntu
                      │       ├── BOOTX64.CSV
                      │       ├── grub.cfg
                      │       ├── grubx64.efi
                      │       ├── mmx64.efi
                      │       └── shimx64.efi
                      ├── Shell.efi
                      ├── X99-PRO-USB31-ASUS-3801.CAP
                      ├── X99-PRO-USB31-ASUS-3902.CAP
                      ├── efi-mount-sda.txt
                      ├── efi-mount-sdb.txt
                      ├── efitools
                      │   ├── HashTool.efi
                      │   ├── HelloWorld.efi
                      │   ├── KeyTool.efi
                      │   ├── Loader.efi
                      │   ├── LockDown.efi
                      │   ├── ReadVars.efi
                      │   ├── SetNull.efi
                      │   ├── ShimReplace.efi
                      │   └── UpdateVars.efi
                      ├── kubuntu
                      │   ├── initrd.img
                      │   ├── initrd.img.old
                      │   ├── initrd.img.prev
                      │   ├── vmlinuz
                      │   └── vmlinuz.old
                      ├── loader
                      │   ├── entries
                      │   │   ├── KeyTool.conf
                      │   │   ├── MokManager.conf
                      │   │   ├── efi-kubuntu.conf
                      │   │   ├── grub-kubuntu.conf
                      │   │   ├── kubuntu_.conf
                      │   │   ├── kubuntu_old.conf
                      │   │   └── kubuntu_prev.conf
                      │   ├── loader.conf
                      │   └── random-seed
                      └── startup.nsh
                      
                      8 directories, 43 files
                      Breaking down that output:
                      • Paths inside the ESP are written by convention with <ESP>-prepended and in M$ style, because of the UEFI software. (But can be accessed as usual inside a Linux environment)
                      • <ESP>\EFI\BOOT\BOOTX64.EFI is used in Non-SecureBoot/Legacy boot mode.
                      • <ESP>\EFI\...\*.CSV is used by <ESP>\EFI\BOOT\fbx64.efi (The fallback loader) to re-create UEFI boot entries in case there are none.
                      • <ESP>\EFI\BOOT\mmx64.efi is the MokManager meant to be used to add the Mok-Key used to sign the kernel drivers.
                        (But that process in *ubuntu does not work on my hardware, so i need to add that key manually don't ask ugh)
                      • All subdirectories under <ESP>\EFI, except for <ESP>\EFI\BOOT are meant to be used by vendors to put their boot-loaders in their own vendor-specific directories.
                      • Each <ESP>\EFI\...\shimx64.efi (The signed Shim boot-loader installed by *ubuntu) will try to execute a file called grubx64.efi or mmx64.efi inside the same directory as itself.
                        mmx64.efi is the MokManager that will get started if grubx64.efi does not pass the SecureBoot verification.
                        <ESP>\EFI\systemd\grubx64.efi is actually a renamed systemd-bootx64.efi because of this sequence.
                      • <ESP>\Shell.efi is my UEFI-Shell i can start directly from within my BIOS menus.
                        Which will automatically execute the commands inside <ESP>\startup.nsh on startup.
                      • <ESP>\*.CAP are bios flash files i used to update my BIOS.
                      • <ESP>\*.txt are just personal notes for myself.
                      • <ESP>\EFI\systemd is where Systemd-boot is installed by default. (Vendor = systemd)
                      • <ESP>\EFI\ubuntu is where *ubuntu installs their bootloader by default. (Vendor = ubuntu)
                      • <ESP>\efitools is just a play ground with the efi-tools binaries for me
                      • <ESP>\kubuntu is where my script installs the kernel(s) and ramdisk(s) by default (at moment), that get used by Systemd-boot.
                      • <ESP>\loader and below is where Systemd-boot looks for it's configuration files, with menu entries inside separate config snippets under entries.


                      So your Q: "Can I get GRUB to detect the second Linux install since the EFI stuff is actually there?"
                      A: It is specific to how grub's operating system detection-logic works.
                      The "EFI stuff" you refer-to are actually only boot-loader(s).

                      So your Q: "Can I get GRUB to boot from KDEneon again or am I stuck with Kubuntu?"
                      A: Same as above, with an addition:
                      If KDEneon was installed in it's own vendor directory it would be possible to boot it regardless if grub auto-detects it when you use Systemd-boot with a matching menu config
                      You also wrote:
                      Also note: The EFI folder for neon is named neon, but the second neon install EFI folder was named ubuntu, as is the new Kubuntu install. I can't remember for sure, but I assume I had renamed the EFI folder so it wouldn't be overwritten by the second neon install. I don't know if that's even necessary but as I said at the outset - EFI noob.
                      That means the second neon install has overwritten your previous ubuntu install's boot loader and it's grub.cfg that is used by the grub boot-loader inside <ESP>\EFI\ubuntu, but has not touched your first neon install as you have luckily renamed the directory of it.
                      Taking a look inside mine it shows:
                      Code:
                      search.fs_uuid deb6ca8a-f40e-44c8-aa8d-db16d41219a9 root hd1,gpt1 
                      set prefix=($root)'/boot/grub'
                      configfile $prefix/grub.cfg
                      That first line instructs grub where to find the root partition.
                      In my case it says to search for the partition with UUID deb6ca8a-f40e-44c8-aa8d-db16d41219a9 inside the first GPT partition-table on hd1, and use it as the $root variable (If i'm not mistaken)...
                      Well thats all for now, 3M

                      Comment


                        #12
                        Sorry for the long post above, but it's sometimes better to be as thorough as possible right?
                        Well thats all for now, 3M

                        Comment


                          #13
                          Alright so why does my EFI look like this:

                          Code:
                          [FONT=monospace]root@hpelitebook725g3:/boot/efi# tree
                          [COLOR=#5454FF][B].[/B][/COLOR]
                          └── [COLOR=#5454FF][B]EFI[/B][/COLOR]
                             ├── [COLOR=#5454FF][B]BOOT[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]BOOTX64.EFI[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]fbx64.efi[/B][/COLOR]
                             │   └── [COLOR=#54FF54][B]mmx64.efi[/B][/COLOR]
                             ├── [COLOR=#5454FF][B]neon[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]BOOTX64.CSV[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]grub.cfg[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]grubx64.efi[/B][/COLOR]
                             │   ├── [COLOR=#54FF54][B]mmx64.efi[/B][/COLOR]
                             │   └── [COLOR=#54FF54][B]shimx64.efi[/B][/COLOR]
                             └── [COLOR=#5454FF][B]ubuntu[/B][/COLOR]
                                 └── [COLOR=#54FF54][B]grub.cfg[/B][/COLOR]
                          
                          4 directories, 9 files
                          root@hpelitebook725g3:/boot/efi# 
                          
                          [/FONT]
                          This is a single boot KDE neon install. WHy do I have both directories?

                          Comment


                            #14
                            Originally posted by mr_raider View Post
                            Alright so why does my EFI look like this:

                            Code:
                            [FONT=monospace]root@hpelitebook725g3:/boot/efi# tree
                            [COLOR=#5454FF][B].[/B][/COLOR]
                            └── [COLOR=#5454FF][B]EFI[/B][/COLOR]
                               ├── [COLOR=#5454FF][B]BOOT[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]BOOTX64.EFI[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]fbx64.efi[/B][/COLOR]
                               │   └── [COLOR=#54FF54][B]mmx64.efi[/B][/COLOR]
                               ├── [COLOR=#5454FF][B]neon[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]BOOTX64.CSV[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]grub.cfg[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]grubx64.efi[/B][/COLOR]
                               │   ├── [COLOR=#54FF54][B]mmx64.efi[/B][/COLOR]
                               │   └── [COLOR=#54FF54][B]shimx64.efi[/B][/COLOR]
                               └── [COLOR=#5454FF][B]ubuntu[/B][/COLOR]
                                   └── [COLOR=#54FF54][B]grub.cfg[/B][/COLOR]
                            
                            4 directories, 9 files
                            root@hpelitebook725g3:/boot/efi# 
                            
                            [/FONT]

                            This is a single boot KDE neon install. WHy do I have both directories?
                            Ooh ooh ooh I know this one:

                            https://community.kde.org/Neon/Insta...tion_June_2016

                            Ubiquity installs the EFI files into /boot/EFI/efi/neon on the installed system but the path "efi/ubuntu" is hardcoded in two places. The fwup efi binary which is used for firmware updates installs to efi/ubuntu and the Grub efi binary is hardcoded to look in efi/ubuntu for grub.cfg so a patch in ubiquity copies our grub.cfg into efi/ubuntu (this patch need to be applied manually in the archive as the package is native not quilt). This has the downside that any changes in the ubuntu grub.cfg will get removed, similarly a later Ubuntu install will wipe the neon grub.cfg but as the code to set up grub.cfg is the same for both having a custom config which differs is a very niche setup.

                            Now, my Neon upgraded to 20.04 has this now (including WIndows 10):

                            Code:
                            root@dohbuoy-FLEX-15IIL:/boot/efi# tree
                            .
                            ├── EFI
                            │   ├── BOOT
                            │   │   ├── BOOTX64.EFI
                            │   │   ├── fbx64.efi
                            │   │   └── mmx64.efi
                            │   ├── neon
                            │   │   ├── BOOTX64.CSV
                            │   │   ├── grub.cfg
                            │   │   ├── grubx64.efi
                            │   │   ├── mmx64.efi
                            │   │   └── shimx64.efi
                            │   └── ubuntu
                            │       ├── BOOTX64.CSV
                            │       ├── grub.cfg
                            │       ├── grubx64.efi
                            │       ├── mmx64.efi
                            │       └── shimx64.efi
                            └── System Volume Information
                                ├── AadRecoveryPasswordDelete
                                └── ClientRecoveryPasswordRotation
                            
                            7 directories, 13 files
                            I have not done a fresh 20.04 Neon, which is using Calamares, which probably does not have this hard-coding. However, since Neon is using Ubuntu's packaging for all this, some Ubuntu structure may be a bit required. I have no clue, really.

                            I do have Win 10 on a separate SSD, including its own efi partition.
                            Last edited by claydoh; Jul 28, 2020, 12:35 PM.

                            Comment


                              #15
                              Mine with Neon, Kubuntu, and Win10:

                              Code:
                              /boot/efi/EFI
                              ├── Boot
                              │   ├── bootx64.efi
                              │   ├── fbx64.efi
                              │   └── mmx64.efi
                              ├── Microsoft
                              │   ├── Boot
                              │   │
                              ~~~~snip~~~~
                              ├── neon
                              │   ├── BOOTX64.CSV
                              │   ├── grub.cfg
                              │   ├── grubx64.efi
                              │   ├── mmx64.efi
                              │   └── shimx64.efi
                              └── ubuntu
                                  ├── BOOTX64.CSV
                                  ├── grub.cfg
                                  ├── grubx64.efi
                                  ├── mmx64.efi
                                  └── shimx64.efi

                              Please Read Me

                              Comment

                              Working...
                              X