Can't boot a btrfs partition

[doctordruidphd]

I boot my system from oneiric on /dev/sda1. I have an experimental 12.04 system on /dev/sda2, both are formatted as ext4. I want to set up /dev/sda3 as a btrfs system just to play with it, so I formatted the /dev/sda3 partition with btrfs, then rsync'd my system on sda2 to it, changed its fstab to reflect the new / and format. I can mount and read/write the experimental system from oneiric Then ran update-grub on the oneiric system, and rebooted.

Problem is grub is unable to load the kernel from /dev/sda3. It keeps reporting "file not found" when it tries to load the kernel.

Is this still a problem with grub; thought it was fixed by now? Is there some other step I need to take to boot a btrfs partition from grub on an ext4? One possible issue: I notice that btrfs systems have 2 UUID's, a main and a sub. fstab wants the main UUID to mount it. That's also what is found in grub.cfg.

Not sure why it isn't loading.

[james147]

Are you useing compression at all? I know grub2 cannot handle that yet... But it should be able to boot off a btrfs filesystem now (I am currently using two, a compressed / and an uncompressed /boot). Also, are you using subvolumes at all?

[doctordruidphd]

Not using compression at all.
As far as the filesystems are concerned, I am not (knowingly) using subvolumes, but blkid returns the following:

/dev/sda3: UUID="34ac162d-4165-41c9-bf5e-e047fc08ebdd" UUID_SUB="ed8dc5aa-ca78-4564-aa47-a7ddcb5ea12d" TYPE="btrfs"

Should grub be looking for the UUID or the UUID_SUB? Looks like fstab wants the UUID.

Could you post the section of your grub.cfg that boots the btrfs system? Thanks.

[Snowhog]

Did you create separate / and /boot partitions on the btrfs formatted installation?

[james147]

Should grub be looking for the UUID or the UUID_SUB? Looks like fstab wants the UUID.

I just used the UUID, never had to use the UUID_SUB.

Could you post the section of your grub.cfg that boots the btrfs system? Thanks.

menuentry 'Arch Linux, with Linux linux' --class archlinux --class gnu-linux --class gnu --class os {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod btrfs
        set root='hd0,gpt2'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  216c8ae4-b14b-4f3c-9382-b21ed576d641
        else
          search --no-floppy --fs-uuid --set=root 216c8ae4-b14b-4f3c-9382-b21ed576d641
        fi
        echo    'Loading Linux linux ...'
        linux   /vmlinuz-linux root=/dev/disk/by-uuid/b4705ae4-7cda-4ec4-9c57-d9ad2b41b4b5 ro rootflags=subvol=__active  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /initramfs-linux.img
}

This was generated by grub-mkconfig. Also I am using a gpt partition table, though it should work with msdos tables.

Edit: And my root file system is in __active subvol and compressed and boot in on a separate partition (also btrfs) without subvolumes or compression.

I have got this to work with only one partition and subvolumes (where boot is not a subvol but root is)

[doctordruidphd]

Did you create separate / and /boot partitions on the btrfs formatted installation?

I created the btrfs partition by formatting /dev/sda3 with gparted. There aren't a whole lot of options -- none for sub-filesystems, etc. Maybe it's not too well supported by gparted at this point. I did not specify any subvolumes, it just did what it did, which is what the blkid command shows.

james147: thanks for posting that. The entries in my grub.cfg don't look anything like that. I'll try some experimenting.

Edit: looking at this, the "rootflags=subvol" part is missing in my grub.cfg, and since I don't have a volume label (and evidently can't add one at this stage of development) then I'm not sure how to set it up. Maybe this just has to wait for better support?

[james147]

I created the btrfs partition by formatting /dev/sda3 with gparted. There aren't a whole lot of options -- none for sub-filesystems, etc.

Subvolumes are created after the filesystem is created, so gparted dosn't care about them:

Just so you know:
You can create a subvolume with "btrfs subvolume create <path>" (as user or root depending on where you want to create it)
You can snapshot a subvolume with "btrfs subvolume snapshot <subvol> <path>" (snapshots are basically subvolumes but start with the same content of the subvol specified... very handy for being able to roll back things)

I would recommend reading "man btrfs" for more info on the commands you can use to control btrfs at some point

but this isn't your issue as you haven 't created any...

james147: thanks for posting that. The entries in my grub.cfg don't look anything like that. I'll try some experimenting.

Mine was generated by grub-mkconfig (basically what update-grub calls) so I am not sure why its not creating the relevant parts :S

Edit: looking at this, the "rootflags=subvol" part is missing in my grub.cfg

This is NOT the issue, the just tells grub that my root is actually in the subvol __active and not the root of the btrfs drive. Since you don't have subvols you don't need it.

my disk layout is:
/dev/sda3 btrfs
/__active/<root of my system>
/__snapshots/<snapshots of subvolumes>
/dev/sda2 btrfs
/<boot directory>

where /dev/sda3 subvol __active is mounted to /
and /dev/sda2 is mounted to /boot
(and /dev/sda3 is mounted to /var/lib/btrfs/root just to confuse du and find YEY circular filesystems)

Maybe this just has to wait for better support?

The support is there... Just you are setting it up in an weird way... You could try the installer, it has an option for btrfs now.

As far as I know ubuntus schema for btrfs root is similar to mine but with @ instead of __active and @_snapshots for the snapshots. I don't know how important these are, but I don't think its that strict.
(note that if you want that setup without a separate boot partition /boot needs to be in the root of the btrfs (ie not a subvol) and bind mounted back to where it should be).

As for your setup I am not sure why its not working... Just to confirm you have a single partition with no subvolumes on it? This should be the simplest btrfs setup.

The only thing i can think of thats different (other then I am useing archlinux... ) is that I am installing grub from within the system, not trying to link it via another system... what happens if you chroot to the btrfs and install grub and update-grub from that? (Note you can restore grub from a livecd if needed)

Other then that it might be better to just install it form the disk? The installer supports btrfs.

[doctordruidphd]

Looks like, for some reason, the forum didn't post my last response...

Yes, the btrfs partition was formatted with no subvolumes, other than what gparted (and mkfs.btrfs on the newly formatted partition) did. The mount point for /dev/sda3 (the btrfs system) on Oneiric is /media/experimental, and "sudo btrfs subvolume list /media/experimental" returns nothing. Oneiric is able to mount, read and write the partition just fine.

Anyway, I reformatted the btrfs partition, re-rsynced the file system, changed fstab appropriately, and still no go.

If I boot with the following menu entry, I get, as I would expect, "unknown file system":

menuentry "Kubuntu Experimental" {
insmod ext2
gfxpayload=1680x1050
set root='(hd0,3)'
linux /vmlinuz root=/dev/sda3 ro
initrd /initrd.img
}


If I add in btrfs, I get the "file not found" error, even if I change the root= to the appropriate uuid:

menuentry "Kubuntu Experimental" {
insmod ext2
insmod btrfs
gfxpayload=1680x1050
set root='(hd0,3)'
linux /vmlinuz root=/dev/sda3 ro
initrd /initrd.img
}

That says to me that grub is reading the file system, but unable to locate the required files. It's not at all clear to me why, as it finds everything just fine on ext4 systems. Maybe ubuntu's version of btrfs (and/or grub) is expecting subvolumes or something. I guess if/when I get the time, I will install a cd into a vm and see if i can figure out what makes it tick, then try migrating the system again. At this point I'm just banging on a brick wall with it.

[oshunluvr]

Not an answer: but one of the real advantages of btrfs (there are many IMO) is space_cache and compress options, which I believe are not yet supported by grub. I use btrfs on two computers daily, but both have separate /boot partitions. I tried to install to a btrfs partition with 11.10 awhile back but Ubiquity in 11.10 has a btrfs bug so I quit trying.

I guess my point is if you're looking to play with btrfs why not make a boot partition?

Check out this page if you haven't already http://www.funtoo.org/wiki/BTRFS_Fun lots of good info.

Also AFAIK the sub_uuids for btrfs partitions are used by the btrfs tools to tell the partitions apart in a multi-partition btrfs filesystem. You would want to use the uuid in fstab or grub. Here's one of mine:
/dev/sda8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="e520735b-5de8-409a-aeb1-bf8c17105cde" TYPE="btrfs"
/dev/sdb8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="674756a5-17b9-4197-be5c-131b83cb42d8" TYPE="btrfs"
/dev/sdc8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="7103e4d0-5332-42c1-9596-2b2b9bb335b6" TYPE="btrfs"

[james147]

space_cache and compress options, which I believe are not yet supported by grub.

As far as I know they are both supported by the latest grub (not sure about the one in kubuntu though) but the compression support is limited to one type (cannot remember which), the better lzo compression is not supported yet

I use btrfs on two computers daily, but both have separate /boot partitions.

This is what I generally do as well, but it should work on a single partition if there is no compression or subvolumes...

I am guessing there is some config missing, but I am not familiar enough with ubuntus startup process to know where to start looking... on arch I have to enable btrfs support in the initramfs image as well as the arch start up scripts to get it to work... (but it fails at initramfs or later if I don't)... So I am really not sure why its failing before that.

Check out this page if you haven't already http://www.funtoo.org/wiki/BTRFS_Fun lots of good info.

Looks like an interesting read, thanks for the link

Also AFAIK the sub_uuids for btrfs partitions are used by the btrfs tools to tell the partitions apart in a multi-partition btrfs filesystem. You would want to use the uuid in fstab or grub.

Yeah, I am seeing the same thing here on my multi device btrfs partition.

[doctordruidphd]

OK, thanks for the info. I followed all of the advice and info on the ubuntu btrfs wiki, and it's still no go. Meaning, I can't boot a system on a btrfs partition from oneiric --haven't tried from precise as yet. I did make the @ subvol, put the system in there, fix fstab & update-grub, but I still keep getting the same, by now tedious and unhelpful, "file not found" message.

If someone actually has this working -- namely, able to boot an ubuntu system on a btrfs partition from a non-btrfs partition (which should, as far as booting is concerned, be the same thing as a separate boot partition) I'd sure like to see a blkid, fstab, and grub.cfg on how it's done. Until then, unless there is a sudden flash of insight, I think I will just put this to sleep.

Thanks for the links, will check them out. The btrfs file system works fine for storing data, just can't boot.

[oshunluvr]

If someone actually has this working -- namely, able to boot an ubuntu system on a btrfs partition from a non-btrfs partition

I guess I'm unclear on exactly what you're attempting. I boot to a /boot (ext2) partition with the OS (11.04) on a btrfs multi-disk partition. I did a clean install from the Kubuntu LiveUSB after pre-formatting the btrfs array. I assume your troubles are with the grub.cfg?

Here's some data from my computer:

grub.cfg stanza:
menuentry 'Ubuntu, with Linux 2.6.38-13-generic' --class ubuntu --class gnu-linux --class gnu --class os {
recordfail
set gfxpayload=$linux_gfx_mode
insmod part_msdos
insmod ext2
set root='(/dev/sdc,msdos2)'
search --no-floppy --fs-uuid --set=root ebf2fb2e-1215-4a40-828c-053a18ffb398
linux /vmlinuz-2.6.38-13-generic root=UUID=723e5746-2458-458b-9242-3289ab7b58de ro rootflags=subvol=@ quiet splash vmalloc=512MB vga=795 vt.handoff=7
initrd /initrd.img-2.6.38-13-generic
}


applicable lines from fstab:
UUID=ebf2fb2e-1215-4a40-828c-053a18ffb398 /boot ext2 defaults 0 0
UUID=723e5746-2458-458b-9242-3289ab7b58de / btrfs defaults,space_cache,compress,subvol=@ 0 0

applicable lines from blkid:
/dev/sdc2: UUID="ebf2fb2e-1215-4a40-828c-053a18ffb398" TYPE="ext2"
/dev/sda8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="e520735b-5de8-409a-aeb1-bf8c17105cde" TYPE="btrfs"
/dev/sdb8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="674756a5-17b9-4197-be5c-131b83cb42d8" TYPE="btrfs"
/dev/sdc8: UUID="723e5746-2458-458b-9242-3289ab7b58de" UUID_SUB="7103e4d0-5332-42c1-9596-2b2b9bb335b6" TYPE="btrfs"

[doctordruidphd]

Thanks for the info.

Here is what I have on the disk:

/dev/sda1 ext4 kubuntu-oneiric
/dev/sda2 ext4 kubuntu-precise
/dev/sda3 btrfs
subvol=@ kubuntu-precise ## rsync'd from /dev/sda2
(other data partitions)
/dev/sdb1 ext4 aptosid
/dev/sdb2 ext4 kubuntu-LTS
/dev/sdb3 ntfs vista
(other data partitions)

The system boots from /dev/sda1. So I don't have a separate /boot, and I think that might get real messy with several different OS and versions.

I normally boot from a custom menu that runs ahead of 10_linux, so i don't have to scroll through all the kernels and options to find the system I want to start. boot.cfg has entires that look like this:

menuentry "Kubuntu Stable" --class ubuntu --class gnu-linux --class gnu --class os {
insmod ext2
gfxpayload=1680x1050
set root='(hd0,1)'
linux /vmlinuz root=/dev/sda1 ro text
initrd /initrd.img
}

menuentry "Kubuntu Testing" {
insmod ext2
gfxpayload=1680x1050
set root='(hd0,2)'
linux /vmlinuz root=/dev/sda2 ro
initrd /initrd.img
}

menuentry "Kubuntu Experimental" {
insmod btrfs
insmod ext2
gfxpayload=1680x1050
set root='(hd0,3)'
linux /vmlinuz-exp root=/dev/sda3 ro text rootflags=subvol=@
initrd /initrd.img-exp
}

The last entry is the one that isn't working. So I tried this: I copied the current vmlinuz and initrd.img to /dev/sda1/boot as vmlinux-exp and initrd.img-exp. I then modified the menu item as follows:

menuentry "Kubuntu Experimental" {
insmod btrfs
insmod ext2
gfxpayload=1680x1050
set root='(hd0,1)'
linux /boot/vmlinuz-exp root=/dev/sda3 ro text rootflags=subvol=@
initrd /boot/initrd.img-exp
}

Still says "file not found". I then tried removing the "insmod btrfs", and it did actually load the vmlinuz and initrd.img, though it choked at the point where it should have mounted / (as expected without btrfs support).

I dunno. I'm missing something very fundamental here.

[james147]

subvol=@ kubuntu-precise ## rsync'd from /dev/sda2

Don't think grub can handle /boot being side a btrfs subvolume, if you don't want a separate partition for boot then place /boot directly at the root of the btrfs drive (ie next to @) and bind mount it back in fstab (this will require you to mount the root btrfs volume somewhere as well)

something like:
/ btrfs subvol=@
/var/lib/btrfs-root btrfs subvolid=0 (id 0 is ALWAYS the root of the btrfs volume)
/boot /var/lib/btrfs-root bind (yey circular file system )

where the disk layout is:
/
@/<root of os>
boot/

Though an easier option might be a clean install form the installer.

lucky btrfs don't care if you recurse mounts like this

[oshunluvr]

As of 11.04 it is possible to use only btrfs file systems with the caveat that grub _MUST_NOT_ be installed to the boot sector of the btrfs volume containing /boot. As long as your booting via MBR, you can have /boot within /

Sorry, drdru - I just can't figure out what you're trying to do. By "The system boots from /dev/sda1" you mean you computer boots directly to a partition rather than the MBR or are you chainloading?

There are some btrfs bugs in 12.04. For example, with 12.04 you can't boot to a btrfs partition if /home is not on the same partition. It's very possible you're running into some odd bug. I know you've spent days on this already, but it would be interesting to see if 11.10 would successfully boot with the exact setup.