btrfs adventure with gentoo

[jlittle]

After recently boasting about how many distros I had in the same btrfs on a 240 GB SSD, I realized that they were all ubuntu derivatives, kind of watering down my point.

Last year I did a gentoo install to a partition on an old hard disc, to check something out. It was a vanilla gentoo, with a 4.19 kernel, but no DE or X11.

Now it is possible to do a gentoo install to btrfs subvolumes. With gentoo it's like building a house brick by brick, carrying every brick, beam, joist, glass pane et cetera to its place yourself, and if you know what you're doing you can use a btrfs at the appropriate times. But I didn't know what I was doing, I was following the step by step instructions in the wiki, pages of them (note to self, have two computers, or print them all out).

So, I created a subvolume, @gentoo, used cp -a to copy the gentoo root to it, fixed up the /etc/fstab, and copied a grub menu entry adding rootflags=subvol=@gentoo to the linux line.

No trouble, and it boots much more quickly.

[GreyGeek]

How "much more quickly"?

Did gentoo have btrfs as a root_fs that can be selected during install, or did you install it on an EXT4 partition and then copy it to your @gentoo subvolume?

[jlittle]

How "much more quickly"?

About 10 s compared to about 17 s.

Did gentoo have btrfs as a root_fs that can be selected during install

With gentoo there's no installer, or rather the user is the installer. It's all command line, and few scripts either. When it comes to file systems, for ext4 mkfs.ext4 is run, and for btrfs makefs.btrfs is run. The user might have to install btrfs-progs during the install, depending on what's included in the iso used.

... or did you install it on an EXT4 partition and then copy it to your @gentoo subvolume?

Yes, the original install was to ext4. I found it interesting, and so posted about it, that an install to a partition could be easily moved to a btrfs subvolume, even for a distro unrelated to Ubuntu. Now, for gentoo, I'm pretty sure it's easy to install to a btrfs subvolume directly if you know the commands to use, but other distros have graphical installers that may not do the subvolume thing like the *buntus.

[GreyGeek]

Thanks! Interesting!
A brtfs subvolume cannot contain another subvolume, which is why when you browse @ you will notice that /home is empty and @home is mounted to /home in fstab. So, are you using only one subvolume, @gentoo, and it contains the entire file hierarchy including / and /home?

[jlittle]

So, are you using only one subvolume, @gentoo, and it contains the entire file hierarchy including / and /home?

Yes, but only because I installed gentoo like that originally. I could have used a more elaborate subvolume set up as I was hacking the fstab anyway.

It's services like Timeshift and snapper that would need further work.

[oshunluvr]

Manjaro follows the same basic btrfs install layout as the *buntu do - @ and @home.

[mr_raider]

Thanks! Interesting!
A brtfs subvolume cannot contain another subvolume, which is why when you browse @ you will notice that /home is empty and @home is mounted to /home in fstab. So, are you using only one subvolume, @gentoo, and it contains the entire file hierarchy including / and /home?

But isn't the top level / in a btrfs partition always a subvolume?

[oshunluvr]

But isn't the top level / in a btrfs partition always a subvolume?

No. The top volume cannot be a subvolume. Correction; you're right it is technically a subvolume. I just checked that to verify. It's always "level 5" which makes it the main container. It is where subvolumes can reside but it can also hold regular files and folders. Subvolumes can be nested as well as the root (top) file system

One distro I played with a year or so ago would let you install to BTRFS but the installer didn't use subvolumes, so it was rather useless. No subvolumes, no snapshots. No snapshots, no backups.

[oshunluvr]

Thanks! Interesting!
A brtfs subvolume cannot contain another subvolume...

GG, why do you believe this? Subvolumes can reside inside other subvolumes. Here, I made a snapshot of @grub inside @KDEnenon1804:

[FONT=monospace][COLOR=#54FF54][B]stuart@office[/B][/COLOR][COLOR=#000000]:[/COLOR][COLOR=#54FFFF][B]/subvol[/B][/COLOR][COLOR=#000000]$ bt su list /[/COLOR]
ID 707 gen 1559252 top level 5 path @grub
ID 1740 gen 1559252 top level 5 path @KDEneon1804
ID 2321 gen 1559252 top level 1740 path @grub
[/FONT]

Note the "top level" of the second @grub is the ID for @KDEneon1804

From the BTRFS wiki:

Subvolumes can be created at any place within the file system hierarchy, and they can also be nested. Nested subvolumes appear as subdirectories within their parent subvolumes, similarly to the way a top-level subvolume presents its subvolumes as subdirectories. Deleting a subvolume is not possible until all subvolumes below it in the nesting hierarchy are deleted...

[oshunluvr]

Re. nesting subvolumes - one way this can be a benefit is to make mounting simpler, as the nested subvolumes are accessible when the parent subvolume is mounted.

One could easily unmount @home, delete the /home folder, snapshot @home to @/home, delete @home, and then remove the fstab line mounting @home and all would be well. You could still snapshot home separately.

One unanswered question I have is: If I have a subvolume nested this way, is it included in a snapshot of the parent subvolume? Hmm, off to a VM...

ADDED COMMENT - Ok, I went to a Manjaro VM and Kubuntu VM tried this with the same results;

1. Unmounted "@home"
2. Deleted the /home folder
3. Took a snapshot of "@home" as "/home"
4. Deleted "@home"

This resulted in a fully accessible home subvolume in the needed location /home.
From Kubuntu:

#btrfs su list /
ID 257 gen 95 top level 5 path @
ID 263 gen 96 top level 257 path home

I was also able to re-snapshot "home as "@home" no problem.

HOWEVER, when I snapshoted "@" the "/home" folder was shown in the snapshot, but it was totally empty. Odd behavior IMO. My expectation was it would either be there entirely of not at all. I did not expect it to leave behind a folder that previously didn't exist.

This seems like it would make using a subvolume this way a bit more difficult because you would have to remove the /home folder before you could re-construct (restore) your install completely.

[mr_raider]

Re. nesting subvolumes - one way this can be a benefit is to make mounting simpler, as the nested subvolumes are accessible when the parent subvolume is mounted.

One could easily unmount @home, delete the /home folder, snapshot @home to @/home, delete @home, and then remove the fstab line mounting @home and all would be well. You could still snapshot home separately.

One unanswered question I have is: If I have a subvolume nested this way, is it included in a snapshot of the parent subvolume? Hmm, off to a VM...

ADDED COMMENT - Ok, I went to a Manjaro VM and Kubuntu VM tried this with the same results;

1. Unmounted "@home"
2. Deleted the /home folder
3. Took a snapshot of "@home" as "/home"
4. Deleted "@home"

This resulted in a fully accessible home subvolume in the needed location /home.
From Kubuntu:

#btrfs su list /
ID 257 gen 95 top level 5 path @
ID 263 gen 96 top level 257 path home

I was also able to re-snapshot "home as "@home" no problem.

HOWEVER, when I snapshoted "@" the "/home" folder was shown in the snapshot, but it was totally empty. Odd behavior IMO. My expectation was it would either be there entirely of not at all. I did not expect it to leave behind a folder that previously didn't exist.

This seems like it would make using a subvolume this way a bit more difficult because you would have to remove the /home folder before you could re-construct (restore) your install completely.

i'm guessing this to prevent useless snapshots. If the user wants a /home to be snapshotted, he/she should specifically request it. You can also place swap files in subvolumes now as of kernel 5.3 I think. SO the last think you want is your swap file to be snapshotted

I've taken the habit of only creating level 5 subvolumes to avoid this confusion.

[oshunluvr]

i'm guessing this to prevent useless snapshots. If the user wants a /home to be snapshotted, he/she should specifically request it. You can also place swap files in subvolumes now as of kernel 5.3 I think. SO the last think you want is your swap file to be snapshotted

I've taken the habit of only creating level 5 subvolumes to avoid this confusion.

I just discussed this with one of the developers. He said it isn't intentional, just was an unintended bread crumb. He thinks they fixed it but didn't know for sure which kernel release got the fix or if it had been integrated yet. I'll probably make a bug report on it.

[GreyGeek]

GG, why do you believe this?

Hummm.... racking my brain ... I don't remember. I think it was way back when I was exploring BTRFS. I was playing with snapper and noticed that their snapshot hierarchy contained alternate layers of snapshots and directories. But, I couldn't say for sure. Just another item to add to the list of things I got wrong over my life.
Better go and correct the BTRFS posts in that subforum.

EDIT: I couldn't find any references in the BTRFS subforum about nested subvolumes, but I did clear up another error of mine. I thought I had been using BTRFS for almost 5 years. I found my first post about it on Jan 16, 2014. So, I've been using it for 6yrs and 8 months. Oshunluver goes back to Jan of 2011.

I guess I stunk up the forum with a brain fart. Sorry.

Subvolumes can reside inside other subvolumes. Here, I made a snapshot of @grub inside @KDEnenon1804:

[FONT=monospace][COLOR=#54FF54][B]stuart@office[/B][/COLOR][COLOR=#000000]:[/COLOR][COLOR=#54FFFF][B]/subvol[/B][/COLOR][COLOR=#000000]$ bt su list /[/COLOR]
ID 707 gen 1559252 top level 5 path @grub
ID 1740 gen 1559252 top level 5 path @KDEneon1804
ID 2321 gen 1559252 top level 1740 path @grub
[/FONT]

Note the "top level" of the second @grub is the ID for @KDEneon1804

From the BTRFS wiki:

That's why I could never get your multi-distro grub idea to work. I was using subdirectories and during boot grub could never find my distros.

[oshunluvr]

Hey, that's why we're here - to help ourselves learn. No stinkiness detected! LOL

GRUB still can't find other installs on a BTRFS fs. It isn't "smart" enough to search the subvolumes for boot info. My multi-boot BTRFS installs have to be manually managed using /etc/grub.d/40_custom. I just link to each installs' GRUB menu so they update their own GRUB menus when kernel updates happen.

[jlittle]

. My multi-boot BTRFS installs have to be manually managed using /etc/grub.d/40_custom..

Alternatively, I manually maintain a very simplified grub.cfg. Easier, and far more stable, IMO.