BTRFS - lack of subvolume on Ubuntu and Debian?

[jfabernathy]

I've been experimenting with BTRFS for over a year. I've seen a lot of strange things done to get it working with snapper, timeshift and manually snapshots. When I created my Kubuntu 22.04 LTS server I stuck to Manual snapshots and just created @ and @home. It seemed fine at the time. However I had to play with the installation files to configure those subvolumes while I did the install of Kubuntu.

Since then I have found that some Distros do automatically create subvolumes to match the snapshot software that they favor. Linuxmint 21 creates @ and @home if you choose BTRFS instead fo EXT4. EndeavourOS creates @ and @home, plus @log and @cache so that /var/log and /var/cache can be mounted and not change if a snapshot is restored.

However, I noticed that the Ubuntu family and Debian have not added any special subvolumes. I was playing with Debian 12 RC1 for bookworm and Ubuntu 23.04 in virtual machines to see if any of that changed. They still only have the toplevel subvoume and nothing else, but I also found that the latest Snapper now is okay with that. If you install snapper and snapper-gui, you find that the software now automatically takes pre and post install snapshots when APT does anything. All you have to do is one config command:

sudo snapper create-config /

So editing of the snapper config file for the root config is still advisable to adjust the number of hourly, daily, monthly snapshots, but not required.

I thought Ubuntu was ignoring BTRFS but with the work on snapper I see that it's much easier to use and setup. I particularly like one subvolume for / and /home because when you install apps somethings they put stuff all over include your home directory.

[oshunluvr]

Ubuntu family (which includes Linux Mint) only use @ and @home. If you want to see an wildly and unnecessarily complicated btrfs install, try openSuse, although it's been a few years since I installed it.

Personally, I prefer simple over complicated unless there's some real benefit. IMO, snapshots aren't a reason to have 10 different subvolumes. Snapshots only take up as much space as you allow - by letting them age. I snapshot @ when I'm about to install something that might break the system and a rotating seven day snapshot event and sunday backups have served me just fine. I doubt most users need or care about having 1 or 2 or some other number of subvolumes.

Backups are a different matter. Home backups are more important than system backups in my world - at least on my desktop PCs. My server has 24 subvolumes for the various media files, home backups for the local users, other storage, WWW, and Plex library data. The server takes regular snapshots and makes backups by a cron job script I wrote, as does my main desktop.

Good to hear that snapper is more usable. It was kind of a mess when new. Seems logical that it would be installed by default if one chooses BTRFS at install time.

Curious about what installation files you had to "play with" on a fresh Kubuntu install to make snapshots. I suppose you mean the the root file system isn't mounted by default? That's the first thing I do on a new install is create the mount for the root FS and make a snapshot folder in there. I've never used snapper or any other tools other than the one I've created like my Dolphin subvolume submenus.

[GreyGeek]

As oshunluvr says, when you install Kubuntu and select the manual partition of the disk, you assign @ to / and Kubuntu adds @home as /home, into which your home account is created. OpenSuse creates several subvolumes, including /home, /opt, /root, /srv, /tmp, /usr/local and /var. It also installs snapper, which has a Yast GUI interface. Snapper creates links to these accounts under /run, which makes them accessible as root and is the cause, IMO, of a LOT of problems. Setting up the snapper config files results in /.snapshots and /home/.snapshots being created. Unistalling snapper (or TimeShift) without FIRST removing all of the snapshots that have been created puts one in a world of hurt.

Kubuntu (Ubuntu) has, IMO, the best setup. If you installed Kubuntu on, for example, /dev/sda3 then when you sudo to root and issue "mount /dev/sda3 /mnt" you make /mnt the ROOTFS for BTRFS and a directory listing shows
/mnt/@
/mnt/@home

I merged @home into @ so I have only /mnt/@ for my system. It avoids making two subvolume snapshots, which for my use case is unnecessary.

I created /mnt/snapshots and use it to store my @YYYYMMDDHHMM snapshots, and use btrfs send -p to move copies of the snapshot to removable storage.

Also like oshunluvr does, I wrote my own script to create my snapshots and keep a set of 7, adding a new one and deleting the oldest every evening.

In order to use "btrfs send -p ..." a snapshot must be read only. Sometimes I create a rw snapshot in /mnt/ (so it won't interfere with my script) when I'm playing around with some code. While snapper allows one to change or roll back a specific file I can do the same using mc riding on my rw snapshot.

[jfabernathy]

Ubuntu family (which includes Linux Mint) only use @ and @home. If you want to see an wildly and unnecessarily complicated btrfs install, try openSuse, although it's been a few years since I installed it.

Curious about what installation files you had to "play with" on a fresh Kubuntu install to make snapshots. I suppose you mean the the root file system isn't mounted by default? That's the first thing I do on a new install is create the mount for the root FS and make a snapshot folder in there. I've never used snapper or any other tools other than the one I've created like my Dolphin subvolume submenus.

EDITED:
The mod I did was documented a while back at https://www.kubuntuforums.net/forum/...-after-install

The good news is you don't need that for snapper. However the new install of Ubuntu 23.04 didn't include an @home subvolume. That's not a problem for snapper, but is for Timeshift.

I'm going to test the latest Kubuntu 23.04 and see if it's any different from Ubuntu. Linux Mint 21.1 does create the @ and @home which is what Timeshift needs and they auto-installs it.

As long as your system will boot after a bad change, you can easily restore to a previous snapshot with either Timeshift or Snapper. Your can boot to a known good snapshot if your current one is hosed if you install grub-btrfs from github. Since Timeshift takes R/W snapshots that's usually not a problem unless you change something bad while booted to the snapshot.

Snapper does R-O snapshots and booting them requires an overlayfs that initramfs (mkinitcpio or dracut) understand how to load if your image is R-O. I've been playing a lot with this on rolling releases like Arch and EndeavourOS. It's pretty cool to see a none booting system boot from a snapshot and then you run a easy to use GUI app like btrfs-assistant and restore @ and @home with a few clicks of the mouse.

I couldn't imagine running a rolling release without that. It's fun to play with on development systems. Meanwhile my production server/workstation keeps chugging along with Kubuntu 22.04 in the back room.

[jfabernathy]

I just spun up a VM with Kubuntu 23.04 and it has both @ and @home subvolumes if you select BTRFS as your filesystem in the manual partitioning. It's strange to me that Ubuntu 23.04 didn't do that. So Kubuntu 23.04 can easily use either Snapper or Timeshift without any additional work during install. Just Manual partitioning and set FS type to BTRFS.

EDITED: I went back to Ubuntu 23.04 and spun up a completely new VM and it did indeed only have on 1 subvol and it was "/" not "@"

[Schwarzer Kater]

OT:

[…] If you want to see an wildly and unnecessarily complicated btrfs install, try openSuse, although it's been a few years since I installed it. […]

The reason why SUSE Linux Enterprise (and thereby openSUSE) does this is roughly answered in this 6 years old post: https://unix.stackexchange.com/quest...subvol-entries - although this is no current SUSE btrfs setup anymore.

The number of different subvolumes has been reduced a little bit during the past years AFAIK (at least this is what it can look nowadays for a desktop installation - no server):

UUID=xyzxyzxyz  /                       btrfs  noatime                       0  0
UUID=xyzxyzxyz  /var                    btrfs  subvol=/@/var                 0  0
UUID=xyzxyzxyz  /usr/local              btrfs  subvol=/@/usr/local           0  0
UUID=xyzxyzxyz  /srv                    btrfs  subvol=/@/srv                 0  0
UUID=xyzxyzxyz  /root                   btrfs  subvol=/@/root                0  0
UUID=xyzxyzxyz  /opt                    btrfs  subvol=/@/opt                 0  0
UUID=xyzxyzxyz  /home                   btrfs  subvol=/@/home                0  0
UUID=xyzxyzxyz  /boot/grub2/x86_64-efi  btrfs  subvol=/@/boot/grub2/x86_64-efi  0  0
UUID=xyzxyzxyz  /boot/grub2/i386-pc     btrfs  subvol=/@/boot/grub2/i386-pc  0  0
UUID=xyzxyzxyz  /.snapshots             btrfs  subvol=/@/.snapshots          0  0
UUID=zzz        /boot/efi               vfat   utf8                          0  2
UUID=xxxyyyzzz  swap                    swap   defaults                      0  0​

You also have to keep in mind that those setups are originally designed for their major enterprise customers and not for the run-of-the-mill desktop user…

[jlittle]

IMO it's best to tailor a system's subvolumes to match the backup strategy for the sets of data in the system. (I'm very slightly obsessive with backups compared to most Linux users.) btrfs send/receive encourages good backup practices because it's so quick.

If your backup strategy has not been thought through, maybe all that is important is in the cloud, or there's nothing important, or something, @ and @home will do fine, or if there's no data, just have @.

For my work laptop this has meant adding @cache, @scratch, @appimages and @workdrive, to @ (renamed to @r) and @home (renamed to @h). ~/.cache is linked to somewhere in @cache; that's nearly 2 GB of rapidly changing data I don't back up.

[jfabernathy]

IMO it's best to tailor a system's subvolumes to match the backup strategy for the sets of data in the system. (I'm very slightly obsessive with backups compared to most Linux users.) btrfs send/receive encourages good backup practices because it's so quick.

If your backup strategy has not been thought through, maybe all that is important is in the cloud, or there's nothing important, or something, @ and @home will do fine, or if there's no data, just have @.

For my work laptop this has meant adding @cache, @scratch, @appimages and @workdrive, to @ (renamed to @r) and @home (renamed to @h). ~/.cache is linked to somewhere in @cache; that's nearly 2 GB of rapidly changing data I don't back up.

All my systems put there data that is important on a NAS directly, no locally editing. All hobby stuff that's not that important is backed up with Deja Dup daily to the same NAS. Timeshift or Snapper depending on the system do a rolling ~5 hourly snapshots and 7 rolling snapshots all locally. These are for when you do something that you immediately regret you can restore a snapshot with a few clicks of the mouse. No remembering how and where your snapshots are stored and how to manually restore. I also have snapshotting set to happen anytime APT runs. This is included with snapper automatically when installed on Kubuntu or Ubuntu 23.04. These snapshots have to be manually deleted when not needed.

My NAS is complete backed up to a Cloud service every night. This works for me and has been tested because of screw-ups and hardware failure, so it works for me.

[oshunluvr]

I did just read that 23.04 has a new installer. I wonder if the lack of a @home subvolume is an intentional change or an oversite.

[claydoh]

I did just read that 23.04 has a new installer.

Only for Ubuntu, currently. Kubuntu is still using Ubiquity. Lubuntu actually switched to Calamares with 22.04.

I just did as test install, and setting up btrfs was identical to how it always has been. You manually create a btrfs partition, set as "/", and it sets up both @ and @home.

[jfabernathy]

I did just read that 23.04 has a new installer. I wonder if the lack of a @home subvolume is an intentional change or an oversite.

It is odd that Ubuntu 23.04 installer creates absolutely no subvolumes. Not even @. If you do

btrfs su list /

you get no output.

However, if you install Kubuntu 23.04 you get @ and @home.

There have been some review of Ubuntu 23.04 on Youtube that have trashed the new installer. It will not effect me as I don't like Gnome DE and will be using Kubuntu.

I've done some more experimenting with Kubuntu 23.04 and both Timeshift and Snapper.

Timershift once installed is seamless. You can create snapshots of @ and include @home, which is key for me since I use snapshots primarily to recover from a bad install. As a test I installed gnome-mahjongg and then restored back to a snapshot taken prior to the sudo apt install... I noticed that the program was gone but in the KDE menu, there was still an icon for the game but it of course didn't work. Restoring @home at the same time solves that. So I setup Timeshift to always include @home with @. If you want automatic snapshots prior to installing anything with APT you need a 3rd party app call timeshift-autosnap-apt. You have to get it from github.

Once Snapper is installed you just need to create and configure the root and home config files. I find the restore process overly complicated for snapper without a great 3rd party app from gitlab. It's btrfs-assistant. It handles all the configuration of snapper as well as making and restoring snapshots.

I looked back on my production system at the scripts if use for daily snapshots and send/receive of those snapshots and could not remember how they worked. Thank goodness I documented that here.https://www.kubuntuforums.net/forum/...ential-backups

[claydoh]

It is odd that Ubuntu 23.04 installer creates absolutely no subvolumes. Not even @. If you do

Completely different installers. So either the new one has a bug, or there was a design change on purpose there, maybe. I suspect that being an initial release, on an interim Ubuntu release, it simply hasn't been implemented, and/or they decided to leave it a manual setup for now. There is no mention of this I can see on the installer's github tracker.

[oshunluvr]

The majority of BTRFS benefits revolve around subvolumes. If they did that on purpose, it was a real dumb move and a huge disservice to the users.

Personally, I'd report it as a bug.

[GreyGeek]

I just spun up a VM with Kubuntu 23.04 and it has both @ and @home subvolumes if you select BTRFS as your filesystem in the manual partitioning. It's strange to me that Ubuntu 23.04 didn't do that. So Kubuntu 23.04 can easily use either Snapper or Timeshift without any additional work during install. Just Manual partitioning and set FS type to BTRFS.

EDITED: I went back to Ubuntu 23.04 and spun up a completely new VM and it did indeed only have on 1 subvol and it was "/" not "@"

Just to make sure I understand: Ubuntu 23.04 abandons @ and @home and just creates "/" while Kubuntu 23.04 does create @ and @home, AND, under Ubuntu that makes /home automatically created as a directory under / and not a subvolume. Right?

Question: Would I be able, as root, to mount the btrfs formatted partition to /mnt and work with "/"? IOW, can I mount the rootfs under Ubuntu's 23.04 regime?

Being able to mount a BTRFS partition, say /dev/sda3, to /mnt as root and expose both @ and @home as /mnt/@ and /mnt/@home, making /mnt the "rootfs", is a feature of Kubuntu that is critical, IMO. To boot to a previous snapshot all I have to do as root is
mv /mnt/@ /mnt/@old
btrfs su snapshot /mnt/snapshots/@YYYYMMDDHHMM /mnt/@
and reboot.
I can also do that by booting a LiveUSB, mounting that drive and doing the same.
Otherwise, the rootfs is hidden and inaccessible by the user, requiring a rollback be done via software which may or may not expose all of btrfs's features.

IF Kubuntu, which is based on Ubuntu, eliminates @ and @home, and adopts making the boot subvolume just '/' then that would force me to look for another distro.

[jfabernathy]

Just to make sure I understand: Ubuntu 23.04 abandons @ and @home and just creates "/" while Kubuntu 23.04 does create @ and @home, AND, under Ubuntu that makes /home automatically created as a directory under / and not a subvolume. Right?

Question: Would I be able, as root, to mount the btrfs formatted partition to /mnt and work with "/"? IOW, can I mount the rootfs under Ubuntu's 23.04 regime?

If you look at the fstab on Ubuntu 23.04 they mount the root file system without any subvolume i.e.:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/vda2 during curtin installation
/dev/disk/by-uuid/705003dc-e242-4e8a-9b59-7559da8e702e / btrfs defaults 0 1
# /boot/efi was on /dev/vda1 during curtin installation
/dev/disk/by-uuid/C100-174B /boot/efi vfat defaults 0 1
​

So the first subvolume that got created on my system was when Snapper got installed and it created /.snapshot

EDITED:
To contrast, Kubuntu 23.04 installs this fstab:

# / was on /dev/vda3 during installation
UUID=6c8023a5-ebf3-4583-83a2-296432ebfe43 / btrfs defaults,subvol=@ 0 1
# /boot/efi was on /dev/vda1 during installation
UUID=26E1-867D /boot/efi vfat umask=0077 0 1
# /home was on /dev/vda3 during installation
UUID=6c8023a5-ebf3-4583-83a2-296432ebfe43 /home btrfs defaults,subvol=@home 0 2
# swap was on /dev/vda2 during installation
UUID=5ed2033a-459e-4fda-ba01-ab26e157621a none swap sw 0 0
​