BTRFS for new users. Why you should consider it.

[GreyGeek]

And when Oshunluver says he “re-snapshots the snapshot” he’s taking a rw snapshot of a ro snapshot so that when he makes it a new @ or @home or whatever, it can be read and written to.


Sent from my iPhone using Tapatalk

[oshunluvr]

This confuses me; can't quite wrap my head around it. As I'm "visually oriented"; I need to 'see' things (actual, ideas, concepts, etc). So let me ask this: I take a snapshot. As you say, it is "initially empty". I immediately delete EVERYTHING except the created snapshot (this is a mental exercise, so go with me here). What good is the snapshot now? I can't recover from my self-inflicted gun shot with it, can I? I'm expecting the answer is "No, you cannot."

To be technically correct, snapshots do take space, but it's just metadata space, not file space. Just like when you delete a file from an EXT4 file system, the file isn't actually deleted. The space it occupies is marked as available and the metadata (the part that your file system uses to locate the file) is deleted. BTRFS works the same way.

Snapshots are not copies of files. The files are already there. It copies the metadata. So a "pointer" to a file now exists in two places. If you "delete" a file from subvolume A, the metadata in that subvolume is deleted, but subvolume B remains untouched so the file is still there in B.

So to your example Snowhog:

Subvolume A contains 1000 files using 10GB.
A snapshot of A is made and it's named subvolume B.
All 1000 files that are in A are also in B.
The used space on your drive does not change - still 10GB in use.
You delete subvolume A.
All 1000 files that formerly were in A are still in B.
The used space on your drive does not change - still 10GB in use.

The changes occur when you change a file that is part of a snapshot. So given that A and B still exist:

You delete a file from subvolume A of 500MB.
Subvolume A now contains 999 files totaling 9.5GB.
All 1000 files that formerly were in A are still in B.
The used space on your drive does not change - still 10GB in use.
You add 10 files totaling 2GB to subvolume B.
Subvolume A still contains 999 files totaling 9.5GB.
Subvolume B now contains all the original 1000 files plus the 10 new ones and totals 12GB.
The used space on your drive grows to 12GB.

As you add and subtract files from each subvolume, they expand and contract accordingly, but the total space occupied on the file system (remember a snapshot must be co-located on the same file system as it's source) is the total amount of SHARED files (not two times the amount, just once) plus the total amount of DIFFERENT files.

In the EXT4 world, if you copied an entire partition, they have no relationship to each other. Therefore, the amount of used space doubles. Snapshots share filesystem space until a difference is introduced into one or both of the subvolumes.

A real world example:

Prior to a new kernel install, take a snapshot.
Zero bytes added to the file system.
Install the new kernel and delete an old kernel.
New kernel adds 100MB and old kernel removes 90MB.
The file system grows by 100MB. It does not reduce by 90MB because the old kernel is still alive and well in the snapshot.
All is well? - Delete the snapshot. file system shrinks by 90MB.
All is not well? - Reboot to the snapshot and delete the source subvolume, file system shrinks by 100MB.

Deleting the snapshot removes the old kernel from the file system. Deleting the source subvolume removes the new kernel. ALL the other files remain in place and unchanged.

Does this help at all?

[oshunluvr]

If you do a technical test of file system speeds, BTRFS is equivalent in some areas and slightly slower in some others when compared to EXT4. If performance is your only goal, then you would likely be using several different file systems as they all have their strengths and weaknesses. The point of BTRFS isn't speed, but ease of use and expanded functionality.
The hours I save not making or restoring backups or doing re-installs when I do something stupid (read the first post in this thread ) more than make up for any milliseconds lost to transfer rates. Honestly, with a high-end machine and an SSD, I doubt there's any noticeable difference at all in real-world use.

Let me make my position clear:
There simply is no other file system that is as easy to use as BTRFS, period.
There is no other file system with the features that BTRFS has, period.

If simple and easy-to-use isn't what you're looking for, by all means use something else. There are plenty to choose from and Linux is all about choice.

As far as snapshots go, as GG said - they take zero space and grow as they age. If you have a small SSD, you have to monitor your usage more closely. @Here, just this week, I did some housekeeping and deleted 14 snapshots of varying ages of my install and home subvolumes. The oldest was 3 months or so old. I recovered 10GB of space from 14 snapshots, so really not a lot of space was used. By way of comparison, my @ subvol is 17.3GB and my home is 197.6GB. My habit is to keep 3 once-daily snapshots and weekly send 2 backups onto other devices. This is done in the background via a cronjob.

My roll-back procedure is to boot to another install (or liveUSB), mount the location that contains my install subvolume and it's snapshots, rename the failed install subvolume, re-snapshot the snapshot I want to roll-back to to the correct bootable subvolume name, and reboot. Takes a minute or less. To explain the naming thing just a bit further: My install subvolume is @KDEneon. My snapshots are @KDEneon_YYMMDD_HHMMSS. That way I know when they were taken. So if @KDEneon fails to boot, I rename it @KDEneon_bad, take a snapshot of @KDEneon_YYMMDDHHMMSS as @KDEneon, and reboot. Couldn't be easier or faster. Also, to roll-back from a non-failed install, you don't have to boot into another install first - just rename the subvolumes and reboot.

An EXT4 user wanting the same functionality would have to backup to another partition or device and then restore from that partition or device. Anyone do that every day? I doubt it. I snapshot anytime I think I might be doing something dangerous and daily just because. Obviously, a snapshot isn't going to save you from a failed drive. That's why I also do a weekly backup to two other drives. That too, happens automatically (via a cron activated script) while I'm working with no noticeable loss in usability of the system. I challenge anyone to find an easier way to do this.

[Snowhog]

A snapshot is initially empty but...

This confuses me; can't quite wrap my head around it. As I'm "visually oriented"; I need to 'see' things (actual, ideas, concepts, etc). So let me ask this: I take a snapshot. As you say, it is "initially empty". I immediately delete EVERYTHING except the created snapshot (this is a mental exercise, so go with me here). What good is the snapshot now? I can't recover from my self-inflicted gun shot with it, can I? I'm expecting the answer is "No, you cannot."

[GreyGeek]

A snapshot is initially empty but as a file is added, changed or deleted the original is copied to the snapshot.
So, it gets larger over time. A read only snapshot is like a save-point. Those are the ones you want to use as a roll back.

My HD’s are 5400 RPM and I can’t see any performance degradation over EXT4.

[claydoh]

When distros pick it as the default, maybe with user-friendly aids to utlize the features, then I probably will use it. Then again, I may just be lucky as I have not had to do a reinstall of an OS other than replacing old, dead spinning drives in a loooong long time (other than for fun or out of boredom)

As a side note:
https://help.ubuntu.com/community/UbuntuReinstallation
I have used this method in the past, iirc probably back in the 8.10-12.04 era when I was running pre-releases as my daily driver. It did work. Dunno why I forgot this.

How much space do snapshots take, for those who have tiny SSDs?
What about performance on spinning 5400 rpm drives?

[GreyGeek]

...
My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

...

Btrfs has capabilities to expand or contract its FS size. Being a COW system it would be important, as I understand it, to make the virtual HD a FIXED size even on EXT4 so that VirtualBox doesn't expand or contract it as your usage varies. Inside the guest OS that uses Btrfs as the FS you can contract the size but not expand it, unless you've previously contracted it.

btrfs filesystem resize [<devid>:][+/-]<size>[kKmMgGtTpPeE]|[<devid>:]max <path>

The resize command does not manipulate the size of underlying partition. If you wish to enlarge/reduce a filesystem, you must make sure you can expand the partition before enlarging the filesystem and shrink the partition after reducing the size of the filesystem. This can done using fdisk(8) or parted(8) to delete the existing partition and recreate it with the new desired size. When recreating the partition make sure to use the same starting partition offset as before.

Growing is usually instant as it only updates the size. However, shrinking could take a long time if there are data in the device area that’s beyond the new end. Relocation of the data takes time.

[oshunluvr]

I'm sorry for not being clear.

My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

Meanwhile, I came across something related (?) to your caveat:from It’s 2016: and BTRFS could really be your next filesystem

I use BTRFS on virtual machines all the time, so yes you can use BTRFS in a VM and learn about it there. Note that the current Kubuntu installer will automatically create a swap file at installation time if there is not a swap partition present. A bug has been filed on this behavior, but you should be aware of it. Either manually create a swap partition on your virtual disk at installation or let it do it's thing and when you boot into the VM the first time, turn swap off and edit /etc/fstab removing the swap file reference. I doubt you'll need swap or want in a VM anyway.

Once you get your new SSD, I suggest making it bootable, creating a swap partition, then use the remainder as a single BTRFS file system. You could do a fresh Kubuntu 18.04 install there and leave your current one in place as a backup to boot to if necessary. I have six distros and their homes installed to my single BTRFS file system (480GB total - 2x250GB SSDs in RAID0). I use two BTRFS file systems on two hard drives for backups. BTRFS allows you to "send" (copy) an entire subvolume to another BTRFS file system. The copies can even be made bootable so I can boot to two other drives if needed. My system automatically makes two backups weekly (one of my primary distro, one of my home) and send them to each of the two backup drives. I also take daily snapshots of my subvolumes and extras when I'm doing something I may want to back out of later. It's all pretty slick and almost totally hands-off at this point.

[chimak111]

Are you asking about a Virtual Machine? If so, yes the guest file system on it's virtual drive is independent of the hosts.

This brings up one caveat about BTRFS: If you use virtual drives on a BTRFS file system (the host), you must use FIXED size drives, not dynamic. The way BTRFS works can cause corruption of dynamically allocated virtual devices, which includes swap files. If you use BTRFS as your host file system, you must use a swap partition not a swap file.

I'm sorry for not being clear.

My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

Meanwhile, I came across something related (?) to your caveat:

As in any technology, BTRFS is not perfect. For example, it suffers when there are heavy write activities in the middle of an existing files, so probably it’s not the best candidate for virtualization (the virtual disks are updated in-place at each write).

from It’s 2016: and BTRFS could really be your next filesystem

[Bings]

Words can't do snapshots and send & receive justice. IF you have the storage space I'd recommend using Oshunluver's setup where several distros are installed on one partition.

Right now, I have one 128gb SSD and 1TB HDD. The SSD I dual boot Kubuntu and Windows 7 on. Windows takes up so much space though that I put the /home folder on a partition on the HDD. I want to have a Ubuntu Studio install and also an install where I can see how well I can play the games I have on steam with. So, the plan is to get a new SSD drive, use BTRFS and have the linux installs on there. Use the HDD for media and game storage.

[oshunluvr]

Silly question but ... can a guest Linux OS be BTRFS if the host OS (Kubuntu 16.04) is ext4?

Are you asking about a Virtual Machine? If so, yes the guest file system on it's virtual drive is independent of the hosts.

This brings up one caveat about BTRFS: If you use virtual drives on a BTRFS file system (the host), you must use FIXED size drives, not dynamic. The way BTRFS works can cause corruption of dynamically allocated virtual devices, which includes swap files. If you use BTRFS as your host file system, you must use a swap partition not a swap file.

[chimak111]

Silly question but ... can a guest Linux OS be BTRFS if the host OS (Kubuntu 16.04) is ext4?

[GreyGeek]

I plan to use BTRFS for any future installs, Which will most likely be going up to 18.04 in a couple of months. I like the sound of the snapshot stuff.

Words can't do snapshots and send & receive justice. IF you have the storage space I'd recommend using Oshunluver's setup where several distros are installed on one partition.

[Bings]

I plan to use BTRFS for any future installs, Which will most likely be going up to 18.04 in a couple of months. I like the sound of the snapshot stuff.

[vinnywright]

+1 ,,,,,my 1TB storage drive uses it ,,,,and about 300GB of my 500GB system drive dose currently.

VINNY