BTRFS for new users. Why you should consider it.

[GreyGeek]

BTRFS for new users. Why you should consider it.

*deletes brain fart*

[GreyGeek]

Ah, personal taste!
I always ro both my @ and @home snapshots, and send them to remote storage. If I need to rollback I take a snapshots of the backups with out using “-r”.


Sent from my iPhone using Tapatalk

[oldgeek]

OK, I tried the three steps and they worked, so I guess I did get it right. However, when I list the subvolume, I not only get @ and @home, but also two others: './' and '../'. What are those?

[oshunluvr]

Well, I decided to give btrfs a try and installed Kubuntu 18.04 in a virtual machine (QEMU/KVN) with btrfs. When I checked it out, I saw no difference on Dolphin from ext4. I guess I did something wrong. When it came to the installing part, I clicked on Manual and got the menu. I couldn't figure out what to do next, so I clicked on the device name and a new partition came up, with the same value as the original (20 gigas). I clicked on btrfs and let it continue. The result? No discernable difference. The btrfs install is a sub-partition of the original device partition--is that OK? I'm really interested in trying out btrfs before I commit myself to a new install, but I want to try it out first. What did I do wrong?

What should I see when I click on Dolphin is a btrfs system? Thank you in advance.

You won't see any difference at the user level at all. BTRFS allows mounting if the subvolumes as though they were partitions. Open a terminal and type "mount" and enter. You should see both "/" and "/home" are on the same partition. If you look at /etc/fstab, you will see the lines to mount "/" and "/home" are identical except there will be an option subvol=@ on the root line and subvol=@home for the home mount.

To view the subvolumes you need to mount the root volume with no "subvol=" option.

I'll assume your partition is sda1, if not substitute the correct partition device name. Open a terminal and;

Create a mount point:
sudo mkdir /subvol

Mount the root file system:
sudo mount /dev/sda1 /subvol

Look at the listing:
ll /subvol

You should see

@
@home

in the listing.

Take a snapshot:
sudo btrfs subvolume snapshot /subvol/@ /subvol/@_snap1

List the subvolumes again:
ll /subvol

You should see:

@
@home
@_snap1

[oldgeek]

Well, I decided to give btrfs a try and installed Kubuntu 18.04 in a virtual machine (QEMU/KVN) with btrfs. When I checked it out, I saw no difference on Dolphin from ext4. I guess I did something wrong. When it came to the installing part, I clicked on Manual and got the menu. I couldn't figure out what to do next, so I clicked on the device name and a new partition came up, with the same value as the original (20 gigas). I clicked on btrfs and let it continue. The result? No discernable difference. The btrfs install is a sub-partition of the original device partition--is that OK? I'm really interested in trying out btrfs before I commit myself to a new install, but I want to try it out first. What did I do wrong?

What should I see when I click on Dolphin is a btrfs system? Thank you in advance.

[mr_raider]

Actually Clay, I forget to mention here another advantage to BTRFS - unified free space.

In the old EXT4 way, to have a separate home or a second linux install you must partition. Partitioning requires that you guess how much space you'll need for each partition and it makes it incredibly difficult to amend.

EXT4 example:
128GB SSD partitions:
1 - swap - 4GB
2 - kubuntu - 16GB
3 - other Linux - 16GB
4 - home - 92GB

Kubuntu ends up using 12GB - 4 wasted GB, "other" linux uses 8GB - 8 wasted GB. Or wait - what if other linux needs 20GB? Time to re-partition and hope gparted doesn't let you down.

BTRFS Example, same drive:
1 - swap 4GB
2 - everything else - 124GB.

Kubuntu, other linux, and home are in separate subvolumes but on the same filesystem. Free space is available to all the subvolumes. If other linux only uses 8 or grows to 20GB no matter - it expands and contracts as needed and no free space is languishing.

This is a big reason I use BTRFS on all my installs, no more playing Tetris with SSD partitions.

I have one ext4 install left, it's an ubuntu server where / and /home actually reside on separate physical drives. All my other PCs are BTRFS.

[GreyGeek]

To be brutally honest, only a bit. I'm a tough nut to crack!

Okay, let's add to the scenario I outlined. After I immediately delete everything except the snapshot, I then (again, a mental exercise) zero out the space occupied by the contents I deleted (don't worry about how I did that; again, an exercise), but again, leave the snapshot intact. What happens to the 'contents' of the snapshot now? Is it still able to restore what I nuked, or is it now also, and empty container?

I have these three cups ... and a pea. Follow closely!

[oshunluvr]

How much space do snapshots take, for those who have tiny SSDs?

Actually Clay, I forget to mention here another advantage to BTRFS - unified free space.

In the old EXT4 way, to have a separate home or a second linux install you must partition. Partitioning requires that you guess how much space you'll need for each partition and it makes it incredibly difficult to amend.

EXT4 example:
128GB SSD partitions:
1 - swap - 4GB
2 - kubuntu - 16GB
3 - other Linux - 16GB
4 - home - 92GB

Kubuntu ends up using 12GB - 4 wasted GB, "other" linux uses 8GB - 8 wasted GB. Or wait - what if other linux needs 20GB? Time to re-partition and hope gparted doesn't let you down.

BTRFS Example, same drive:
1 - swap 4GB
2 - everything else - 124GB.

Kubuntu, other linux, and home are in separate subvolumes but on the same filesystem. Free space is available to all the subvolumes. If other linux only uses 8 or grows to 20GB no matter - it expands and contracts as needed and no free space is languishing.

[oshunluvr]

Answer: 50%.

The steps to the answer:
Subvol A with folders 1 and 2 occupy 100% of the space.
Subvol B with folders 1 and 2 occupy 100% of the space - the same 100% occupied by subvol A. Therefore no additional space is required.
When folder 2 is deleted from subvol A, no space is freed because folder 2 is still in subvol B. 100% of the space is still occupied.
When folder 1 is deleted from subvol B, no space is freed because folder 1 is still in subvol A. 100% of the space is still occupied.
After the above actions, Folder 1 exists only in subvolume A and folder 2 exists only in subvolume B.
When you delete subvolume A, folder 1 is also totally deleted. Only subvolume B and folder 2 remain. Therefore, 50% of the space is freed - the portion formerly occupied by folder 1.

[oshunluvr]

Well, the point is, if a snapshot exists the space the file occupies isn't flagged as available for use just because you deleted it in one place. This doesn't happen until all subvolumes (snapshots are subvolumes) that the file belongs in have released the space - either by deleting the file from every subvolume OR deleting all the subvolumes containing the file.

This is super easy to demonstrate yourself if you have a BTRFS file system to play with.

Mount the root BTRFS file system and make a subvolume: sudo btrfs su cr A

You now have an empty subvolume named "A". Enter the subvolume and add a folder named "1" and put some files in it, then make another folder named "2" in the same subvolume and copy all the files from 1 into 2. You now have a single subvolume with 2 folders containing identical files.

Note the amount of free space on the BTRFS file system.

Return to the root filesystem and take a snapshot of A: sudo btrfs su sn A B

You now have two subvolumes: "A" and "B". Observe the amount of free space on the BTRFS file system is unchanged. Observe that B contains the same folders and files as A.

Now, open subvolume A and delete folder "2". Observe the file system still has the same amount of free space.

Open subvolume B and delete folder "1". Note the file system space is unchanged.

You still have a copy of all the files you started with, but folder 1 is only in A and folder 2 is only in B.

Exercise: By percentage of the original space used by subvolume A and it's two folders "1" and "2"; How much free space is recovered if you now delete subvolume A?

[Snowhog]

I sort of understand that. When I said 'zero out' I meant writing zeros over the spaces the files that were flagged as deleted occupy. I do understand that a deleted file only has first bit/character replaced with a deletion flag, telling the system to no longer display it 'normally', and to let the system know that future disk writes can use that space when needed. So if I were to accomplish that, but didn't touch the snapshot 'file', it's metadata (would that still exist?) would point to locations that no longer contained useful data, yes? Truly not trying to be difficult, but these type of examples, and the answers to the questions I raise in them, help me to "see".

[oshunluvr]

I'm struggling to come up with a parallel example...

Your files are in a file cabinet.
The file system is a manager that is responsible to maintain the contents of the file cabinet.
The metadata is an index that tells the manager where to find a file.

A snapshot makes a copy of the index, so now the manager has index A and index B but the files haven't changed and the cabinet is the same size.

You go to the manager and tell him to delete a file. But you are only accessing index A. The manager draws a line though the file name on index A but index B remains unchanged and the file remains in the cabinet.

Now you got to the manager again and hand him a new file to put in the cabinet. He does so and adds the file to index A - but not to B. The cabinet got fuller, but index B is remains unchanged.

Then you go to the manager and tell him to throw away the entire file cabinet. He throws away index A and pulls the one file out of the cabinet that you added to index A and throws it away also.

The remainder of the files, as listed on index B, remain in the file cabinet.

[oshunluvr]

To be brutally honest, only a bit. I'm a tough nut to crack!

Okay, let's add to the scenario I outlined. After I immediately delete everything except the snapshot, I then (again, a mental exercise) zero out the space occupied by the contents I deleted (don't worry about how I did that; again, an exercise), but again, leave the snapshot intact. What happens to the 'contents' of the snapshot now? Is it still able to restore what I nuked, or is it now also, and empty container?

Well, yeah - if you nuke your file system with dd you get what you paid for, but if you mean you zero out all the unused space in the file system - No you don't lose a file. The space is still used by the snapshot, therefore it's not unused. Point being - deleting the subvolume but not the snapshot deletes nothing with regards to data. It just removes the metadata.

So your illustration here doesn't hold water - by deleting the subvolume that has a snapshot you haven't released any space formerly occupied by the subvolume, therefore there is no space to zero out.

[Snowhog]

...
Does this help at all?

To be brutally honest, only a bit. I'm a tough nut to crack!

Okay, let's add to the scenario I outlined. After I immediately delete everything except the snapshot, I then (again, a mental exercise) zero out the space occupied by the contents I deleted (don't worry about how I did that; again, an exercise), but again, leave the snapshot intact. What happens to the 'contents' of the snapshot now? Is it still able to restore what I nuked, or is it now also, and empty container?

[oshunluvr]

And when Oshunluver says he “re-snapshots the snapshot” he’s taking a rw snapshot of a ro snapshot so that when he makes it a new @ or @home or whatever, it can be read and written to.

My primary reason for the re-snapshot-ing is to preserve the snapshot in it's current state in case I need it again. Generally, and specifically in the case I outlined above, I intend to re-attempt whatever went wrong - like a kernel installation - and I want to be able to try again and again until I figure it out. Think of it like restoring from a backup, if you have to restore from a backup you don't so it and then delete the backup. You restore and preserve the backup until it's replaced by a newer one. Same principal here.

I personally don't bother with ro snapshots unless I'm preparing to send a snapshot to another file system - then ro status is required. I don't mount or access any of my snapshots except to restore one like I did here. I figure as long as I'm not mucking about with them, they're safe. However, in this example the snapshot could be ro or rw and I still would go through the same process. There's certainly nothing wrong using ro as a default status for snapshots.

When I do my automated backups, I have the script automatically re-snapshot the received snapshot so it has rw status, then I (again automated) replace the UUIDs within the snapshot so that it's bootable from the backup location as is. I actually haven't had to test this, but I should one day just to be sure it works.