More and more users here are starting to use the BTRFS file system, many at my encouragement. I thought it was a good time to discuss some of the reasons why BTRFS is so great and some of the differences between BTRFS and the default EXT4 file systems. Today a primer on:
File System Terms, How BTRFS Works with Devices, and SUBVOLUMES
I brought these topics together because they are related to each other.
First, a little bit on basic file system terminology:
This is important because using the correct terms helps tremendously with communication and understanding. Most of us incorrectly say we mount a partition. However, we never actually mount a partition, you can only mount a file system. Here's how it works: You want to divide a device into sections so you create partitions on it. Then you create a file system (format) on a partition. Then you mount the file system. Then you can begin using it.
BTRFS handles partitions and devices slightly differently than most file systems. Most file systems require and wholly occupy a partition - even if you're using the entire device. For example, you buy a new hard drive (a DEVICE) and install it into your computer;
For example, here's the output of "lsblk" from my server:
Notice that sda and sdc are not partitioned and have no partition table. These two drives are a single BTRFS file system. The other three drives are partitioned.
BTRFS SUBVOLUMES
Partitions can be quite useful. With devices of 1 terabyte or more, using all the space in a single file system can be cumbersome and difficult to organize. Partitioning seems like a necessity. BTRFS can replace partitioning with SUBVOLUMES. Similar to partitions with their separate file systems, subvolumes can be individually mounted. One big advantage to this approach is while portions of data are segregated (separately existing) from other portions of data just like with partitioning, all the free space on the file system is available to ALL of the subvolumes that are on the same file system. The subvolumes share the file system space without sharing data.
If you've ever had to resize a couple of partitions or had to wipe a hard drive clean and start over because your partitions were not the right sizes, you can see that subvolumes are a huge advantage over partitions in this regard. Subvolumes resize themselves as the amount of data they hold increases or decreases. The only limit is the size of the BTRFS file system the subvolumes reside on. Subvolume-to-subvolume data is kept separate, but free space is shared. Partitions can't do this. Subvolumes can be created or deleted instantly and at will without risking other data or lengthy and dangerous movement of data along with partition boundaries.
Subvolumes can even be easily transferred from one BTRFS file system to another. To do this, BTRFS uses the send and receive commands. You can send a subvolume to a file or you can receive it on another BTRFS file system with a single command.
BTRFS SNAPSHOTS
Snapshots are a form of subvolume. The snapshot command makes a functioning copy of a source subvolume on the same BTRFS file system. You can mount, view and edit the files within it, delete or add files to within the snapshot just like you could with the source subvolume. When a snapshot is initially created, it takes no data space because at this point the data is not actually copied. Instead, a copy of the metadata (the location information of each file) is held in the snapshot. As changes are made to either the source subvolume or the snapshot, the changes are stored so the snapshot can begin to grow and start to use data space. One of the effects of this is if you delete a file or number of files from the source subvolume you would not see a corresponding increase in available space on the file system. This is because, although you've deleted a file from the source subvolume, the snapshot retains it's metadata and now the deleted file(s) are in the snapshot subvolume only. If you edit or delete files in the snapshot rather than in the source subvolume the same thing occurs - the deleted file remains in the source subvolume although it has been deleted from the snapshot.
So what's the point or purpose of a snapshot?
Snapshots can protect you from accidental deletions: You can retrieve a file deleted from the source subvolume from the previously made snapshot (or vice-versa).
Snapshots can protect you from a failed upgrade or installation: Prior to upgrading your video driver or your install version, take a snapshot. If the upgrade doesn't go well - you can actually delete the source subvolume, replace it with the snapshot, and be right back where you were before the upgrade just moments before.
Snapshots are also part of the backup process. To create a full copy of a subvolume and put it on different file system for backup purposes, you first take a read-only snapshot of a subvolume. The read-only status insures no data within the snapshot itself is manipulated during the send procedure which might cause data corruption. You can "send" the snapshot directly to another btrfs file system or if the backup device is not using the btrfs file system (or is only available over a network connection), you can send the snapshot to a file. The file then can be "received" to a btrfs file system later or on another computer.
Conclusion
With BTRFS and subvolumes and their snapshots, you can make easy and simple backups, prevent unintended consequences from upgrade or install operations, and protect yourself from your own mistakes. All of this and more right at your fingertips!
A couple of final notes: If you use the whole disk for btrfs instead of a partition, you can't boot from that disk. btrfs will occupy the space normally reserved for grub. BTRFS can also span multiple devices for a single filesystem like LVM or RAID. This topic will be covered another time.
File System Terms, How BTRFS Works with Devices, and SUBVOLUMES
I brought these topics together because they are related to each other.
First, a little bit on basic file system terminology:
- DEVICE = A hard drive, USB drive, SD card, etc.
- PARTITION = A divided section on a device.
- PARTITION TABLE = The list and definitions (locations) of partitions on a device.
- FORMAT verb = The action taken to create a file system on a partition or device.
- FORMAT adjective = The type of file system on a partition or device.
- FILE SYSTEM = The methods and data structures that an operating system uses to keep track of files on a device or partition; i.e. the way the files are stored on a device.
This is important because using the correct terms helps tremendously with communication and understanding. Most of us incorrectly say we mount a partition. However, we never actually mount a partition, you can only mount a file system. Here's how it works: You want to divide a device into sections so you create partitions on it. Then you create a file system (format) on a partition. Then you mount the file system. Then you can begin using it.
BTRFS handles partitions and devices slightly differently than most file systems. Most file systems require and wholly occupy a partition - even if you're using the entire device. For example, you buy a new hard drive (a DEVICE) and install it into your computer;
- Using EXT4: You must first create a partition table and at least one partition - even if you have no intention of dividing the available space. Assuming this drive's device name is SDB (the second SATA or SCSI device), each partition would have it's own device name as SDB followed by the partition number: SDB1, SDB2, and so on. After partitioning, you would have to create a file system on each partition. Then you could mount and use the space on the drive for files.
- Using BTRFS: You may first create a partition table and at least one partition if you wish, but you are not required to. BTRFS can occupy an entire device. You can skip the partition table and go directly to creating (formatting) your BTRFS file system on the entire SDB device.
For example, here's the output of "lsblk" from my server:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 5.5T 0 disk
sdb 8:16 0 1.8T 0 disk
├─sdb1 8:17 0 1007K 0 part
├─sdb2 8:18 0 55.9G 0 part
└─sdb3 8:19 0 1.8T 0 part
sdc 8:32 0 5.5T 0 disk
sdd 8:48 0 1.8T 0 disk
├─sdd1 8:49 0 1007K 0 part
├─sdd2 8:50 0 55.9G 0 part
└─sdd3 8:51 0 1.8T 0 part
sde 8:64 0 55.9G 0 disk
├─sde1 8:65 0 1007K 0 part
└─sde2 8:66 0 55.9G 0 part
sda 8:0 0 5.5T 0 disk
sdb 8:16 0 1.8T 0 disk
├─sdb1 8:17 0 1007K 0 part
├─sdb2 8:18 0 55.9G 0 part
└─sdb3 8:19 0 1.8T 0 part
sdc 8:32 0 5.5T 0 disk
sdd 8:48 0 1.8T 0 disk
├─sdd1 8:49 0 1007K 0 part
├─sdd2 8:50 0 55.9G 0 part
└─sdd3 8:51 0 1.8T 0 part
sde 8:64 0 55.9G 0 disk
├─sde1 8:65 0 1007K 0 part
└─sde2 8:66 0 55.9G 0 part
BTRFS SUBVOLUMES
Partitions can be quite useful. With devices of 1 terabyte or more, using all the space in a single file system can be cumbersome and difficult to organize. Partitioning seems like a necessity. BTRFS can replace partitioning with SUBVOLUMES. Similar to partitions with their separate file systems, subvolumes can be individually mounted. One big advantage to this approach is while portions of data are segregated (separately existing) from other portions of data just like with partitioning, all the free space on the file system is available to ALL of the subvolumes that are on the same file system. The subvolumes share the file system space without sharing data.
If you've ever had to resize a couple of partitions or had to wipe a hard drive clean and start over because your partitions were not the right sizes, you can see that subvolumes are a huge advantage over partitions in this regard. Subvolumes resize themselves as the amount of data they hold increases or decreases. The only limit is the size of the BTRFS file system the subvolumes reside on. Subvolume-to-subvolume data is kept separate, but free space is shared. Partitions can't do this. Subvolumes can be created or deleted instantly and at will without risking other data or lengthy and dangerous movement of data along with partition boundaries.
Subvolumes can even be easily transferred from one BTRFS file system to another. To do this, BTRFS uses the send and receive commands. You can send a subvolume to a file or you can receive it on another BTRFS file system with a single command.
BTRFS SNAPSHOTS
Snapshots are a form of subvolume. The snapshot command makes a functioning copy of a source subvolume on the same BTRFS file system. You can mount, view and edit the files within it, delete or add files to within the snapshot just like you could with the source subvolume. When a snapshot is initially created, it takes no data space because at this point the data is not actually copied. Instead, a copy of the metadata (the location information of each file) is held in the snapshot. As changes are made to either the source subvolume or the snapshot, the changes are stored so the snapshot can begin to grow and start to use data space. One of the effects of this is if you delete a file or number of files from the source subvolume you would not see a corresponding increase in available space on the file system. This is because, although you've deleted a file from the source subvolume, the snapshot retains it's metadata and now the deleted file(s) are in the snapshot subvolume only. If you edit or delete files in the snapshot rather than in the source subvolume the same thing occurs - the deleted file remains in the source subvolume although it has been deleted from the snapshot.
So what's the point or purpose of a snapshot?
Snapshots can protect you from accidental deletions: You can retrieve a file deleted from the source subvolume from the previously made snapshot (or vice-versa).
Snapshots can protect you from a failed upgrade or installation: Prior to upgrading your video driver or your install version, take a snapshot. If the upgrade doesn't go well - you can actually delete the source subvolume, replace it with the snapshot, and be right back where you were before the upgrade just moments before.
Snapshots are also part of the backup process. To create a full copy of a subvolume and put it on different file system for backup purposes, you first take a read-only snapshot of a subvolume. The read-only status insures no data within the snapshot itself is manipulated during the send procedure which might cause data corruption. You can "send" the snapshot directly to another btrfs file system or if the backup device is not using the btrfs file system (or is only available over a network connection), you can send the snapshot to a file. The file then can be "received" to a btrfs file system later or on another computer.
Conclusion
With BTRFS and subvolumes and their snapshots, you can make easy and simple backups, prevent unintended consequences from upgrade or install operations, and protect yourself from your own mistakes. All of this and more right at your fingertips!
A couple of final notes: If you use the whole disk for btrfs instead of a partition, you can't boot from that disk. btrfs will occupy the space normally reserved for grub. BTRFS can also span multiple devices for a single filesystem like LVM or RAID. This topic will be covered another time.
Comment