Maintenance on Btrfs
https://wiki.debian.org/Btrfs
The Btrfs-filesystem man page states:
My Neon installation is currently using the 4.8.0-42-generic #45~16.04.1-Ubuntu SMP kernel. The warning highlighted in red cautions against using a kernel greater than 3.13.4. This suggests that running a defragmentation would slow the system down, not speed it up, especially if using cp with --rflink is used. However, I suspect that may only apply to the 3.x kernel series. The 4.x series may be safe to use defragmentation.
[#]BTRFS[/#]
Maintenance
As a btrfs volume ages, you may notice performance degrade. This is because btrfs is a Copy On Write file system, and all COW filesystems eventually reach a heavily fragmented state; this includes ZFS. Over time, logs in /var/log/journal will become split across tens of thousands of extents. This is also the case for sqlite databases such as those that are used for Firefox and a variety of common desktop software. Fragmentation is a major contributing factor to why COW volumes become slower over time.
ZFS addresses the performance problems of fragmentation using an intelligent Adaptive Replacement Cache (ARC); the ARC requires massive amounts of RAM. Btrfs took a different approach, and benefits from—some would say requires—periodic defragmentation. In the future, maintenance of btrfs volumes on Debian systems will be automated using btrfsmaintenance. For now use:
sudo ionice -c idle btrfs filesystem defragment -t 32M -r $PATH
This command must be run as root, and it is recommended to ionice it to reduce the load on the system. To further reduce the IO load, flush data after defragmenting each file using:
sudo ionice -c idle btrfs filesystem defragment -f -t 32M -r $PATH
Target extent size is a little known, but for practical purposes absolutely essential argument. By default btrfs fi defrag only defrags files of less than 256KiB, because does not touch extents bigger than $SIZE, where $SIZE is by default 256KiB! While argument "-t 1G" would seem to be better than "-t 32M", because most volumes will have 1GiB chunk size, in practise this is not the case. Additionally, if you have a lot of snapshots or reflinked files, please use "-f" to flush data for each file before going to the next file. As of btrfs-progs-4.6.1, "-t 32M" is still necessary, but "-t 32M" is the default after btrfs-progs-4.7. Please consult the following linux-btrfs thread for more information.
As a btrfs volume ages, you may notice performance degrade. This is because btrfs is a Copy On Write file system, and all COW filesystems eventually reach a heavily fragmented state; this includes ZFS. Over time, logs in /var/log/journal will become split across tens of thousands of extents. This is also the case for sqlite databases such as those that are used for Firefox and a variety of common desktop software. Fragmentation is a major contributing factor to why COW volumes become slower over time.
ZFS addresses the performance problems of fragmentation using an intelligent Adaptive Replacement Cache (ARC); the ARC requires massive amounts of RAM. Btrfs took a different approach, and benefits from—some would say requires—periodic defragmentation. In the future, maintenance of btrfs volumes on Debian systems will be automated using btrfsmaintenance. For now use:
sudo ionice -c idle btrfs filesystem defragment -t 32M -r $PATH
This command must be run as root, and it is recommended to ionice it to reduce the load on the system. To further reduce the IO load, flush data after defragmenting each file using:
sudo ionice -c idle btrfs filesystem defragment -f -t 32M -r $PATH
Target extent size is a little known, but for practical purposes absolutely essential argument. By default btrfs fi defrag only defrags files of less than 256KiB, because does not touch extents bigger than $SIZE, where $SIZE is by default 256KiB! While argument "-t 1G" would seem to be better than "-t 32M", because most volumes will have 1GiB chunk size, in practise this is not the case. Additionally, if you have a lot of snapshots or reflinked files, please use "-f" to flush data for each file before going to the next file. As of btrfs-progs-4.6.1, "-t 32M" is still necessary, but "-t 32M" is the default after btrfs-progs-4.7. Please consult the following linux-btrfs thread for more information.
defragment [options] <file>|<dir> [<file>|<dir>...]
Defragment file data on a mounted filesytem.
If -r is passed, files in dir will be defragmented recursively. The start position and the number of bytes
to defragment can be specified by start and len using -s and -l options below. Extents bigger than value
given by -t will be skipped, otherwise this value is used as a target extent size, but is only advisory
and may not be reached if the free space is too fragmented. Use 0 to take the kernel default, which is
256kB but may change in the future. You can also turn on compression in defragment operations.
Warning
Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2 as well as with Linux stable kernel
versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4 will break up the ref-links of COW data (for example files
copied with cp --reflink, snapshots or de-duplicated data). This may cause considerable increase of
space usage depending on the broken up ref-links.
Options
-v
be verbose, print file names as they’re submitted for defragmentation
-c[<algo>]
compress file contents while defragmenting. Optional argument selects the compression algorithm, zlib
(default) or lzo. Currently it’s not possible to select no compression. See also section EXAMPLES.
-r
defragment files recursively in given directories
-f
flush data for each file before going to the next file. This will limit the amount of dirty data to
current file, otherwise the amount cumulates from several files and may increase system load.
-s <start>[kKmMgGtTpPeE]
defragmentation will start from the given offset, default is beginning of a file
-l <len>[kKmMgGtTpPeE]
defragment only up to len bytes, default is the file size
-t <size>[kKmMgGtTpPeE]
target extent size, do not touch extents bigger than size
For start, len, size it is possible to append units designator: 'K', 'M', 'G', 'T', 'P', or 'E', which
represent KiB, MiB, GiB, TiB, PiB, or EiB, respectively (case does not matter).
Note
Directory arguments without -r do not defragment files recursively but will defragment certain
internal trees (extent tree and the subvolume tree). This has been confusing and could be removed
in the future.
Defragment file data on a mounted filesytem.
If -r is passed, files in dir will be defragmented recursively. The start position and the number of bytes
to defragment can be specified by start and len using -s and -l options below. Extents bigger than value
given by -t will be skipped, otherwise this value is used as a target extent size, but is only advisory
and may not be reached if the free space is too fragmented. Use 0 to take the kernel default, which is
256kB but may change in the future. You can also turn on compression in defragment operations.
Warning
Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2 as well as with Linux stable kernel
versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4 will break up the ref-links of COW data (for example files
copied with cp --reflink, snapshots or de-duplicated data). This may cause considerable increase of
space usage depending on the broken up ref-links.
Options
-v
be verbose, print file names as they’re submitted for defragmentation
-c[<algo>]
compress file contents while defragmenting. Optional argument selects the compression algorithm, zlib
(default) or lzo. Currently it’s not possible to select no compression. See also section EXAMPLES.
-r
defragment files recursively in given directories
-f
flush data for each file before going to the next file. This will limit the amount of dirty data to
current file, otherwise the amount cumulates from several files and may increase system load.
-s <start>[kKmMgGtTpPeE]
defragmentation will start from the given offset, default is beginning of a file
-l <len>[kKmMgGtTpPeE]
defragment only up to len bytes, default is the file size
-t <size>[kKmMgGtTpPeE]
target extent size, do not touch extents bigger than size
For start, len, size it is possible to append units designator: 'K', 'M', 'G', 'T', 'P', or 'E', which
represent KiB, MiB, GiB, TiB, PiB, or EiB, respectively (case does not matter).
Note
Directory arguments without -r do not defragment files recursively but will defragment certain
internal trees (extent tree and the subvolume tree). This has been confusing and could be removed
in the future.
[#]BTRFS[/#]
Comment