Announcement

Collapse
No announcement yet.

BTRFS for new users. Why you should consider it.

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    BTRFS for new users. Why you should consider it.

    I see a lot of new users that come here for advice when installing for the first time. Sometimes someone will suggest that they use BTRFS instead of EXT4 (the default). I usually don't. I'm a huge fan of and experienced user of BTRFS, but I feel new Linux users are struggling with enough without having one more new thing to learn. However, I think I might have been wrong about that. In many ways BTRFS is actually much easier than using EXT4 and has some benefits that can save your rear-end. If new users need anything, it's as many lifelines as they can get.

    Before I explain how you might use BRTFS, let me explain two important reasons why;

    Partitioning: Common advice is to have a separate partition for /home. This is a great idea for several reasons. However, partitioning is not something most Windows users have ever done, and it's inherently dangerous. The less time you spend mucking about with your partition table the better. BTRFS allows you to have your home separate from your install without having a separate partition. Everything on a single partition but still separate. BTRFS does this with virtual containers referred to as "subvolumes". Subvolumes act like partitions - they're mountable and they contain files - but unlike partition-based file systems you can move them, copy them, and they expand and contract automatically as needed.

    Disasters: Sometimes, not as often as the old days but still sometimes, a package update goes sideways and your OS is unbootable. This happened to me today. Sometimes you, the user, will make a fatal mistake. It happens to experienced and new users. How do you recover from this? All too often, you end up having to re-install. This means all your tweaks, setups, configurations, are lost. With BTRFS and subvolumes, you can easily and simply take a "snapshot" of your home or your install subvolumes. A snapshot is exactly what it sounds like - a image of your subvolume frozen in time. With a recent snapshot of your install, you need only replace your broken or damaged subvolume with the snapshot and reboot. In that case, you would only lose the tweaks, setups, configurations that occurred between the snapshot and the disaster. In my case, my computer takes a snapshot everyday so this morning - when I broke it - within a couple of minutes I was right back to where I had been just 20 minutes before. A lifesaver if there ever was one.

    How do you use BTRFS?
    Initially, it's as simple as choosing BTRFS instead of EXT4 when you install. The Kubuntu installer will automatically create two subvolumes for you, one for the install and one for home. Use a single partition. It's preferred and easier than trying to create and use multiple partitions.

    Once you're up and running, from there it's learning how to use the advanced features the BTRFS offers you. I've touched on only two, subvolumes and snapshots, but there are many more. A couple of examples: I use BTRFS to make backups and have multiple different Linux installs on the same partition. BTRFS can use multiple devices - more than one partition or hard drive - as single file system and multiple devices can be configured as RAID or JBOD.

    Here's what I did wrong this morning and how BTRFS saved me:
    I have set up my system to take a snapshot every day at 3am. This morning about 8am I booted up and within a half hour did some package updates and installed a new kernel version. When I rebooted into the new kernel, the system locked up with a "kernel panic". No big deal, I rebooted and selected the previous kernel. I decided that maybe rebuilding the initramfs would allow the new kernel to boot, so I issued the process to rebuild the initramfs - but when I did, I selected ALL kernels instead of just the one that wasn't working. As soon as I hit the enter key I knew I had potentially made a mistake, but it was too late. I let the rebuild process continue and crossed my fingers. When I rebooted, the new kernel still panic. Second reboot - the previous kernel panicked. Third reboot - the only other kernel panicked. I broke them all. Now what do I do?

    Had I been using EXT4, I might have been finished - a new installation may have been my only course of action. But I'm not using EXT4, I'm using BTRFS and I had a very recent snapshot. I booted into one of my other installs (you could boot to a liveUSB if you only had one install), navigated to the location my now broken install and it's snapshots reside, renamed the broken subvolume and then gave the snapshot the name the install subvolume previously had, and rebooted. That was it. The entire fix consisted of these actions: Boot, Mount, Rename 2 files, Reboot. That's it. In about 3 minutes I was back in business. I then re-did the package updates and installed the new kernel again. This time I'll try and not break ALL the kernels again.

    Consider using BTRFS for your next install.

    Please Read Me

    #2
    Amen!
    Preach it, Brother!

    3 minutes to restore. A task I've done several times to revert from experiments. When I tested the four major P2P files systems I always restored from my last snapshot so that each P2P FS began with the same environment.

    I send a backup snapshot to a third HD on my laptop, which I mount for that purpose and then umount when I am done. I haven't had to use any snapshot from that HD, yet. My snapshots average 119Gb. For offsite I send my snapshots to an external USB HD, but it's only 320Gb so it holds only my latest two.

    After using Btrfs for 2 1/2 years I can't tell the difference in performance between EXT4 and Btrfs, until it comes to backup and recovery, then the difference is too huge to compare. I'll never use any FS except Btrfs. (That includes ZFS after I explored it very thoroughly. If I were running a server I would use ZFS for it, though.)
    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
    – John F. Kennedy, February 26, 1962.

    Comment


      #3
      +1 ,,,,,my 1TB storage drive uses it ,,,,and about 300GB of my 500GB system drive dose currently.

      VINNY
      i7 4core HT 8MB L3 2.9GHz
      16GB RAM
      Nvidia GTX 860M 4GB RAM 1152 cuda cores

      Comment


        #4
        I plan to use BTRFS for any future installs, Which will most likely be going up to 18.04 in a couple of months. I like the sound of the snapshot stuff.

        Comment


          #5
          Originally posted by Bings View Post
          I plan to use BTRFS for any future installs, Which will most likely be going up to 18.04 in a couple of months. I like the sound of the snapshot stuff.
          Words can't do snapshots and send & receive justice. IF you have the storage space I'd recommend using Oshunluver's setup where several distros are installed on one partition.
          "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
          – John F. Kennedy, February 26, 1962.

          Comment


            #6
            Silly question but ... can a guest Linux OS be BTRFS if the host OS (Kubuntu 16.04) is ext4?
            Kubuntu 20.04

            Comment


              #7
              Originally posted by chimak111 View Post
              Silly question but ... can a guest Linux OS be BTRFS if the host OS (Kubuntu 16.04) is ext4?
              Are you asking about a Virtual Machine? If so, yes the guest file system on it's virtual drive is independent of the hosts.

              This brings up one caveat about BTRFS: If you use virtual drives on a BTRFS file system (the host), you must use FIXED size drives, not dynamic. The way BTRFS works can cause corruption of dynamically allocated virtual devices, which includes swap files. If you use BTRFS as your host file system, you must use a swap partition not a swap file.

              Please Read Me

              Comment


                #8
                Originally posted by GreyGeek View Post
                Words can't do snapshots and send & receive justice. IF you have the storage space I'd recommend using Oshunluver's setup where several distros are installed on one partition.
                Right now, I have one 128gb SSD and 1TB HDD. The SSD I dual boot Kubuntu and Windows 7 on. Windows takes up so much space though that I put the /home folder on a partition on the HDD. I want to have a Ubuntu Studio install and also an install where I can see how well I can play the games I have on steam with. So, the plan is to get a new SSD drive, use BTRFS and have the linux installs on there. Use the HDD for media and game storage.

                Comment


                  #9
                  Originally posted by oshunluvr View Post
                  Are you asking about a Virtual Machine? If so, yes the guest file system on it's virtual drive is independent of the hosts.

                  This brings up one caveat about BTRFS: If you use virtual drives on a BTRFS file system (the host), you must use FIXED size drives, not dynamic. The way BTRFS works can cause corruption of dynamically allocated virtual devices, which includes swap files. If you use BTRFS as your host file system, you must use a swap partition not a swap file.
                  I'm sorry for not being clear.

                  My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

                  Meanwhile, I came across something related (?) to your caveat:
                  As in any technology, BTRFS is not perfect. For example, it suffers when there are heavy write activities in the middle of an existing files, so probably it’s not the best candidate for virtualization (the virtual disks are updated in-place at each write).
                  from It’s 2016: and BTRFS could really be your next filesystem
                  Kubuntu 20.04

                  Comment


                    #10
                    Originally posted by chimak111 View Post
                    I'm sorry for not being clear.

                    My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

                    Meanwhile, I came across something related (?) to your caveat:from It’s 2016: and BTRFS could really be your next filesystem
                    I use BTRFS on virtual machines all the time, so yes you can use BTRFS in a VM and learn about it there. Note that the current Kubuntu installer will automatically create a swap file at installation time if there is not a swap partition present. A bug has been filed on this behavior, but you should be aware of it. Either manually create a swap partition on your virtual disk at installation or let it do it's thing and when you boot into the VM the first time, turn swap off and edit /etc/fstab removing the swap file reference. I doubt you'll need swap or want in a VM anyway.

                    Once you get your new SSD, I suggest making it bootable, creating a swap partition, then use the remainder as a single BTRFS file system. You could do a fresh Kubuntu 18.04 install there and leave your current one in place as a backup to boot to if necessary. I have six distros and their homes installed to my single BTRFS file system (480GB total - 2x250GB SSDs in RAID0). I use two BTRFS file systems on two hard drives for backups. BTRFS allows you to "send" (copy) an entire subvolume to another BTRFS file system. The copies can even be made bootable so I can boot to two other drives if needed. My system automatically makes two backups weekly (one of my primary distro, one of my home) and send them to each of the two backup drives. I also take daily snapshots of my subvolumes and extras when I'm doing something I may want to back out of later. It's all pretty slick and almost totally hands-off at this point.

                    Please Read Me

                    Comment


                      #11
                      Originally posted by chimak111 View Post
                      ...
                      My Kubuntu 16.04 is the host and it is ext4 because that's the default. I wanted to learn about BTRFS by having a guest OS in a VM and using BTRFS for that guest. So my question was about whether learning about BTRFS that way is practical.

                      ...
                      Btrfs has capabilities to expand or contract its FS size. Being a COW system it would be important, as I understand it, to make the virtual HD a FIXED size even on EXT4 so that VirtualBox doesn't expand or contract it as your usage varies. Inside the guest OS that uses Btrfs as the FS you can contract the size but not expand it, unless you've previously contracted it.

                      btrfs filesystem resize [<devid>:][+/-]<size>[kKmMgGtTpPeE]|[<devid>:]max <path>

                      The resize command does not manipulate the size of underlying partition. If you wish to enlarge/reduce a filesystem, you must make sure you can expand the partition before enlarging the filesystem and shrink the partition after reducing the size of the filesystem. This can done using fdisk(8) or parted(8) to delete the existing partition and recreate it with the new desired size. When recreating the partition make sure to use the same starting partition offset as before.

                      Growing is usually instant as it only updates the size. However, shrinking could take a long time if there are data in the device area that’s beyond the new end. Relocation of the data takes time.
                      "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                      – John F. Kennedy, February 26, 1962.

                      Comment


                        #12
                        When distros pick it as the default, maybe with user-friendly aids to utlize the features, then I probably will use it. Then again, I may just be lucky as I have not had to do a reinstall of an OS other than replacing old, dead spinning drives in a loooong long time (other than for fun or out of boredom)

                        As a side note:
                        https://help.ubuntu.com/community/UbuntuReinstallation
                        I have used this method in the past, iirc probably back in the 8.10-12.04 era when I was running pre-releases as my daily driver. It did work. Dunno why I forgot this.

                        How much space do snapshots take, for those who have tiny SSDs?
                        What about performance on spinning 5400 rpm drives?

                        Comment


                          #13
                          A snapshot is initially empty but as a file is added, changed or deleted the original is copied to the snapshot.
                          So, it gets larger over time. A read only snapshot is like a save-point. Those are the ones you want to use as a roll back.

                          My HD’s are 5400 RPM and I can’t see any performance degradation over EXT4.
                          "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                          – John F. Kennedy, February 26, 1962.

                          Comment


                            #14
                            Originally posted by GreyGeek View Post
                            A snapshot is initially empty but...
                            This confuses me; can't quite wrap my head around it. As I'm "visually oriented"; I need to 'see' things (actual, ideas, concepts, etc). So let me ask this: I take a snapshot. As you say, it is "initially empty". I immediately delete EVERYTHING except the created snapshot (this is a mental exercise, so go with me here). What good is the snapshot now? I can't recover from my self-inflicted gun shot with it, can I? I'm expecting the answer is "No, you cannot."
                            Windows no longer obstructs my view.
                            Using Kubuntu Linux since March 23, 2007.
                            "It is a capital mistake to theorize before one has data." - Sherlock Holmes

                            Comment


                              #15
                              If you do a technical test of file system speeds, BTRFS is equivalent in some areas and slightly slower in some others when compared to EXT4. If performance is your only goal, then you would likely be using several different file systems as they all have their strengths and weaknesses. The point of BTRFS isn't speed, but ease of use and expanded functionality.
                              The hours I save not making or restoring backups or doing re-installs when I do something stupid (read the first post in this thread ) more than make up for any milliseconds lost to transfer rates. Honestly, with a high-end machine and an SSD, I doubt there's any noticeable difference at all in real-world use.

                              Let me make my position clear:
                              There simply is no other file system that is as easy to use as BTRFS, period.
                              There is no other file system with the features that BTRFS has, period.

                              If simple and easy-to-use isn't what you're looking for, by all means use something else. There are plenty to choose from and Linux is all about choice.

                              As far as snapshots go, as GG said - they take zero space and grow as they age. If you have a small SSD, you have to monitor your usage more closely. @Here, just this week, I did some housekeeping and deleted 14 snapshots of varying ages of my install and home subvolumes. The oldest was 3 months or so old. I recovered 10GB of space from 14 snapshots, so really not a lot of space was used. By way of comparison, my @ subvol is 17.3GB and my home is 197.6GB. My habit is to keep 3 once-daily snapshots and weekly send 2 backups onto other devices. This is done in the background via a cronjob.

                              My roll-back procedure is to boot to another install (or liveUSB), mount the location that contains my install subvolume and it's snapshots, rename the failed install subvolume, re-snapshot the snapshot I want to roll-back to to the correct bootable subvolume name, and reboot. Takes a minute or less. To explain the naming thing just a bit further: My install subvolume is @KDEneon. My snapshots are @KDEneon_YYMMDD_HHMMSS. That way I know when they were taken. So if @KDEneon fails to boot, I rename it @KDEneon_bad, take a snapshot of @KDEneon_YYMMDDHHMMSS as @KDEneon, and reboot. Couldn't be easier or faster. Also, to roll-back from a non-failed install, you don't have to boot into another install first - just rename the subvolumes and reboot.

                              An EXT4 user wanting the same functionality would have to backup to another partition or device and then restore from that partition or device. Anyone do that every day? I doubt it. I snapshot anytime I think I might be doing something dangerous and daily just because. Obviously, a snapshot isn't going to save you from a failed drive. That's why I also do a weekly backup to two other drives. That too, happens automatically (via a cron activated script) while I'm working with no noticeable loss in usability of the system. I challenge anyone to find an easier way to do this.

                              Please Read Me

                              Comment

                              Working...
                              X