Rolling back with Btrfs

[GreyGeek]

On August 10th I decided to install Mycroft and play with it. After looking over the installation procedures and source code I realized that the developers didn't include an uninstall script. So, reading the dev setup script it became obvious that i would have to manually uninstall about 30 programs, delete half a dozen directories, remove settings in bashrc and other places, delete the plasmoid files, etc.... It could take 30 minutes or three hours. Assuming, of course, that some of the installed packages don't get dependency connections to important packages. Who knows?

So, before the install I made backup snapshots of @ and @home. This morning I decided to rollback to those two snapshots instead of going through all the manual agony mentioned above. NOTE: The following steps were performed on the LIVE system. I did NOT have to boot into a LiveUSB to do these steps!

1) Open a Konsole and issue
"sudo -i"

2) Mount sda1 on /mnt
"mount /dev/sda1 /mnt"
(Note: I am running a RAID1 system with two 750Gb Hds. Mounting either sda1 or sdc is sufficient.)

EDIT: while using sda1 will work OK on a laptop with only one HD, adding a second or third HD may cause the dev/sdX1 designation to switch X values, making sda1 become sdc1. Don't assume! The better method is to use "blkid" to list the UUIDs of your HDs, select the UUID of what ever Btrfs root was installed on, and
mount /dev/disk/by-uuid/whatevertheuuidis /mnt


3) Move @ to @_old
"mv /mnt/@ /mnt/@_old", followed by "sync"

4) Move @home to @home_old
"mv /mnt/@home /mnt/@home_old" followed by "sync"

5) Copy /mnt/snapshot/@_bkup20170810 to /mnt/@
"btrfs subvol snapshot /mnt/snapshots/@_bkup20170810 /mnt/@" followed by "sync"

6)Copy /mnt/snapshots/@home_bkup20170810 to /mnt/@home
"btrfs subvol snapshot /mnt/snapshots/@home_bkup20170810 /mnt/@home"

7) Delete @_old and @home_old
"btrfs subvol delete -c /mnt/@_old"
"btrfs subvol delete -c /mnt/@home_old"

8) Exit root and Konsole
"exit"
"exit"

9) Reboot.

Total elapsed time, about three minutes for steps 1 to 8. Reboot time was normal. Now, reapply the updates that were applied while I was tinkering with Mycroft.

I cannot see myself returning to EXT4 and giving up the power of Btrfs.

EDIT: It took me MUCH longer to do this writeup than to do the actual rollback!

[#]BTRFS[/#]

[oshunluvr]

+1

Right there with ya...

[jlittle]

Thanks for that GreyGeek.
Your step 2 has me scratching my head; aren't @ and @home already mounted, on / and /home? Can you mount a btrfs subvolume more than once? And, why?
Doing step 7 before step 9, well, it shows confidence. Would it be okay to delay step 7?

Regards, John Little

[GreyGeek]

Thanks for that GreyGeek.
Your step 2 has me scratching my head; aren't @ and @home already mounted, on / and /home? Can you mount a btrfs subvolume more than once? And, why?
Doing step 7 before step 9, well, it shows confidence. Would it be okay to delay step 7?

Regards, John Little

Yes, @ and @home are mounted as "/" and "/home" in fstab using UUIDs. Being able to work on a live system is a feature of BTRFS. It has many other features as well, and it is GPL and not encumbered by Oracle, like ZFS is.

I'll document the steps:
1) Open a Konsole and issue
"sudo -i"

You can't mount a /dev/sdX unless you are root.

2) Mount sda1 on /mnt
"mount /dev/sda1 /mnt"

This gives you access to @ and @home, which are inaccessible from within the system, without using this method, because you can't access the ROOT of BTRFS from / or /home. From within /home/jerry I can access /home by using "cd .." Then I can access / from /home by using "cd ..". However, using cd.." again does not take me "above" root (/), so I cannot access the ROOT of BTRFS without the mount procedure, as the user root.


3) Move @ to @_old
"mv /mnt/@ /mnt/@_old", followed by "sync"
4) Move @home to @home_old
"mv /mnt/@home /mnt/@home_old" followed by "sync"

I can use the move command, mv, to move and/or rename a subvolume, but I cannot use mv to move a subvolume outside of the drive pool on which its ROOT exists. I have a third 750Gb HD in an HD Caddy residing where the CDROM used to be. It is seen as /dev/sdb1. I mount it to /backup when I want to store a snapshot on another drive that is not part of my system pool. To do that I use BTRFS's "send" and "receive" commands, designed specifically to move copies of snapshots to storage devices that are not in the pool or are remote. After I send one or more snapshots to /backup I unmount it.

5) Copy /mnt/snapshot/@_bkup20170810 to /mnt/@
"btrfs subvol snapshot /mnt/snapshots/@_bkup20170810 /mnt/@" followed by "sync"
6)Copy /mnt/snapshots/@home_bkup20170810 to /mnt/@home
"btrfs subvol snapshot /mnt/snapshots/@home_bkup20170810 /mnt/@home" followed by "sync"

When I make a snapshot of @ and @home I use the "-r" parameter to make it read-only. That fixes the snapshot at the point in time at which it was made. All of my snapshots on /mnt/snapshots are read-only. However, I cannot use mv to move a copy of /mnt/snapshots/@bckup to /mnt/@ because that would make @ read-only and it would fail to function properly when rebooting. So, I make a rw snapshot of my ro snapshot, naming it @ or @home, as the case may be. Since @ and @home have already been mv'd to @_old and @home_old I do not have a naming conflict. Note also, that I cannot use mv to move a snapshot from /backup to /mnt because even though /backup is formatted using BTRFS it is not part of the BTRFS pool, i.e., it has not beed added to /dev/sda1 and /dev/sdc, which make up my RAID1 pool.

7) Delete @_old and @home_old
"btrfs subvol delete -c /mnt/@_old"
"btrfs subvol delete -c /mnt/@home_old"

This is the standard BTRFS subvolume delete command, with the commit parameter, "-c", added to avoid the use of "sync".Here is a list of some of the features of BTRFS:

ext4 is a journaling filesystem.

btrfs is a journaling filesystem that also has these features not in ext4:
1. copy-on-write
2. checksums on data and metadata to detect file and filesystem corruption. This feature is very handy if you have a lot of data, but calculating and storing checksums every time you write data and calculating checksums again every time you read data does make it slower than filesystems without the feature. Ext4 does checksums for metadata but not on file data.
...
4. Transparent compression.
5. Snapshots.
6. Subvolumes. You can mount a subvolume without mounting the parent volume, snapshot individual subvolumes, and designate different replication levels for subvolumes than for the parent volume (RAID 1 or RAID 10 or similar).

I don't use compression or encryption, but they can be added at creation, or later. You can also switch your BTRFS installation from being a SINGLE to some level of RAID, with either or both the metadata and the data being SINGLE or RAID, or switch back to SINGLE. It's how I switched my single drive SINGLE BTRFS system to a multi-drive RAID BTRFS system. I've been using BTRFS for almost three years and have yet to experience a single problem with it. Here are some of my stats:

[FONT=monospace][B][COLOR=#000000]$ btrfs filesystem df -h /[/COLOR][/B]
Data, RAID1: total=132.00GiB, used=129.85GiB
System, RAID1: total=32.00MiB, used=48.00KiB
Metadata, RAID1: total=2.00GiB, used=1.17GiB
GlobalReserve, single: total=219.98MiB, used=0.00B

[B][COLOR=#000000]$ btrfs filesystem show /[/COLOR][/B]
Label: none  uuid: 12980ae8-4117-4cc5-bbb8-8065e82af93d
        Total devices 2 FS bytes used 131.02GiB
        devid    1 size 691.19GiB used 134.03GiB path /dev/sda1
        devid    2 size 698.64GiB used 134.03GiB path /dev/sdc

[B][COLOR=#000000]$ sudo btrfs subvolume list -q /[/COLOR][/B]
ID 515 gen 197241 top level 5 parent_uuid 1c1bdacd-fac1-894b-91cd-ad7218df1d51 path snapshots/@home_bkup20170725
ID 516 gen 197242 top level 5 parent_uuid 4e69c5ac-6801-a545-9324-8d22ce662a31 path snapshots/@_bkup20170725
ID 518 gen 205754 top level 5 parent_uuid 4e69c5ac-6801-a545-9324-8d22ce662a31 path snapshots/@_bkup20170801
ID 519 gen 205755 top level 5 parent_uuid 1c1bdacd-fac1-894b-91cd-ad7218df1d51 path snapshots/@home_bkup20170801
ID 549 gen 223415 top level 5 parent_uuid 1c1bdacd-fac1-894b-91cd-ad7218df1d51 path snapshots/@home_bkup20170810
ID 550 gen 223414 top level 5 parent_uuid 4e69c5ac-6801-a545-9324-8d22ce662a31 path snapshots/@_bkup20170810
ID 554 gen 224453 top level 5 parent_uuid 14ab98d9-ea35-5241-8608-ea159b23661c path @
ID 555 gen 224453 top level 5 parent_uuid 8ef29493-b73a-c84c-ac41-14f057884dbc path @home
[/FONT]

[GreyGeek]

jlittle, I forgot to explain "why?".

So BTRFS users can do a 3 minute rollback with no difficulty.

Also, I forgot to mention that anything one has accumulated since the date of the snapshot used for the rollback must be saved off to a different media and restored after the rollback is complete.

EDIT:
Or, one can do a daily incremental backup using the send command "-p" parameter to create the snapshot on a backup HD.
https://btrfs.wiki.kernel.org/index....emental_Backup

[NietGiftig]

@GreyGeek
Nice, all that BTRFS.
I have mastered just enough (5% ?) of Linux past year to make it my only OS, after >25 years of Windows.
And with my age, I do still have mostly the same color hair I had for the last 64 years.
Some grey hairs, but I would not connect that with Linux.
But reading post #4 of this thread and many other threads over BTRFS, I think that BTRFS would greatly expand the number of grey hairs on my head.
I think that BTRFS is not intended for the casual user, but more for geeks, some with grey hair.
No offence intended of-course.
I will continue to read your articles and advice about Linux and BTRFS.

Hope there will be someone who can bring all those lovely features of BTRFS in a nice, logical GUI together.
Of course aimed at normal (simple?) users.

[GreyGeek]

@GreyGeek
Nice, all that BTRFS.
I have mastered just enough (5% ?) of Linux past year to make it my only OS, after >25 years of Windows.
And with my age, I do still have mostly the same color hair I had for the last 64 years.
Some grey hairs, but I would not connect that with Linux.

Stress caused by Windows, which I used from Win1.0 (1985), after 7 yrs of MSDOS, through Win7.0, caused a lot of my lack of hair and greying of what didn't fall out.

I began using Linux on May 1, 1998, with RH 5.0. Life on my computers has been fun and easy every since. BTRFS is the crowning jewel that makes Linux life a breeze. Three minutes and a reboot to do a rollback can't be any easier. I've been using BTRFS for 2 1/2 years and NO filesystem has been easier to use or maintain.

But reading post #4 of this thread and many other threads over BTRFS, I think that BTRFS would greatly expand the number of grey hairs on my head.

I could have fired up mc, or Krusader or, perhaps even Dolphin, to do the rename/move, but the CLI (Command Line Interface) "mv" command is easy. And, after I did it once, I merely recall the bash history and re-execute the command. And, as a bonus, CLI commands are faster than similar commands using GUI (but not always). Actually, I am NOT a CLI guru and try to use a GUI as much as possible. That's because at 76 my greying brain can't remember switches or parameters that many CLI commands require. About the only time I use BTRFS commands is when I make a ro (read only) snapshot of @ (root) and @home (/home), and the send & receive commands. I consult a printed cheat sheet of BTRFS commands when ever I do anything with my file system.

I actually considered writing a small GUI to make the snapshots and do a rollback but as you can see the steps are so easy that it is a LOT easier and quicker to use the CLI. The CLI would have to prompt for which subvolume to snapshot, what the destination snapshot name would be, if it were to be a ro or rw, and a lot of other possible parameters which I never use, or used only once, like when I converted my BTRFS "SINGLE" fs, residing on a single 750Gb HD, to a RAID1 residing on two 750Hds. A change that went off without a hitch.

A couple years ago I was trying out a tool called "snapper" (it's in the repository), which automatically created pre and post snapshot when ever an update was done, or when called by the user. I don't recommend using it because it makes TOO MANY snapshots and can eat up HD space if the default config is not modified. I decided I could turn off automatic snapshots in snapper and do exactly what I wanted to do with a simple bash script to call it when I wanted to do something:

#!/bin/bash
# created by Jerry L Kreps on July 20, 2015 and released under the GPL 2.0.
# This script merely creates a snapshot in both root and home with the designation of PRE or POST as the type, 
# which indicates that the user created it before an action or afterwards.
# This script is run with PRE as the TYPE before an action which you may want to reverse
# AND this script run again with POST as the TYPE after that action has been completed.  
# Both snapshots are singletons because no timeline is used.  
# Only the "PRE" or"POST" in the description, along with a timestamp, links the pre to the post snapshot.
# To reverse the action run the following two snapper commands:
#
# snapper -c home udochange n..m   where n or o is the number of the PRE and m or p is the number of the POST snapshot
# sudo snapper -c root undochange o..p
#
# After running those two commands both of the empty snapshots folders can be deleted using
#
# snapper -c home delete n-m
# sudo snapper -c root delete o-p
#
#
#
NOW=$(date +%Y%m%d%H%M)
echo Enter snapshot type PRE or POST:
read TYPE
echo Enter description
read DESC
STR=$TYPE" "$DESC" "$NOW
echo $STR
HCMD='snapper -c home create -d "'${STR}'"'
RCMD='sudo snapper -c root create -d "'${STR}'"'
eval "$HCMD"
eval "$RCMD"

Snapper created its snapshots at the "/" and "/home" level, which I did not like, so I dropped it and started doing things manually.

One thing that is neat is that if, by chance, I deleted something on my working system that I wanted back I could mount /dev/sda1 on /mnt and then use mc to browse to /mnt/snapshots/@home_bkupsomedate and dive into my home directory and copy the file I deleted back to my live system. Instant file rollback!

I think that BTRFS is not intended for the casual user, but more for geeks, some with grey hair.
No offence intended of-course.
I will continue to read your articles and advice about Linux and BTRFS.

None taken. The maintainer of EXT4, T'so, says that BTRFS is the fs future of Linux, and I agree with him. Oracle license problems will keep most of ZFS off of Linux unless Ellison gives up trying to buy more fighter jets or Pacific Islands.

BTRFS had only one weakness, the RAID56 "write hole", a loss of data integrity if a RAID 5 or 6 installation suffered a sudden power loss. A patch was submitted to fix that bug on August 1st, 14 days ago. It is for the 4..12 kernel.

Hope there will be someone who can bring all those lovely features of BTRFS in a nice, logical GUI together.
Of course aimed at normal (simple?) users.

I retired in 2008, after nearly 40 years of writing software, ending up using the Qt SDK for the last four, with the promise to myself that I would write the code that I wanted to write, for me and not because a client asked for it. However, I have not written a single line of code, except for some paltry bash scripts, since 2008. But, I did teach my grandsons how to fish for Bass, identify eatable insects, make pond-scum soup, create fire with sticks and with a striker, and my youngest now wants to learn how to fly fish. That, and helping my wife recover from her 2nd open heart surgery and then a broken hip, didn't leave much time for coding. Besides, I've probably forgotten how to use the tools.

[GreyGeek]

Yesterday I ran into an "opps!" attempting to rollback after an update destroyed my laser printer configuration and stopped the printer working. After spending a couple hours checking logs, reconfiguring and reinstalling hplip and cups I decided to chuck it and do a Btrfs rollback.

When I rebooted I got a grub error about not being able to find "@/boot/grub/something.mod".

I rebooted using my Neon LiveUSB stick and did some exploring. The "@_bkup20170918" root snapshot wasn't a snapshot of @. It turned out to be a snapshot of @home. How did that happen

Carelessness. I use the CLI. I also use bash history to recall previous snapshot commands, changing the date part to reflect the current date. So, what should have been
btrfs subvol snapshot -r /mnt/@ /mnt/snapshots/@_bkup20170918
was actually
btrfs subvol snapshot -r /mnt/@home /mnt/snapshots/@_bkup20170918
and I didn't catch the /mnt/@home part. Going too fast, typing too fast. Not thinking enough.

Fortunately, I had a slightly older backup set that wasn't goofed up, and now my printer is back working the way it used to, which is perfect.

Ya gota love Btrfs!!

[oshunluvr]

Yesterday I ran into an "opps!" attempting to rollback after an update destroyed my laser printer configuration and stopped the printer working. After spending a couple hours checking logs, reconfiguring and reinstalling hplip and cups I decided to chuck it and do a Btrfs rollback.

When I rebooted I got a grub error about not being able to find "@/boot/grub/something.mod".

I rebooted using my Neon LiveUSB stick and did some exploring. The "@_bkup20170918" root snapshot wasn't a snapshot of @. It turned out to be a snapshot of @home. How did that happen

Carelessness. I use the CLI. I also use bash history to recall previous snapshot commands, changing the date part to reflect the current date. So, what should have been
btrfs subvol snapshot -r /mnt/@ /mnt/snapshots/@_bkup20170918
was actually
btrfs subvol snapshot -r /mnt/@home /mnt/snapshots/@_bkup20170918
and I didn't catch the /mnt/@home part. Going too fast, typing too fast. Not thinking enough.

Fortunately, I had a slightly older backup set that wasn't goofed up, and now my printer is back working the way it used to, which is perfect.

Ya gota love Btrfs!!

Lucky!

Hey, what about a simple bash cronjob to do a regular snapshot, retaining a set number of snapshots on a set time table, and automatically delete them as they age? Seems simple enough. Follow it up with a couple simple scripts to take special snapshots - like just before installation or updating - using the same structure. Just because snapper wasn't written well does mean it can't be done properly.

This is also a good example of how a small but well written GUI would be of major benefit. One could configure target subvolumes (@ and @home) and click a button to shot a snapshot or create a simple cronjob to do a daily snap.

I think this might bring more users to btrfs.

Hmm, now thinking about a Dolphin service menu...

[GreyGeek]

I have three HD's in my laptop. Two are pooled to hold the fs. The third is a Btrfs pool mounted as /backup, and used solely for the purpose of storing Btrfs snapshots and mounted only when send & receive is used.

I have reverted from a two HD RAID1 system back to a two HD singleton. I keep two to four timestamped @ and @home ro snapshots on /mnt/snapshots subvolume, and I send & receive all snapshots to /backup. I make room on /backup for the latest snapshots, so it tends to be 70-80% full.

After yesterday's debacle I decided to use a script to do snapshots instead of up arrowing to the previous sequence of snapshot commands and modifying them manually. I am going to store the script on /mnt so it won't be accessible from within the system, only when /dev/sda1 is mounted to /mnt with root.

I've considered a cron job but using snapper led me to the conclusion that I don't need regular snapshots. I need to snapshot my system only when significant changes may be made to it. Primarily large updates or installation of or changes in major programs. When I tested the P2P programs, for example.

I thought about a Btrfs GUI as well. Where I retired from programmers were required to fill out a manual form of the hours they worked, the vacations and sick leave they took and bring all previous totals forward, to be added to the current totals for yearly values. I got tired of that and wrote a GUI for my own use. It printed out copies of the manual forms with my data in it. Within a few months all the programmers and staff in our department were using it. The head of personal talked my boss into having me write a client-server version with tons of stuff they needed but I didn't. Before long over 450 people at the main office were using it and several thousand state employees scattered around the state. What was written to make MY job easier became my job. I thought about that Btrfs GUI and at 76 I don't need another programming jobs, especially when a script will be more than necessary for my needs. (BTW, I gave my program the name "time_records". After a while the staff called it "T-Rex". LOL!
Back in the late 1980s the President of the college where I was teaching at asked me to computerize the school. I wrote a 10 level enterprise accounting system (GAAP) which required double entry book keeping. The staff at the business office seemed to have done everything "on the cuff" and the program forced them into general journals, posting, etc.... They called that portion of the system "Beelzebub".

To avoid future misnaming problems I am going to modify a script I found on the web to suit my needs:
btrfs_backup.sh

#!/usr/bin/env bash

# based on https://btrfs.wiki.kernel.org/index.php/Incremental_Backup

if [ "$(id -u)" != "0" ]; then
    echo "[!] must be root." 1>&2
    exit 1
fi

_SOURCE=$1
_TARGET=$2
_NAME=$3

if [ -e "$_SOURCE/.BACKUP" ] || [ -e "$_TARGET/$_NAME/.BACKUP" ]; then
    echo "[!] found existing working directory. either a backup is in process or last backup failed leaving the directory not cleaned up."
    exit 1
fi

if [ ! -e "$_TARGET/$_NAME" ]; then
    btrfs subvolume create "$_TARGET/$_NAME"
fi

btrfs subvolume snapshot -r "$_SOURCE" "$_SOURCE/.BACKUP"

if [ -e "$_TARGET/$_NAME/.BACKUP-last" ] && [ -e "$_SOURCE/.BACKUP-last" ]; then
    echo "[i] performing a incremental backup."
    btrfs send -p "$_SOURCE/.BACKUP-last" "$_SOURCE/.BACKUP" | btrfs receive "$_TARGET/$_NAME"
else
    echo "[i] performing a full backup."
    btrfs send "$_SOURCE/.BACKUP" | btrfs receive "$_TARGET/$_NAME"
fi

btrfs subvolume snapshot -r "$_TARGET/$_NAME/.BACKUP" "$_TARGET/$_NAME/$(date +%Y-%m-%d-%H%M%S)"

if [ -e "$_SOURCE/.BACKUP-last" ] && [ -e "$_SOURCE/.BACKUP" ]; then
    btrfs subvolume delete "$_SOURCE/.BACKUP-last"
fi

if [ -e "$_TARGET/$_NAME/.BACKUP-last" ] && [ -e "$_TARGET/$_NAME/.BACKUP" ]; then
    btrfs subvolume delete "$_TARGET/$_NAME/.BACKUP-last"
fi

mv "$_SOURCE/.BACKUP" "$_SOURCE/.BACKUP-last"
mv "$_TARGET/$_NAME/.BACKUP" "$_TARGET/$_NAME/.BACKUP-last"

Or, I may modify my snapper script.

[woodsmoke]

Ok i am just a stoopid guy...an old hardware kinda guy... i put colored fans on ole MOBOS that I grabbed off the beltways...



For us old hardware kinda guys...noobs to some...

Please, in simple terms that the ol woodsmoker can understand...tell me

a) what you started out with
b) what you wanted to end up with
c) what you ended up with

in simple...woodsmokian terms...

[GreyGeek]

Didn't we have a thread about that a short while ago? And didn't you start it?\
GreymymemoryaintwhatitusedtobeGeek