[SOLVED] Would like some assistance

[Snowhog]

For me, understanding btrfs is voodoo magic. I just have a difficult time wrapping my head around it. That said, I have my laptop formatted with btrfs. I also have an external 500 GB HDD, also formatted with btrfs that has nothing on it.

I'd like to play with btrfs full and incremental backups, and that's where I'd like some assistance. If one of you btrfs witch doctors would be so kind as to write two scripts (or two .bash_aliases file entries) that would let me manually do full backups and incremental backups, I'd be indebted. I don't want anything complicated. KISS. The laptop isn't a production unit, and it doesn't have anything important on it.

If one of you is game, let me know what you need to know about my laptop and the external HDD.

[GreyGeek]

Here is the script that I use when I don't do it manually.

#!/bin/bash
#
# script to create backup snapshots to /mnt/snapshots and a differential backup to /backup
# To be run as root from /
#
echo "Mounting drives"
eval "mount /dev/disk/by-uuid/038bd596-59b4-4bb0-98e5-3cbeec1dd339 /mnt"
eval "mount /dev/disk/by-uuid/c16e0247-d866-47f3-8d5e-770e746cd51c /backup"

NOW=$(date +%Y%m%d%H%M)
echo "Making today's snapshot"
MKSNAP='btrfs su snapshot -r /mnt/@ /mnt/snapshots/@'$NOW
eval $MKSNAP
eval 'sync'
eval 'sync'

echo "Finding previous snapshot as parent "
PREVSNAP=""
list=($(ls /mnt/snapshots/))
PREVSNAP=${list[-2]}
NOW=${list[-1]}
echo "Attempting incremental backup"
if [[ -s "/mnt/snapshots/"$PREVSNAP ]];
then
  MKINC='btrfs send -p /mnt/snapshots/'$PREVSNAP
  MKINC=$MKINC' /mnt/snapshots/'$NOW
  MKINC=$MKINC' | btrfs receive /backup'
  echo $MKINC
  eval $MKINC
  eval 'sync'
  eval 'sync'
  eval "sync"
  DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
  eval $DELSNAP
  eval 'sync'
  eval 'sync'
  eval 'sync'
  list=''
  list=($(ls /backup))
  DELSNAP='btrfs subvol delete -C /backup/'${list[0]}
  eval $DELSNAP
  eval 'sync'
  eval 'sync'
  eval 'sync'
  echo "Snapshots completed, oldest snapshots deleted"
  eval 'umount /backup'
  eval 'umount /mnt'
  eval 'sync'
  eval 'sleep 1'
  eval 'sync'
  eval 'sleep 1'
  eval 'sync'
  eval 'sleep 1'
  echo "Drives unmounted and 3 syncs done"
else
  echo 'Incremental backup failed using '$PREVSNAP' and '$NOW
  echo 'Drives still mounted, ready to clean up'
fi

The "PREVSNAP" has to exist on the rootfs subvolume and on the backup subvolume.

Here is what one of my incremental snapshots looked like:
btrfs send -p /mnt/snapshots/@202202262040 /mnt/snapshots/@202202281609 | btrfs receive /backup

The @202202262040, taken on Feb 26, 2022 at 8:40PM, resided on both /mnt/snapshots and /backup. It was sent to the /backup drive by my previous incremental send command. It is the parent snapshot. The snapshot taken on Feb 28, 2022 at 4:09PM is the child snapshot. The receive portion of the incremental command will see the name of the parent snapshot and look for it on the subvolume /backup. Using it as a template, the send portion of the command sends ONLY THE DIFFERENCE between the parent and child snapshots to the receive command. The receive command takes those differences and builds the child snapshot on /backup.

My external 500Gb spinner is plugged in before I run the script. I do not mount it. The script does that. It mounts the external drive to /backup, creates the child snapshot (which becomes the latest snapshot) and then using the previous snapshot, as determined by the list command, the incremental snapshot is taken.

I also have an internal NVMe 1Tb SSD which I mount manually on occasions to make incremental snapshots that may be 3 or more days between the parent and child. Even so, the incremental snapshot rarely takes more than 15 to 20 seconds to complete. Usually less than 10 secs.

Note that in order to use the send command the parent and child snapshots must be "read only", that is, made using the -r switch. My /mnt/snapshot contains:

:~# !1996
mount /dev/sda3 /mnt
root@jerry-hp17cn1xxx:~# vdir /mnt/snapshots/
total 0
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202202242057
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202202251913
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202202262040
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202202281609
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202203011815
root@jerry-hp17cn1xxx:~#

My usage shows:

:~# btrfs filesystem usage /mnt
Overall:
   Device size:                 441.04GiB
   Device allocated:            160.05GiB
   Device unallocated:          280.99GiB
   Device missing:                  0.00B
   Used:                        133.96GiB
   Free (estimated):            305.43GiB      (min: 305.43GiB)
   Data ratio:                       1.00
   Metadata ratio:                   1.00
   Global reserve:              218.44MiB      (used: 0.00B)

Data,single: Size:157.01GiB, Used:132.56GiB (84.43%)
  /dev/sda3     157.01GiB

Metadata,single: Size:3.01GiB, Used:1.39GiB (46.30%)
  /dev/sda3       3.01GiB

And it shows, with my entire system and 5 snapshots, 280.99Gib unallocated. Those same 5 snapshots would fill up my 500Gb spinner and the sixth one would fail to complete because it would run out of space. I learned that by experience.

PS- The UUID's I got by using "blkid" with the external HDD plugged in.

[GreyGeek]

Here is the result of my backup this evening.

$ sudo -i
root@jerry-hp17cn1xxx:~# cd /
root@jerry-hp17cn1xxx:/# /make_snapshot.sh
Mounting drives
mount: /mnt: /dev/sda3 already mounted on /.
Making today's snapshot
Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@202203022213'
Finding previous snapshot as parent
Attempting incremental backup
btrfs send -p /mnt/snapshots/@202203011815 /mnt/snapshots/@202203022213 | btrfs receive /backup
At subvol /mnt/snapshots/@202203022213
At snapshot @202203022213
Delete subvolume (commit): '/mnt/snapshots/@202202242057'
Delete subvolume (commit): '/backup/@202202171957'
Snapshots completed, oldest snapshots deleted
Drives unmounted and 3 syncs done
root@jerry-hp17cn1xxx:/#

[Snowhog]

Thanks, but that's just way too much voodoo foo for me. Consider that your script is "at the marathon running stage", and I'm not even able to crawl yet.

My internal HDD (/dev/sda) that Kubuntu 22.04 is installed on is formatted btrfs with one partition (/dev/sda1) (UUID=67fab1ac-b53c-4e10-9faf-f9a434156e) and two sub-volumes: subvol=@ and subvol=@home.

My external USB HDD (/dev/sdb) is formatted btrfs with one partition (/dev/sdb1) (UUID=7baf740a-eab6-40ce-9225-a99a1d0ee969) and no sub-volumes.

K(eep) I(t) S(imple) S(tupid) for me.

All I really want at this point are extremely simple, step-by-step instructions that will create a snapshot of @ that includes @home, and sends it to the external USB.

[GreyGeek]

Sure enough ...

sudo -i
cd /

to get to root.

Mount the btrfs root file system (<ROOTFS>) to /mnt

mount /dev/sda1 /mnt

If you do
vdir /mnt
you will see

/mnt/@
/mnt/@home

Here's mine:
/mnt
├── @
└── /mnt/snapshots/
├─── @202202251913
├─── @202202262040
├─── @202202281609
├─── @202203011815
└─── @202203022213

You don't see a @home subvolume because I moved @home into @ and commented out the stanza in /etc/fstab that mounts it.

Those two subvolumes, @ and @home, are your live, running system, / and /home. You didn't have to stop the system to do this step. You can continue to use the system while /dev/sda1 is mounted to /mnt.


Plug in and mount the External HDD drive

mount /dev/sdb1 /backup

Create a place to store snapshots that won't be visible to users (you) when /mnt is umounted, and because you don't want to clutter up /mnt with snapshots.

mkdir /mnt/snapshots

Create READ-ONLY snapshots for both @ and @home. Only read-only snapshots can be sent to external subvolumes using the btrfs send command. Using any other tool to create a copy on an external drive can corrupt the subvolume.

btrfs su snapshot -r /mnt/@           /mnt/snapshots/@yyyymmddhhmm
btrfs su snapshot -r /mnt/@home  /mnt/snapshots/@homeyyyymmddhhmm

Now, send the newly creates snapshots to the external USB drive.

btrfs send /mnt/snapshots/@yyyymmddhhmm  | btrfs receive /backup
btrfs send /mnt/snapshots/@homeyyyymmddhhmm  | btrfs receive /backup

Depending on the size of @ and @home, each send can take up to 20-30 minutes. After @yyyymmddhhmm and @homeyyyymmddhhmm are on /backup they will constitute the "parent" snapshot for a subsequent incremental backup, which will take only a few seconds for each one.

Note: do not attempt to use mv to send a snapshot to /backup. It won't work. The mv command only works within a subvolume, NOT between subvolumes. If, in the future, you want to restore your system using a snapshot on HDD then reversing the source and destination is how it is done.

I use the history command to recall previous mount commands and then issue !nnnn, where nnnn is the number of the specific history command I want to use. Saves lots of typing. If done manually, the btrfs send command is also among the history listings and can be copy+pasted and then edited for the new date before hitting the return key.

Next lesson when you are ready.

[Snowhog]

That's much more helpful (to me). Thank you.

First Issue:

mount /dev/sdb /backup

/backup didn't exist, so I created it on /dev/sda1

Second Issue:

When I plug in my external HDD, the Disk & Devices pop-up notifier appears. I just close the pop-up; I don't click on Mount and Open. In the konsole I type:

mount /dev/sdb /backup

and am told:

mount: /backup: wrong fs type, bad option, bad superblock on /dev/sdb, missing codepage or helper program, or other error.

If I then click on Mount and Open in Disk & Devices, the drive gets mounted and Dolphin opens showing the drive in a separate tab (465.8 GiB Removable Media). If I then repeat the mount command in konsole I get:

mount: /backup: /dev/sdb already mounted or mount point busy.



Update:

After the external HDD was mounted via Disk & Devices, I opened KDE Partition Manager. I selected /dev/sdb and highlighted /dev/sdb1 and right-clicked and selected Unmount.

Opened Konsole and typed:

sudo -i
mount /dev/sdb1 /backup

That worked.

So your instruction: mount /dev/sdb /backup is wrong? Needs to be /dev/sdb1?

[GreyGeek]

That's much more helpful (to me). Thank you.

First Issue:

mount /dev/sdb /backup

/backup didn't exist, so I created it on /dev/sda1

Second Issue:

When I plug in my external HDD, the Disk & Devices pop-up notifier appears. I just close the pop-up; I don't click on Mount and Open. In the konsole I type:

mount /dev/sdb /backup

and am told:

mount: /backup: wrong fs type, bad option, bad superblock on /dev/sdb, missing codepage or helper program, or other error.

If I then click on Mount and Open in Disk & Devices, the drive gets mounted and Dolphin opens showing the drive in a separate tab (465.8 GiB Removable Media). If I then repeat the mount command in konsole I get:

mount: /backup: /dev/sdb already mounted or mount point busy.



Update:

After the external HDD was mounted via Disk & Devices, I opened KDE Partition Manager. I selected /dev/sdb and highlighted /dev/sdb1 and right-clicked and selected Unmount.

Opened Konsole and typed:

sudo -i
mount /dev/sdb1 /backup

That worked.

So your instruction: mount /dev/sdb /backup is wrong? Needs to be /dev/sdb1?

Correct. Typo on my part. Fixed it in the post so that others won't trip over it.

How did your first snapshots go? How long did they take to send to the external HDD?

[Snowhog]

Correct. Typo on my part. Fixed it in the post so that others won't trip over it.

How did your first snapshots go? How long did they take to send to the external HDD?

Thanks for confirming and correcting the typo.

I haven't done a snapshot yet. I'd like to understand why my external HDD is being automounted. I get that that is likely the default configuration for this action in KDE Plasma. But it does make following your steps 'complicated'. When plugged in, it is being mounted to /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I guess I can just issue:

umount /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
mount /dev/sdb1 /backup

I just tried that, and that does work.

I've got /dev/sda1 mounted to /mnt
I've got /dev/sdb1 mounted to /backup
I created /mnt/snapshots
I created snapshots of @ and @home in /mnt/snapshots
I'm in the process of sending the @ snapshot to the external drive. When it's finished, I'll send @home.
Both sent/received without errors. Sweet.

The process of sending @ didn't take very long; five+ minutes I think. @home took even less time. This isn't a production laptop, so there isn't much on it other than the Kubuntu 22.04 install.

[GreyGeek]

Using the blkid requires the use of "/dev/disk/by-uuid/..." as in

mount /dev/disk/by-uuid/038bd596-59b4-4bb0-98e5-3cbeec1dd339 /mnt

The blkid command gives a list of UUID's for attached devices. There are several paths that can be used.

$ vdir /dev/disk
total 0
drwxr-xr-x 2 root root 320 Mar  4 07:26 by-id
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-label
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-partuuid
drwxr-xr-x 2 root root 180 Mar  4 07:26 by-path
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-uuid

The /dev/sdXn path of a device can change randomly without warning. That's why UUID is used. The same UUID points to the /dev/sdXn path as it currently is following bootup. When mounting manually using /dev/sdXn is fine because it is determined at bootup or plugin. Inside a script using /dev/sdXn can fail because /dev/sdXn may not point to the same device. When first testing btrfs I used a raid1 using two devices, sda1 and sda2. When I plugged another SATA drive into my Acer and booted up I expected it to be sdc1. To my shock I discovered that my raid1 was sda1 and sdc1, even though they were the same two devices. The new SATA drive was sdb1.

Here are the paths given in MY /dev/disk/by-uuid directory

$ vdir /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 15 Mar  4 07:26 2b08ee29-bf29-4b5c-bb40-6aafaf66cec8 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 15 Mar  4 07:26 4ea5991d-7da6-4387-907d-acef20d047de -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 10 Mar  4 07:26 83cb8dfd-aadc-4558-884d-eeafac2303bf -> ../../sda2
lrwxrwxrwx 1 root root 10 Mar  4 07:26 9bc19383-57db-4a32-938d-8466a133d94d -> ../../sda3
lrwxrwxrwx 1 root root 10 Mar  4 07:26 C853-95B5 -> ../../sda1

The paths following the "-->" are determined at boot, but the UUID is always the same for the same device. That's why I use the UUID in my scripts but I use the /dev/sdXn manually.

[Snowhog]

So I had a thought, and no, it didn't hurt much.

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I can take advantage of that and use:

btrfs send /mnt/snapshots/@yyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
btrfs send /mnt/snapshots/@homeyyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969

[GreyGeek]

So I had a thought, and no, it didn't hurt much.

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I can take advantage of that and use:

btrfs send /mnt/snapshots/@yyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
btrfs send /mnt/snapshots/@homeyyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969

Mmm.. never used that approach. Interesting.

Those times aren't bad, but once you have a "parent" on your backup drive the incremental send is extremely fast. In your case I'd guess that it would complete in under 5 seconds.

[Snowhog]

Here is the script that I use when I don't do it manually.

You run this script from / as root, but where do you have the script saved at?

So, your script, it creates snapshots that are kept on your internal HDD (/mnt/snapshots) and incremental snapshots that are kept on the external HDD (/backup)?

If that is correct, how are you actually protected if your internal HDD fails?

[GreyGeek]

It is save at:

$ sudo vdir /
[sudo] password for jerry:
total 40
drwxr-xr-x 1 root root 0 Jan 3 16:23 backup
lrwxrwxrwx 1 root root 7 Oct 28 08:30 bin -> usr/bin
drwxr-xr-x 1 root root 1422 Feb 24 21:01 boot
drwxr-xr-x 22 root root 4580 Mar 4 07:26 dev
drwxr-xr-x 1 root root 4562 Mar 2 15:29 etc
-rw-r--r-- 1 root root 515 Jan 5 2020 he-ipv6.service
drwxr-xr-x 1 root root 10 Jan 3 16:28 home
lrwxrwxrwx 1 root root 7 Oct 28 08:30 lib -> usr/lib
lrwxrwxrwx 1 root root 9 Oct 28 08:30 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 9 Oct 28 08:30 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 10 Oct 28 08:30 libx32 -> usr/libx32
-rwxr-xr-x 1 root root 1600 Feb 8 15:58 make_snapshot.sh
drwxr-xr-x 1 root root 10 Jan 2 14:59 media
drwxr-xr-x 1 root root 0 Oct 28 08:30 mnt
-rwxr-xr-x 1 root root 343 Feb 8 15:59 mount_drives.sh
...

I have two internal SSD's and a host of external HDDs & and an SSD.
I plug my external 1Tb spinner (which was the HDD that came with this new HP and I put it into a USB HD Caddy) and then run the script using:

sudo -i
cd /
/make_snapshots.sh

The script umounts my external USB HDD and /mnt when it is done.

Using the history command, I the use !nnnn to run the most recent command to mount my NVMe 1Tb SSD to /backup and the system to /mnt. Then I issue

vdir /mnt/snapshots

and

vdir /backup

to see what's on them. I pick the most recent snapshot that is the same on both to use as the parent, and then choose the newest snapshot on /mnt/snapshots as the child, the snapshot I want to send to /backup.

send -p /mnt/snapshots/@parentyyyymmddhhss /mnt/snapshots/@mostrecentsnapshotyyyymmddhhss | btrfs receive /backup

Then I use

btrfs filesystem sync /backup

two or three times, or until it returns immediately, indicating nothing was done. (You can also use 'sync')

Then I umount /mnt and /backup and then exit root and then exit the Konsole.
I use the NVMe SSD less frequently than I do the USB Caddy containing the 1Tb spinner, sometimes only once a week or so. I've used the send command sans -p a couple times and it takes about 25 minutes to send my snapshot to the NVMe SSD. I do that if I don't have a "parent" snapshot on the NVMe.

So, all in all, with the aid of the history command and the ability to execute previous history command using "!nnnn" I've pretty much been doing my snapshots manually. In all I have four USB drives that I send backup snapshots to, and some 128Gb and 256Gb USB sticks as well. For a long time I always kept a LiveUSB stick of Kubuntu in my pants watch pocket and a USB stick containing a recent snapshot of my system. Since I've combined @home into @, I only need to track and keep one snapshot and have no risk of mixing up the @ with the wrong @home.

[jlittle]

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969.

If the volume has a label, I would expect it to be mounted in /media/paul with that label, rather than the UUID.

[Snowhog]

jlittle The drive doesn't have a label, so the UUID is identified.