Would like some assistance

[Snowhog]

Made some cosmetic changes. The script works. As already stated, it just makes full snapshots of my @ and @home subvolumes and then sends them to my external USB HDD. But that's enough for now. Here is the 'latest version' of the script:

#!/bin/bash
# snapshot.sh
#
# Variable declarations
#
NOW=$(date +%Y%m%d%H%M)
MKSNAP1="btrfs su snapshot -r /mnt/@ /mnt/snapshots/@$NOW"
MKSNAP2="btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home$NOW"
SNDSNAP1="btrfs send /mnt/snapshots/@$NOW | btrfs receive /backup"
SNDSNAP2="btrfs send /mnt/snapshots/@home$NOW | btrfs receive /backup"
TIMESTART=$( date '+%H:%M:%S' )
STARTCLOCK=$(date -u -d "$TIMESTART" +"%s")

# To be run as root from /
#
# Before executing this script:
# Plug in external HDD. Ignore Disk & Devices pop-up.
# External drive will be powered and identified by system but not yet mounted.
#
echo "Mounting drives"
# mount internal HDD to /mnt
eval "mount /dev/disk/by-label/LAPTOP /mnt"
echo "Internal HDD mounted to /mnt"
# mount external HDD to /backup
eval "mount /dev/disk/by-label/USBHDD /backup"
echo "External USB HDD mounted to /backup"
echo ""
# capture current date and time
echo "Timestamp: $NOW"
echo ""
# Create snapshots
echo "Making today's snapshot of @"
eval $MKSNAP1
eval 'sync'
eval 'sync'
echo "@$NOW successfully created"
echo ""
echo "Making today's snapshot of @home"
eval $MKSNAP2
eval 'sync'
eval 'sync'
echo "@home$NOW successfully created"
echo ""
# Send snapshots
echo "Sending @$NOW and @home$NOW to external USB HDD"
eval $SNDSNAP1
eval 'sync'
eval 'sync'
eval $SNDSNAP2
eval 'sync'
eval 'sync'
TIMESTOP=$( date '+%H:%M:%S' )
STOPCLOCK=$(date -u -d "$TIMESTOP" +"%s")
DURATION=$(date -u -d "0 $STOPCLOCK sec - $STARTCLOCK sec" +"%H:%M:%S" )
echo "@$NOW and @home$NOW successfully sent to external USB HDD"
echo ""
echo "Creating and Sending Snapshots took: $DURATION"
echo ""
# Show snapshots on Internal and External USB HDD
echo "Contents of /mnt/snapshots"
vdir /mnt/snapshots
echo ""
echo "Contents of /backup"
vdir /backup
# echo ""
echo "Unmounting drives"
eval 'umount /backup'
eval 'umount /mnt'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
echo ""
echo "Drives unmounted and 3 syncs done"

The DURATION variable almost certainly can be written better, but I spent a reasonable amount of time trying to do so, and failed. Using the TIMESTART, TIMESTOP, STARTCLOCK, and STOPCLOCK variables was the only way I could get it to work. Maybe smarter folk here can see how to do it 'cleaner'.

[jlittle]

The DURATION variable almost certainly can be written better, but I spent a reasonable amount of time trying to do so, and failed. Using the TIMESTART, TIMESTOP, STARTCLOCK, and STOPCLOCK variables was the only way I could get it to work. Maybe smarter folk here can see how to do it 'cleaner'.

Shell arithmetic is better for a difference. (Using -u may cause grief if you happen to be running at UTC midnight, which would be 6 pm in Andover, MN; one must use -u with -d @.)

STARTCLOCK=$(date +%s)
# run stuff...
STOPCLOCK=$(date +%s)
SECS=$((STOPCLOCK - STARTCLOCK))
DURATION=$(date -d "0 $SECS sec" +%H:%M:%S)

The last three lines could be merged to one, if you prefer brevity to readability:
DURATION=$(date --date "0 $(($(date +%s) - STARTCLOCK)) sec" +%H:%M:%S)

Nifty to use date --d with a date of "0 somenumber sec" and then a format with only the time. The "0" means today at 00:00. If that's too clever, or the job might run over 24 hours,
DURATION=$(printf "%02d:%02d:%02d" $((SECS / 3600)) $((SECS % 3600 / 60)) $((SECS % 60)) )

[Snowhog]

jlittle Thank you! Getting it 'brief' wasn't the intended goal, but seeing how it can be done 'educates' me, and that is always a plus. I knew there were folks here who could improve on what I had attempted!

[GreyGeek]

Nice code, jlittle !! I'm going to steal it and add it to my script!
Snowhog , how long did it take to send to full snapshots to your remote HDD

[Snowhog]

GreyGeek I'll let you know in a bit. I cleaned up everything; no snapshots in /mnt/snapshots or /backup. I'm rebooting to have a totally fresh environment to begin from.

Finished. It took 16m10s from the time I launched the script until it finished.

My laptop is an older HP Pavilion g7 with an Intel i3 64-bit processor and 8 GiB of RAM. The USB HDD is a spinner (SeaGate 500B FreeAgent Go), so it isn't speedy.

Still, I'm happy.

Your script makes a new full snapshot and then attempts an incremental snapshot. Would you not want to separate those two actions; have separate scripts? When one is implementing a backup scheme, one generally establishes a Full Backup and Incremental Backup schedule, say Full Backups on Mondays, and Incremental backups on Tuesdays through Sundays. Something akin to that.

Still trying to understand and learn.

[Snowhog]

I'm not happy with my external USB HDD staying powered after the script finishes. Yes, it gets unmounted, but...

So, I added this after the last eval 'sleep 1':

# Begin 2022-03-08 modification
echo "Powering off external USB HDD"
eval 'udisksctl power-off -b /dev/sdb1'
# End 2022-03-08 modification

This nicely powers off the drive. Sweet.

[Snowhog]

BTW, comments do not show up on the "New Posts" or "Today's Posts" listings.

Yes, I'm aware of that. It's either 'by design' or a quirk within the vBulletin core programming. It's a mild complaint over on forum.vbulletin.com.

[GreyGeek]

I'm not happy with my external USB HDD staying powered after the script finishes. Yes, it gets unmounted, but...

So, I added this after the last eval 'sleep 1':

This nicely powers off the drive. Sweet.

Sweet indeed!

I plug my external USB HDD into an active (powered) USB3 hub which has buttons to turn the power to each port on or off. When the cursor comes back I push that button, which powers down the drive, and then I unplug the drive. If I was using a passive hub I'd use your technique.

I would recommend one change to the power off command: use the /dev/disk/by-uuiid/<theuuidofthathdd> instead of /dev/sdb1 because, as I've experienced, /dev/sdXn assignments can change without warning but the uuid assignments are guaranteed to be unique and never change.

[Snowhog]

I would recommend one change to the power off command: use the /dev/disk/by-uuiid/<theuuidofthathdd> instead of /dev/sdb1 because, as I've experienced, /dev/sdXn assignments can change without warning but the uuid assignments are guaranteed to be unique and never change.

I'll look into that. The udisksctl would have to support that identification. I don't know that it doesn't, but just say'n.

Added:
Yup. Works. I also tried using by-label. That also works.

[GreyGeek]

GreyGeek I'll let you know in a bit. I cleaned up everything; no snapshots in /mnt/snapshots or /backup. I'm rebooting to have a totally fresh environment to begin from.

Finished. It took 16m10s from the time I launched the script until it finished.

That's is the same general time my full backup takes to complete. That's why I do mainly incremental backups. That, and to avoid stressing my NVMe SSD.

My laptop is an older HP Pavilion g7 with an Intel i3 64-bit processor and 8 GiB of RAM. The USB HDD is a spinner (SeaGate 500B FreeAgent Go), so it isn't speedy.

Still, I'm happy.

That's all that matters.

Your script makes a new full snapshot and then attempts an incremental snapshot. Would you not want to separate those two actions; have separate scripts? When one is implementing a backup scheme, one generally establishes a Full Backup and Incremental Backup schedule, say Full Backups on Mondays, and Incremental backups on Tuesdays through Sundays. Something akin to that.

For my use case separating the two actions merely doubles the work. I make only one snapshot, @yyyymmddhhmm, and immediately after I make it I incrementally send it to the external backup drive. Perfect example of KISS.

My internal MVMe is not removable so I backup to it at least weekly, or on special occasions like massive updates, using manual procedures. As long as I can boot to a root terminal I can recover @ by sending my latest @yyyymmddhh to /mnt/, replacing @

mv /mnt/@ /mnt/@old
btrfs send /backup/@yyyymmddhhmm /mnt/@

and not using the -r switch.

Still trying to understand and learn.

Me too, along with trying not to forget what I've learned in the past, which is turning out to be harder then one can imagine.

[Snowhog]

For my use case separating the two actions merely doubles the work. I make only one snapshot, @yyyymmddhhmm, and immediately after I make it I incrementally send it to the external backup drive.

That's what I don't grok. Every time you run your script it creates a new full snapshot. At least, that's what I understand reading it. Is that not what happens?

[GreyGeek]

That's what I don't grok. Every time you run your script it creates a new full snapshot. At least, that's what I understand reading it. Is that not what happens?

Yes. Using

btrfs su snapshot -r /mnt/@ /mnt/snapshots/@yyyymmddhhmm

creates a read-only snapshot in /mnt/snapshots directory. It has to be read only because read only snapshots are the only snapshots you can use the send command to send a snapshot to a subvolume not under <rootfs>.
IOW, you cannot use

mv /mnt/snapshots/@yyyymmddhhmm /backup

because /backup is not under /mnt, the <rootfs>. /backup is its own <rootfs>, which is not the same <rootfs> that @ is under.

While a snapshot is setting under <rootfs> it is modified by COW changes to the system made by the user. Eventually, if enough changes are made, the snapshot "fills up" with changes and takes up as much disk space as the system itself. So, when I make an ro snapshot and leave it on my system it isn't populated very much until the COW changes began to accumulate. Eventually, a single snapshot could grow to fill the entire disk if I didn't delete it or move it to another subvolume. This is why I can have my entire system plus 6, 8 or 10 snapshots or more on my 500Gb main drive but if I try to send a 6th snapshot to my 500Gb backup drive, which contains only 5 snapshots, I will get an "out of space" error. Unlike my snapshots on my system, the snapshots on my /backup drive are fully populated versions of my file system at the time I used the send command.

How? When I do a send, either full or incremental of a freshly made snapshot, the send command fully populates the snapshot with copies of the system files. In your case to copy all the files took about 16 minutes. Same for me as well. But, when you do an incremental snapshot, the btrfs receive command expects to see the parent snapshot setting in the location being sent to, i.e., /backup. Using its copy of the parent on /backup as a "bucket", it renames that copy with the name of the child and it begins pouring into that child files received from the send command until the send command is finished. That usually takes less than 15 seconds or so, depending on how much has changed since the parent was sent over. The child on /backup is then an exact copy of the child on /mnt/snapshots.
If you do not use the -r switch then the resulting snapshot CANNOT be sent using the btrfs send & receive command. But, you can add or remove files from an rw snapshot without hurting it using the cp command or dragging and dropping using Dolphin. This means that if you inadvertently delete a file or directory you can drag a copy of it out of one of your snapshots. I've used that feature several times because of shaky fingers typing.

If your @ snapshot gets corrupted and you can't boot into it then the first step to recover is to boot using a LiveUSB that has BTRFS installed on it. Once booted up you mount your system's disk ( /dev/.../whatever) to mount /mnt to some directory on the LiveUSB and then use the mv command to move @ out of the way.

mv /mnt/@ /mnt/@old

Then you use

btrfs su snapshot /mnt/snapshots/@yyyymmddhhmm /mnt/@

without using the -r switch. The snapshot was ro, but when snapshotting it again without the -r switch it is converted to a rw snapshot with the new name @. Then you reboot without the LiveUSB.

I do a lot of experimenting and have done this restore procedure so many times I've lost count. It's never failed once.

[Snowhog]

GreyGeek Question: Your script creates, but keeps, the full snapshot on your internal drive; the drive your working OS is installed on; compares the current full snapshot to the previous full snapshot, then creates and sends the incremental difference between the two to your backup drive. Do I have that right?

Right now, my script creates, then sends a full snapshot to my external backup drive. But I'm close to having it ready for incremental backups, which would then mirror the actions your script does. I'm still keeping my separate @ and @home, and I've worked out how to modify your incremental snapshot section so it can create separate incremental @ and @home snapshots from their parents residing in a single location: /mnt/snapshots.

I'm not ready yet to show my script; I have to test it first and make sure there aren't any 'gotchas' I missed.

[GreyGeek]

GreyGeek Question: Your script creates, but keeps, the full snapshot on your internal drive; the drive your working OS is installed on; compares the current full snapshot to the previous full snapshot, then creates and sends the incremental difference between the two to your backup drive. Do I have that right?

Right now, my script creates, then sends a full snapshot to my external backup drive. But I'm close to having it ready for incremental backups, which would then mirror the actions your script does. I'm still keeping my separate @ and @home, and I've worked out how to modify your incremental snapshot section so it can create separate incremental @ and @home snapshots from their parents residing in a single location: /mnt/snapshots.

I'm not ready yet to show my script; I have to test it first and make sure there aren't any 'gotchas' I missed.

Not quite. My main drive, sda3, has @, my total system, and in addition I keep 5 snapshots under <rootfs>. The snapshots in /mnt/snapshots/ are not "full" snapshots. They are ro COW snapshots that will fill up over time IF I let them remain in /mnt/snapshots.

Notice below that I have 5 ro snapshots under /mnt/snapshots and that my "usage" shows that 304 Gb of the SSD is still unallocated by BTRFS. My @ and the 5 snapshots takes only 134.07 Gb of space. IF those 5 snapshots were "full" copies of my system I would have run out of SDD space by the time I attempted to create the 4th or 5th snapshot.

vdir /mnt/snapshots/ 
total 0
drwxr-xr-x 1 root root 334 Feb 10 16:28 @202203051947
drwxr-xr-x 1 root root 334 Mar  6 23:01 @202203062302
drwxr-xr-x 1 root root 334 Mar  6 23:04 @202203071533
drwxr-xr-x 1 root root 334 Mar  6 23:04 @202203072053
drwxr-xr-x 1 root root 334 Mar  8 10:56 @202203082247

[B]sudo btrfs filesystem usage / [/B]
...
Overall:
   Device size:                 441.04GiB
   Device allocated:            137.04GiB
   Device unallocated:          [B]304.00GiB[/B]
   Device missing:                  0.00B
   Used:                        134.07GiB
   Free (estimated):            306.26GiB      (min: 306.26GiB)
   Data ratio:                       1.00
   Metadata ratio:                   1.00
   Global reserve:              214.38MiB      (used: 0.00B)

Data,single: Size:135.01GiB, Used:132.74GiB (98.32%)
  /dev/sda3     135.01GiB

Metadata,single: Size:2.00GiB, Used:1.33GiB (66.41%)
  /dev/sda3       2.00GiB

System,single: Size:32.00MiB, Used:48.00KiB (0.15%)
  /dev/sda3      32.00MiB

Unallocated:
  /dev/sda3     304.00GiB

However, when I start sending ro snapshots to other subvolumes, i.e., not under <rootfs>, my first snapshot cannot be incremental because the receive command has no child to compare it to. After I take 15-20 minutes to send the first snapshot subsequent snapshots can be sent incrementally in only a few seconds IF the destination drive has a full copy of the child present on it.

The BTRFS devs recommend 8 or less per subvolume. This is why I tell people that it is not wise to create 40 or 50 or more snapshots. People who use BTRFS and indiscriminately create snapshots will soon find that their system will slow to a crawl as their allocated space approaches their device size.

The line from the USAGE command that displays the data:

Data,single: Size:135.01GiB, Used:132.74GiB (98.32%)

shows 98.32% of the 135.01GiB being used. That high percentage rate is normal since BTRFS uses almost all of the space it allocates. In 5 years I've never seen that percentage below 97% and rarely that low.

[jlittle]

The BTRFS devs recommend 8 or less per subvolume. This is why I tell people that it is not wise to create 40 or 50 or more snapshots. People who use BTRFS and indiscriminately create snapshots will soon find that their system will slow to a crawl as their allocated space approaches their device size.

I suspect, and hope, that's old advice. On my main desktop for the system and my home subvolumes I have about 45 snapshots each, roughly half from my backup scheme, which does incrementals to several different drives (each needing a different previous), and half from snapper. I've got 25 or so other subvolumes, mostly other installs and for volatile data I don't want in the backups, such as browser caches and large downloads. Sometimes I've got a lot more subvolumes, when I've got an install that I start snapper in; I often do that with an LTS release.

I just make sure the fs doesn't fill up, and performance is good. It's still showing ~ 50 GiB unallocated out of 216 GiB, and 62 GiB free. I think keeping the volatile data out of the main subvolumes is working. Lots of snapshots doesn't in principle take up a lot of space, just metadata. For example, take a subvolume with data that changes or grows gradually, with a snapshot each month for a year; a snapshot each week over the same period mostly requires some more metadata, so won't take up much more space.

With mainstream distros now making btrfs the default file system, and OpenSUSE installing snapper by default, I expect there's many users that have subvolumes with a lot more than 8 snapshots.