Announcement

Collapse
No announcement yet.

My simple btrfs backup script.

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 2
  • Filter
  • Time
  • Show
Clear All
new posts
Previous 1 2 template Next
    #1

    My simple btrfs backup script.

    I use snapper for my regular snapshots, it works very well but it doesn't do full backups to another location. So, I made a script so I could plug in my external HD, run it as sudo and it does all the commands for sending a backup. It does an automatic time stamp in the snapshot name as well, for easy admin. The main weakness is me remembering to run it often enough but there's no point in running some sort of automation as I keep the external HD unplugged when not in use.

    So to explain what it's doing, it mounts the drive in /mnt, takes snapshots with today's date, sends them to the external hard drive, deletes the snapshot on the hard drive and then unmounts from /mnt. As I use snapper, there's no point for me to retain the snapshots on my hard drive.

    Code:
     #!/bin/bash

mount /dev/disk/by-uuid/YOUR UUID HERE /mnt
today=$(date +"%d_%m_%Y")
btrfs su snapshot -r /mnt/@ /mnt/snapshots/@_${today}
btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home_${today}
btrfs send /mnt/snapshots/@_${today} | btrfs receive /media/YOUR BACKUP LOCATION HERE
btrfs send /mnt/snapshots/@home_${today} | btrfs receive /media/YOUR BACKUP LOCATION HERE
btrfs subvolume delete /mnt/snapshots/@_${today}
btrfs subvolume delete /mnt/snapshots/@home_${today}
umount /mnt
    Tags: None
    • Top
    • Bottom
    #2
    I use a script I wrote but my approach is to use a differential backup using "-p"
    Code:
    ...
echo "Mounting drives"
eval "mount /dev/disk/by-uuid/ce2b5741-c01e-4b3d-b6ba-401ad7f7fcdf /mnt"
eval "mount /dev/disk/by-uuid/e84e2cdf-d635-41c5-9f6f-1d0235322f48 /backup"

NOW=$(date +%Y%m%d%H%M)
echo "Making today's snapshot"
MKSNAP='btrfs su snapshot -r /mnt/@ /mnt/snapshots/@'$NOW
eval $MKSNAP
eval 'sync'
eval 'sync'

echo "Finding previous snapshot as parent "
PREVSNAP=""
list=($(ls /mnt/snapshots/))
PREVSNAP=${list[-2]}
NOW=${list[-1]}
MKINC='btrfs send -p /mnt/snapshots/'$PREVSNAP
MKINC=$MKINC' /mnt/snapshots/'$NOW
MKINC=$MKINC' | btrfs receive /backup'
echo $MKINC
eval $MKINC
...
    Using the "-p" switch SIGNIFICANTLY reduces the amount of time necessary to create an archive on a remote drive.
    In my case, with about 130GB, without the "-p" it would take about 15 minutes. With the "-p" switch about 1 minute.
    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
    – John F. Kennedy, February 26, 1962.
    • Top
    • Bottom

    Comment

      #3
      Late to this thread, but I do not understand what the entries in that command are, like maybe it's over-complicated or I've missed something?

      The syntax is:

      btrfs send -p </PATH/SNAPSHOT (previous snapshot)> </PATH/SNAPSHOT (new snapshot)> | btrfs receive </PATH>

      at a minimum you have three entries in the SEND portion and two entries in the RECEIVE portion which are not allowed OR you're trying to escape the space, which means you've used a space in your folder name. A questionable action in my opinion. If my assumption is correct and the escape is working to use the space, your path to the snapshots is "/media/user/SSD Backup/snapshots/" and your destination backup path is also "/media/user/SSD Backup/snapshots/" which would mean you're sending the subvolume to itself which is also not allowed or even useful.

      This might help a bit: https://btrfs.wiki.kernel.org/index....emental_Backup

      Please Read Me
      • Top
      • Bottom

      Comment

        #4
        What I do at the moment is make snapshots on my system, send them to the external drive and then delete the ones on my system. From what I am understanding now, for an incremental backup, I need to keep the previous snapshot on my system as well. This would be kind of annoying for me because of the frequency of how often I am doing it.
        • Top
        • Bottom

        Comment

          #5
          Well yes, the last (previous) snapshot is required so the system has a reference to know what has changed. I can't imagine it working any other way. It's the trade off to potentially vastly reduce time required to do a backup. You only need to keep one snapshot though, You can delete the parent immediately after the incremental send, just not the current one. Call it leap-frogging. If you do a full backup every time, it will always take considerably longer.

          Frankly, unless you are running a system at a 90%+ drive capacity and/or doing your incremental backups months apart, I doubt the drive space used by keeping a fairly fresh snapshot is using enough drive space that you will even notice. I personally make a daily snapshot and keep a full week's worth at all times (for rollback usage) and then send an incremental backup on every Sunday to my server. The amount of space on my heavily used system is inconsequential: 7, 24 hour snapshots stored at all times, incremental backup is 7 days worth of changes.

          The alternative is to not do incremental backups. The cost is potential drive space of an unknown amount vs. time to make the backup. You get one benefit or the other, but not both.

          Remember a snapshot only increases drive usage when changes are made to the source subvolume. Similarly, the time to send an incremental backup increases as the snapshot size increases. Both are a function of the age of the snapshot. Simply doing it more often reduces the space and time required. Monthly will require a 30 day long volume of changes where daily would only be that day's changes. Your sweet spot maybe somewhere in between.

          To explain the leap-frog (if needed):

          Initial backup:
          1) subvolume = @subvol
          2) take snapshot as @subvol1
          3) send @subvol1 to backup

          Some time passes...

          Subsequent backups:
          1) take snapshot of @subvol as @subvol2
          2) send increment using @subvol1 as parent, @subvol2 as backup
          3) delete @subvol1 from snapshot and backup folders

          Now only @subvol and @subvol2 remain. Repeat as often as desired.
          Last edited by oshunluvr; Jul 07, 2021, 03:39 PM.

          Please Read Me
          • Top
          • Bottom

          Comment

            #6
            Well, I was thinking that maybe rsync is a better idea for what I want and then I saw that kup has an incremental rsync setting so I am going to use that instead. I know that removes the ability to restore everything in one go but I am not too bothered about that.
            • Top
            • Bottom

            Comment

              #7
              I've never used rsync so I can't comment on it aside from what I read on the Internet about it vs BTRFS.
              Regardless of if things turn out the way you hope or not, please post your results to the BTRFS forum so we can all benefit.
              "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
              – John F. Kennedy, February 26, 1962.
              • Top
              • Bottom

              Comment

                #8
                Originally posted by Bings View Post
                Well, I was thinking that maybe rsync is a better idea for what I want and then I saw that kup has an incremental rsync setting so I am going to use that instead. I know that removes the ability to restore everything in one go but I am not too bothered about that.
                Well again, just my opinion, but using rsync totally obviates one of the major benefits of BTRFS - native backup and restore. I think you have decided keeping one snapshot is a huge deficit before actually attempting to put it into play. I doubt even a month's worth of changes to your install subvolume will add up to much.

                Let's assume the drive space isn't a factor and you're scripting the backup functionality anyway, "btrfs send -p..." is a way easier to set up than rsync with it's hundreds of options like:

                rsync -av --delete "${SOURCE_DIR}/" --link-dest "${LATEST_LINK}" --exclude=".cache" "${BACKUP_PATH}"

                but your world so, do you.

                Please Read Me
                • Top
                • Bottom

                Comment

                  #9
                  I bought an internal HDD specifically for backups so I am going to set it up for incremental backups. I changed my script to this.

                  Code:
                   #!/bin/bash

mount /dev/disk/by-uuid/34424d7b-13a5-419c-8fb2-9c3da4cb3e1d /mnt
mount /dev/disk/by-uuid/db599790-a9d5-4804-9ad4-a268a7e60834 /backup

#Making time stamped snapshots
today=$(date +"%Y_%m_%d")
btrfs su snapshot -r /mnt/@ /mnt/snapshots/@_${today}
btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home_${today}

#Sending incremental backup
btrfs send -p /mnt/snapshots/@_backup /mnt/snapshots/@_${today} | btrfs receive /backup
btrfs send -p /mnt/snapshots/@home_backup /mnt/snapshots/@home_${today} | btrfs receive /backup

#Deleting reference snapshots
btrfs subvolume delete /mnt/snapshots/@_backup
btrfs subvolume delete /mnt/snapshots/@home_backup
btrfs subvolume delete /backup/@_backup
btrfs subvolume delete /backup/@home_backup

#Creating new reference snapshots for next backup
btrfs su snapshot -r /mnt/@_${today} /mnt/snapshots/@_backup
btrfs su snapshot -r /mnt/@home_${today} /mnt/snapshots/@home_backup
btrfs su snapshot -r /backup/@_${today} /backup/@_backup
btrfs su snapshot -r /backup/@home_${today} /backup/@home_backup

btrfs subvolume delete /mnt/snapshots/@_${today}
btrfs subvolume delete /mnt/snapshots/@home_${today}

umount /mnt
umount /backup
                  I've run this once and it seems to have worked, I got this output..

                  Code:
                  Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@_2021_11_26'
Create a readonly snapshot of '/mnt/@home' in '/mnt/snapshots/@home_2021_11_26'
At subvol /mnt/snapshots/@_2021_11_26
At snapshot @_2021_11_26
At subvol /mnt/snapshots/@home_2021_11_26
At snapshot @home_2021_11_26
Delete subvolume (no-commit): '/mnt/snapshots/@_backup'
Delete subvolume (no-commit): '/mnt/snapshots/@home_backup'
Delete subvolume (no-commit): '/backup/@_backup'
Delete subvolume (no-commit): '/backup/@home_backup'
ERROR: Could not statfs: No such file or directory
ERROR: Could not statfs: No such file or directory
Create a readonly snapshot of '/backup/@_2021_11_26' in '/backup/@_backup'
Create a readonly snapshot of '/backup/@home_2021_11_26' in '/backup/@home_backup'
Delete subvolume (no-commit): '/mnt/snapshots/@_2021_11_26'
Delete subvolume (no-commit): '/mnt/snapshots/@home_2021_11_26'
                  I don't understand the error message, everything seems fine. I'll try it again later.
                  • Top
                  • Bottom

                  Comment

                    #10
                    It appears to me that you are using the root_fs of /mnt to do snapshots on the root_fs of /backup. Two different root_fs's.
                    Not allowed.

                    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                    – John F. Kennedy, February 26, 1962.
                    • Top
                    • Bottom

                    Comment

                      #11
                      I've seen this happen when someone manually deletes a subvol (rf-rm) instead of using btrfs su de. Maybe you're trying to delete a snapshot too soon after creation? Remember btrfs transactions are not instantaneous. Try adding the commit flag to the deletes and see if the error goes away. I assume it will slow the full execution of the script, but not my much.

                      Please Read Me
                      • Top
                      • Bottom

                      Comment

                        #12
                        To what GreyGeek says, is /mnt the root fs of @ or a separate root fs from / ?

                        Please Read Me
                        • Top
                        • Bottom

                        Comment

                          #13
                          I mount the root file system at /mnt. Step by step including the preparation for the script

                          1: Mount the root file system at mnt and the second drive at "/backup"
                          2: Snapshot at /mnt/snapshots called "backup"
                          3: Full of send of "backup" to the second drive, so that I have the two reference snapshots.
                          4: "dated" snapshot in /mnt/snapshots.
                          5: Incremental send of dated snapshot to second drive.
                          6: Delete "backup" snapshots on each drive.
                          7: Make a snapshot of "dated" subvolume in /mnt/snapshots and called that "backup"
                          8: Do the same thing with the one on the second drive.
                          9: Delete the dated snapshot in the /mnt/snapshot but keep the one on the second drive.

                          Is what I am doing in step 7 and/or 8 wrong? (edit: ignore question, see post below)

                          Upon looking again, I hadn't actually ended up with with a "backup" subvolume in the /mnt/snapshots folders. So running it a second time was a collection of errors.
                          Last edited by Bings; Nov 27, 2021, 02:01 PM.
                          • Top
                          • Bottom

                          Comment

                            #14
                            The commit tag worked and then I could see that I really was doing something wrong in step 7 and 8 because it loses the reference somehow and I can't use them as parents. So I renamed them instead of using snapshot and then was able to snapshot the new reference for keeping.

                            Code:
                             #!/bin/bash

mount /dev/disk/by-uuid/34424d7b-13a5-419c-8fb2-9c3da4cb3e1d /mnt
mount /dev/disk/by-uuid/db599790-a9d5-4804-9ad4-a268a7e60834 /backup

#Making time stamped snapshots
today=$(date +"%Y_%m_%d")
btrfs su snapshot -r /mnt/@ /mnt/snapshots/@_${today}
btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home_${today}

#Sending incremental backup
btrfs send -p /mnt/snapshots/@_backup /mnt/snapshots/@_${today} | btrfs receive /backup
btrfs send -p /mnt/snapshots/@home_backup /mnt/snapshots/@home_${today} | btrfs receive /backup

#Deleting reference snapshots
btrfs subvolume delete -c /mnt/snapshots/@_backup
btrfs subvolume delete -c /mnt/snapshots/@home_backup
btrfs subvolume delete -c /backup/@_backup
btrfs subvolume delete -c /backup/@home_backup

#Renaming new snapshots to be reference snapshots for next time
mv /mnt/snapshots/@_${today} /mnt/snapshots/@_backup
mv /backup/@_${today} /backup/@_backup
mv /mnt/snapshots/@home_${today} /mnt/snapshots/@home_backup
mv /backup/@home_${today} /backup/@home_backup

#Snapshot for archive
btrfs su snapshot -r /backup/@_backup /backup/@_${today}
btrfs su snapshot -r /backup/@home_backup /backup/@home_${today}


umount /mnt
umount /backup
                            • Top
                            • Bottom

                            Comment

                              #15
                              I don't know what the error messages are about here, would this cause me problems if I restored from the backup?

                              Code:
                              [FONT=monospace][COLOR=#000000]Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@_2021_12_05' [/COLOR]
Create a readonly snapshot of '/mnt/@home' in '/mnt/snapshots/@home_2021_12_05'
At subvol /mnt/snapshots/@_2021_12_05
At snapshot @_2021_12_05
WARNING: capabilities set multiple times per file: /backup//@_2021_12_05/usr/bin/ping
WARNING: capabilities set multiple times per file: /backup//@_2021_12_05/usr/lib/x86_64-linux-gnu/libexec/kf5/start_kdeinit
At subvol /mnt/snapshots/@home_2021_12_05
At snapshot @home_2021_12_05
Delete subvolume (commit): '/mnt/snapshots/@_backup'
Delete subvolume (commit): '/mnt/snapshots/@home_backup'
Delete subvolume (commit): '/backup/@_backup'
Delete subvolume (commit): '/backup/@home_backup'
Create a readonly snapshot of '/backup/@_backup' in '/backup/@_2021_12_05'
Create a readonly snapshot of '/backup/@home_backup' in '/backup/@home_2021_12_05'[/FONT]
                              • Top
                              • Bottom

                              Comment

                              Previous 1 2 template Next
                              Working...
                              X