Announcement

Collapse
No announcement yet.

HOWTO: One way to back up to an external drive using rsync

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    HOWTO: One way to back up to an external drive using rsync

    rsync is a powerful tool that makes backups fairly easy. I figured I'd share my backup strategy because it touches on a couple areas where a lot of people might like some assistance, like mapping a Windows share before you run the backup. I use a couple of scripts to automate the process so it's pretty much fire and forget and runs every night. Anyway...

    We're gonna assume you have an external hard drive and that all the necessary bits are in place to mount a Windows share. If you just want a quick and dirty archive tool I'd suggest taking a look at grsync - it's kind of a gooey interface to rsync but is a bit more limited than the command line version is.

    Anyway, here we go.

    1. Get a list of currently installed packages like this -

    dpkg --get-selections > ~/Documents/installed-packages.txt

    This will put a text file named 'installed-packages.txt' in the Documents folder in your home directory. We'll use that if we ever need to restore this stuff

    2. Let's make a logfile for all this and set permissions so at least one user other than root can write to the logfile because backing up your data as root makes for some strange file permissions -

    sudo touch /var/log/cron-output.log && sudo chmod 666 /var/log/cron-output.log

    3. Okay, time to look at the crontab. I don't put system-level actions in a user crontab (that's personal preference) so we'll be editing /etc/crontab for this one. You've gotta be root to do this so 'kdesudo kate /etc/crontab' will get you where you wanna be. On to the crontab:

    # date stamp local cron log

    0 0 * * * wizard date >> /var/log/cron-output.log

    This will append the time and date to the logfile so you know what happened when - this one runs at midnight.

    If you've got Windows shares you want to back up you'll need to mount them first. I created a mount point in my home directory so I could mount the My Documents folder on the spousal unit's Windows machine and back it up as well - looks like this:

    # backup stuff

    0 2 * * * root mount -t smbfs -o username=wizard,password=xxxxxxxx //192.168.1.103/documents$ /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1

    Okay, now for the fun part.

    15 2 * * * wizard sh /home/wizard/scripts/rsync_backup.sh >> /var/log/cron-output.log 2>&1

    and then

    0 4 * * * root umount /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1

    There - all done - now let's look at the rsync-backup script.

    I'm pretty far from a scripting guru - I leverage other people's knowledge when I don't have the knowlege myself and I'm not sure who I plagiarized this from but kudos to them

    Anyway, this script is just an executable text file named rsync_backup.sh that lives in ~/scripts

    rsync -rltEmqu --exclude-from=/home/wizard/.rsync/exclude /home/wizard /media/ARCHIVE/home
    rsync -rltEmqu /etc /media/ARCHIVE
    rsync -rltEmqu /boot/grub /media/ARCHIVE
    rsync -rltEmqu /var/cache/apt/archives /media/ARCHIVE/apt

    My external drive is mounted at /media/ARCHIVE. Note that we're backing up my home directory plus /etc, /boot/grub and /var/cache/apt/archives - this is where all your installation packages live. The reason I back up /etc is because there's usually a handful of configuration files I need to copy to a new installation so it's kinda nice to have.

    Now let's talk about rsync command line options a little so you can see what's happening here -

    -r, --recursive recurse into directories
    -l, --links copy symlinks as symlinks
    -t, --times preserve modification times
    -E, --executability preserve executability
    -m, --prune-empty-dirs prune empty directory chains from file-list
    -q, --quiet suppress non-error messages
    -u, --update skip files that are newer on the receiver

    There are other command line options you can use - the rsync man page is a wealth of information.

    The script above mentions an exclude file - this is for stuff I don't want backed up. Mine looks like this - all it is is a text file in ~/.rsync named 'exclude'

    .rnd
    .VirtualBox/
    lock
    Desktop/
    download/
    temp/

    The entries with a trailing slash are directory names as opposed to filenames.

    And that's pretty much it. If I were gonna restore things here's how I'd do it - assuming a clean, empty hard drive:

    1. Install the new OS.

    2. Restore home directories and package cache from the external hard drive. Don't restore /etc just yet - we're only gonna use a little bit of it and we want all our packages installed before we replace configuration files.

    3. If you had medibuntu or other repos in your old /etc/apt/sources.list now's the time to get them added and make sure they work.

    4. Update apt like this -

    sudo apt-get update

    5. Now this is where stuff gets cool. Reselect all the packages you had previously installed like this -

    dpkg --set-selections < ~/Documents/installed-packages.txt

    then reinstall all the packages you had like this -

    sudo apt-get dselect-upgrade

    apt will pull as many packages as it can from local cache and get the rest from repos.

    6. Last, restore only the files from /etc that you know need to be modified for your own use.

    Note that if you're changing architecture from ia32 to amd64 or vice versa there's no reason to restore your apt cache as almost none of the packages will work but when I installed Karmic I went from a bare hard drive to a fully functional desktop with all my applications installed in a bit less than an hour.

    Have fun -
    we see things not as they are, but as we are.
    -- anais nin

    #2
    Re: HOWTO: One way to back up to an external drive using rsync

    Lots of interest/questions at the forum about backups, and there's been talk about rsync vs dd vs Partimage vs etcetera. Very nice job done here wizard10000 -- this is going to be a helpful contribution! (For now, I only scanned it to see how you did it, where it goes, etc.; I have not used rsync but see it mentioned everywhere.)

    The only thing I can think of is that besides not understanding how to use rsync, most users will probably not know much about scripts, running scripts, etc., so that's about the only thing that you might think about. (That is, I know some folks just want to run rsync without automating it.) The site owner (OS) has set a high limit on allowable character counts for these how-to's, so if you ever want to add/modify, it's all your show.

    Good job! Very nice.
    An intellectual says a simple thing in a hard way. An artist says a hard thing in a simple way. Charles Bukowski

    Comment


      #3
      Re: HOWTO: One way to back up to an external drive using rsync

      Thank you -

      Looking back it is a little geekish so I cleaned it up just a little.
      we see things not as they are, but as we are.
      -- anais nin

      Comment

      Working...
      X