rsync is a powerful tool that makes backups fairly easy. I figured I'd share my backup strategy because it touches on a couple areas where a lot of people might like some assistance, like mapping a Windows share before you run the backup. I use a couple of scripts to automate the process so it's pretty much fire and forget and runs every night. Anyway...
We're gonna assume you have an external hard drive and that all the necessary bits are in place to mount a Windows share. If you just want a quick and dirty archive tool I'd suggest taking a look at grsync - it's kind of a gooey interface to rsync but is a bit more limited than the command line version is.
Anyway, here we go.
1. Get a list of currently installed packages like this -
dpkg --get-selections > ~/Documents/installed-packages.txt
This will put a text file named 'installed-packages.txt' in the Documents folder in your home directory. We'll use that if we ever need to restore this stuff
2. Let's make a logfile for all this and set permissions so at least one user other than root can write to the logfile because backing up your data as root makes for some strange file permissions -
sudo touch /var/log/cron-output.log && sudo chmod 666 /var/log/cron-output.log
3. Okay, time to look at the crontab. I don't put system-level actions in a user crontab (that's personal preference) so we'll be editing /etc/crontab for this one. You've gotta be root to do this so 'kdesudo kate /etc/crontab' will get you where you wanna be. On to the crontab:
# date stamp local cron log
0 0 * * * wizard date >> /var/log/cron-output.log
This will append the time and date to the logfile so you know what happened when - this one runs at midnight.
If you've got Windows shares you want to back up you'll need to mount them first. I created a mount point in my home directory so I could mount the My Documents folder on the spousal unit's Windows machine and back it up as well - looks like this:
# backup stuff
0 2 * * * root mount -t smbfs -o username=wizard,password=xxxxxxxx //192.168.1.103/documents$ /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1
Okay, now for the fun part.
15 2 * * * wizard sh /home/wizard/scripts/rsync_backup.sh >> /var/log/cron-output.log 2>&1
and then
0 4 * * * root umount /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1
There - all done - now let's look at the rsync-backup script.
I'm pretty far from a scripting guru - I leverage other people's knowledge when I don't have the knowlege myself and I'm not sure who I plagiarized this from but kudos to them
Anyway, this script is just an executable text file named rsync_backup.sh that lives in ~/scripts
rsync -rltEmqu --exclude-from=/home/wizard/.rsync/exclude /home/wizard /media/ARCHIVE/home
rsync -rltEmqu /etc /media/ARCHIVE
rsync -rltEmqu /boot/grub /media/ARCHIVE
rsync -rltEmqu /var/cache/apt/archives /media/ARCHIVE/apt
My external drive is mounted at /media/ARCHIVE. Note that we're backing up my home directory plus /etc, /boot/grub and /var/cache/apt/archives - this is where all your installation packages live. The reason I back up /etc is because there's usually a handful of configuration files I need to copy to a new installation so it's kinda nice to have.
Now let's talk about rsync command line options a little so you can see what's happening here -
-r, --recursive recurse into directories
-l, --links copy symlinks as symlinks
-t, --times preserve modification times
-E, --executability preserve executability
-m, --prune-empty-dirs prune empty directory chains from file-list
-q, --quiet suppress non-error messages
-u, --update skip files that are newer on the receiver
There are other command line options you can use - the rsync man page is a wealth of information.
The script above mentions an exclude file - this is for stuff I don't want backed up. Mine looks like this - all it is is a text file in ~/.rsync named 'exclude'
.rnd
.VirtualBox/
lock
Desktop/
download/
temp/
The entries with a trailing slash are directory names as opposed to filenames.
And that's pretty much it. If I were gonna restore things here's how I'd do it - assuming a clean, empty hard drive:
1. Install the new OS.
2. Restore home directories and package cache from the external hard drive. Don't restore /etc just yet - we're only gonna use a little bit of it and we want all our packages installed before we replace configuration files.
3. If you had medibuntu or other repos in your old /etc/apt/sources.list now's the time to get them added and make sure they work.
4. Update apt like this -
sudo apt-get update
5. Now this is where stuff gets cool. Reselect all the packages you had previously installed like this -
dpkg --set-selections < ~/Documents/installed-packages.txt
then reinstall all the packages you had like this -
sudo apt-get dselect-upgrade
apt will pull as many packages as it can from local cache and get the rest from repos.
6. Last, restore only the files from /etc that you know need to be modified for your own use.
Note that if you're changing architecture from ia32 to amd64 or vice versa there's no reason to restore your apt cache as almost none of the packages will work but when I installed Karmic I went from a bare hard drive to a fully functional desktop with all my applications installed in a bit less than an hour.
Have fun -
We're gonna assume you have an external hard drive and that all the necessary bits are in place to mount a Windows share. If you just want a quick and dirty archive tool I'd suggest taking a look at grsync - it's kind of a gooey interface to rsync but is a bit more limited than the command line version is.
Anyway, here we go.
1. Get a list of currently installed packages like this -
dpkg --get-selections > ~/Documents/installed-packages.txt
This will put a text file named 'installed-packages.txt' in the Documents folder in your home directory. We'll use that if we ever need to restore this stuff
2. Let's make a logfile for all this and set permissions so at least one user other than root can write to the logfile because backing up your data as root makes for some strange file permissions -
sudo touch /var/log/cron-output.log && sudo chmod 666 /var/log/cron-output.log
3. Okay, time to look at the crontab. I don't put system-level actions in a user crontab (that's personal preference) so we'll be editing /etc/crontab for this one. You've gotta be root to do this so 'kdesudo kate /etc/crontab' will get you where you wanna be. On to the crontab:
# date stamp local cron log
0 0 * * * wizard date >> /var/log/cron-output.log
This will append the time and date to the logfile so you know what happened when - this one runs at midnight.
If you've got Windows shares you want to back up you'll need to mount them first. I created a mount point in my home directory so I could mount the My Documents folder on the spousal unit's Windows machine and back it up as well - looks like this:
# backup stuff
0 2 * * * root mount -t smbfs -o username=wizard,password=xxxxxxxx //192.168.1.103/documents$ /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1
Okay, now for the fun part.
15 2 * * * wizard sh /home/wizard/scripts/rsync_backup.sh >> /var/log/cron-output.log 2>&1
and then
0 4 * * * root umount /home/wizard/mrs-wiz-pc >> /var/log/cron-output.log 2>&1
There - all done - now let's look at the rsync-backup script.
I'm pretty far from a scripting guru - I leverage other people's knowledge when I don't have the knowlege myself and I'm not sure who I plagiarized this from but kudos to them
Anyway, this script is just an executable text file named rsync_backup.sh that lives in ~/scripts
rsync -rltEmqu --exclude-from=/home/wizard/.rsync/exclude /home/wizard /media/ARCHIVE/home
rsync -rltEmqu /etc /media/ARCHIVE
rsync -rltEmqu /boot/grub /media/ARCHIVE
rsync -rltEmqu /var/cache/apt/archives /media/ARCHIVE/apt
My external drive is mounted at /media/ARCHIVE. Note that we're backing up my home directory plus /etc, /boot/grub and /var/cache/apt/archives - this is where all your installation packages live. The reason I back up /etc is because there's usually a handful of configuration files I need to copy to a new installation so it's kinda nice to have.
Now let's talk about rsync command line options a little so you can see what's happening here -
-r, --recursive recurse into directories
-l, --links copy symlinks as symlinks
-t, --times preserve modification times
-E, --executability preserve executability
-m, --prune-empty-dirs prune empty directory chains from file-list
-q, --quiet suppress non-error messages
-u, --update skip files that are newer on the receiver
There are other command line options you can use - the rsync man page is a wealth of information.
The script above mentions an exclude file - this is for stuff I don't want backed up. Mine looks like this - all it is is a text file in ~/.rsync named 'exclude'
.rnd
.VirtualBox/
lock
Desktop/
download/
temp/
The entries with a trailing slash are directory names as opposed to filenames.
And that's pretty much it. If I were gonna restore things here's how I'd do it - assuming a clean, empty hard drive:
1. Install the new OS.
2. Restore home directories and package cache from the external hard drive. Don't restore /etc just yet - we're only gonna use a little bit of it and we want all our packages installed before we replace configuration files.
3. If you had medibuntu or other repos in your old /etc/apt/sources.list now's the time to get them added and make sure they work.
4. Update apt like this -
sudo apt-get update
5. Now this is where stuff gets cool. Reselect all the packages you had previously installed like this -
dpkg --set-selections < ~/Documents/installed-packages.txt
then reinstall all the packages you had like this -
sudo apt-get dselect-upgrade
apt will pull as many packages as it can from local cache and get the rest from repos.
6. Last, restore only the files from /etc that you know need to be modified for your own use.
Note that if you're changing architecture from ia32 to amd64 or vice versa there's no reason to restore your apt cache as almost none of the packages will work but when I installed Karmic I went from a bare hard drive to a fully functional desktop with all my applications installed in a bit less than an hour.
Have fun -
Comment