Setting up an external USB hard drive

[snowdrift]

With windows I bought a USB hard disk caddy and new hard disk for backing up the whole system (using Norton Ghost). I have 2 hard drives in my computer ;dual booting to either windows on one , kubuntu on the other. I want to do the same for my Kubuntu hard drive, however I anticipate problems.
Kubuntu will probably recognise the USB connection, but how do I format the drive? And should it be formatted in ext3 or fat32?
I then want to back up the system following the guidelines in http://www.ubuntuforums.org/showthread.php?t=35087

Any advice before I purchase the hardware would be appreciated

Richard

[askrieger]

Re: Setting up an external USB hard drive

I have no intention or desire to read an 11 page thread about a topic that I think I understand. Of course, I could be wrong. But, on the off chance that I know what I'm takling about, I offer the following advice from my own experience.

For more than 10 years, I backed up Linux with gzipped tarballs as discussed in the first post of that thread. More recently, I use a program called KDAR that I'll discuss below. In my opinion, the only file trees that need backup are /home, and /etc. /home/ includes all your personal documents, music, and pictures. That stuff is irreplacable and changes frequently. /etc/ has all of the configuration files that make your computer work the way you want it to (except for the ones in dotted directories under /home/).

For the last five or six years, I've been storing these backups on (first) firewire and (now) usb2 external hardrives. I use these in preference to cheap hard drives installed in enclosures, because the cost saving isn't worrth the extra work. My present backup device is a Western Digital 120 GB External USB Hard Drive which is formatted with 3 vfat partitions, one for my main linux box, one for my backup and test linux box, and one for my wife's Win XP box. The partitions are formatted with VFAT because Linux recognizes windoze partitions but windoze won't recognize linux. This means I'm safe regardless of which two of three computers fails at once. The WD has replaced a Fantom drive, which still works, but is too small to store all my music and pictures and still have room for backups from three computers. The Fantom replaced a firewire drive, because all computers now come with usb2 and few come with firewire. I transfer data over our household cat5 ethernet LAN, and then store them on the WD drive which is attached to my main linux box.

I store these backups in the form of "disk backups" generated by a program called Kdar, which is a KDE GUI frontend for a CLI program called "dar" (note the intentional resemblance to "tar"). The main reason for the switch to dar was so that I could use "differential backups". This is a form of backup that backups only files that have changed since the last full backup. This means that I only need to store one full backup and the latest differential backup from the two linux boxes. I still use Winzip on my wife's XP box. Because backups are so fast, I backup my two computers every week or two with a full backup every two months or so. When my livelihood depended on my computer, I backed up every day.

To summarize, (1) use a separate hard drive that can be plugged into a different computer, if neccesary. (2) Use Kdar to make compressed differential backups so that backups are fast and convenient. (3) Use VFAT so that you can read your files from a random windows computer if you have to.

[snowdrift]

Re: Setting up an external USB hard drive

Thanks for your help as always! Have downloaded Kdar and it seems perfect ( like so many Linux programs). Will get the external drive in the next few days.
Tell me; what is the simplest way to format / reformat the external drive to vfat ..... making sure I don't accidentally format my main hard drives!!?

Thanks Richard

[askrieger]

Re: Setting up an external USB hard drive

It may already come that way, because the manufacturers want to sell to both windows and mac customers. If not, I'd use qtparted under linux (I'm not sure it does vfat, but it's my favorite linux partitioner) or partition magic under windoze, which definitely does fat32. Both have graphical front ends so you can be sure which drive you're partitioning without fear of noticing a typo 5 minutes in to a ten minute reformat.

[snowdrift]

Re: Setting up an external USB hard drive

Got the external drive - already formatted in fat32.
set up kdar to do full backup of /home and successfully completed the "dry run" - 16 minutes. However when I write for real to the external drive it stops and closes kdar after writing 4,294,967,295 bytes to the external drive - approx 4 minutes.

I thought I would try to back up the smaller /etc folder as recommended. The dry run came up with lots of errors - access denied. Does that mean kdar needs to be run as super user? - but no option given to do that.

Any ideas? Richard

[aysiu]

Re: Setting up an external USB hard drive

You should have access to only /home/snowdrift

Even /home/anotheruser should be off limits to you.

If you plan on backing up your entire system, you need to run KDar as sudo.

[kubicle]

Re: Setting up an external USB hard drive

You should have access to only /home/snowdrift

Even /home/anotheruser should be off limits to you.

If you plan on backing up your entire system, you need to run KDar as sudo.

Hmm...isn't most of the system readable for a regular user by default (if you haven't changed the permissions)?

I'd assume KDar doesn't need write access to the files it backs up...of course some system files are not readable by anyone but root (and there are those files in /etc which would explain the errors).

All that said...if you want to make sure every file is backed up, using sudo is indeed a necessity.

[askrieger]

Re: Setting up an external USB hard drive

Because Kdar has a graphical user interface, the correct way to start it as the superuser is with "kdesu kdar"

[snowdrift]

Re: Setting up an external USB hard drive

Ran kdar as su and it still stopped and closed after storing 4,294,967,295 bytes. Then tried storing a subset of /home and again failed after storing 4,294,967,295 bytes.
I conclude it is not what I am saving but that kdar / kubuntu won't let me create /write a file greater than this number of bytes. Putting that number in Google seems to show that is a significant number but can't find any clue as to what the problem is.

Richard

[snowdrift]

Re: Setting up an external USB hard drive

Just found that the maximum file size that can be stored in fat32 is the aforementioned number. ( ie 2 to the 32 -1).
Would I be right to set the slice size to say 4GB in Kdar and save my backup as a number of slices?

Richard

[askrieger]

Re: Setting up an external USB hard drive

Yes you can save your backup as a set of slices, if you wish. I'm surprised at the size of your backup file, though. Are you using bzip2 compression?

[snowdrift]

Re: Setting up an external USB hard drive

There's a pile of photos and mp3's which take up the space. Could archive these separately onto DVD's if I wished, just more convenient to dump it all onto a big hard drive.
Because of the photos/mp3, I didn't bother using compression at this first attempt. I'll refine my backup strategy now that it is working.

Thanks for all your help, Richard

[askrieger]

Re: Setting up an external USB hard drive

Compression really doesn't work very well with either music files or photos (they're already compressed in jpeg or mp3 format. I just copy my music and photo directories in unarchived form over to the USB drive. Then I exclude them from the compressed KDAR files.