Need reliable setup of manual data backup Kubuntu 14.04

[frankus333]

G'day folks

This is a re-post of a question that attracted no answers in previous post https://www.kubuntuforums.net/showth...s-external-HDD. Again, please let me know if I am doing something wrong in these posts.

I am wanting to know how to set up a backup program to avoid the hassles outlined in that previous post, which I assume may have something to do with permissions when using a limited user account instead of an administrator account. I notice, for example, that I have to use my administrator password to mount the data partition in Dolphin when in the everyday user account. However, this does not seem to be required for external HDDs?

System Information as requested of posters to this forum:

Dell Dimension 4600i desktop dual booting WinXP and Kubuntu 14.04, KDE 4.13.0, 320GB internal HDD, comprising WinXP 30GB, Data 239GB (both NTFS), 22GB ext4 Kubuntu 14.04, 3GB Linux swap. CPU Intel 2.6GHz P4 with hyperthreading (makes it look dual core in System Monitors, but I doubt it), 32 bit, 1.5GB RAM, 2 x optical drives.

Problem Situation:

I was trying to use Backup (installed from Ubuntu Software Centre) to make a backup of my NTFS data partition to a 2TB Seagate USB 2 external HDD (also NTFS). The backup failed early in the process, with a message I didn't save, but which mentioned something about a .tmp file that did not exist. The external HDD was then not visible in Dolphin, although 2 other external drives still were, along with the internal partitions. I rebooted to get the drive to show again, which it did in the side list, but would not mount. Further details in the previous post. As noted there, the drive has come good, but I don't know how, or why it failed initially, and I want to avoid whatever setup mistake I made.

So, I think my question is whether there are any considerations (and what they might be) re working in an everyday user level account instead of the administrator account, when using a backup program such as Ubuntu Backup for manual backups of my data partition. Some of the Home folders of the limited account would be included in the backup, to ensure any recent downloads or files saved prior to sending them to the data partition are not missed.

I would also be interested in recommendations re programs other than Backup for this purpose. I see that Lucky Backup and Back in Time seem to rate well, but are not supported by Canonical.

[frankus333]

Thanks Teunis

The data drive which WinXP wanted to check was the internal data partition (the source for backing up), which I had previously re-sized. The folks at GParted informed me that when it re-sizes a partition, it automatically flags it for checking by chkdsk, which explains why WinXp wanted to do that.

The external HDD (the intended destination for backups) which Dolphin would not initially mount did not show any errors or requests for checks in WinXP, even though the error message from Dolphin indicated that the two MFTs did not match (which I would have thought would trigger a chkdsk check). The only chkdsk checks I allowed to run on either drive while booted into WinXP were those via the Tools menu, without the fix options ticked/checked. I don't know if that process allows any correction of the two MFT differences, but maybe it does? If my original post of the sequence of steps reflects accurate memory, the external HDD was not "fixed" by this initial WinXP process, and needed a number of other connections and disconnections to other Kubuntu system to come good, in some inexplicable way.

Re Kubuntu partition space; there would have been about 9GB free space at the time. My RAM is 1.5GB, with a 3GB swap partition. The data partition I am trying to back up is about 239GB, with about 140GB of data used (and needing to be backed up). Would these specs be a limitation on backing up?

I had considered using Dolphin, and copy and paste, with the overwrite option, to regularly update the manually copied (by Dolphin) backup, but I assume this would take a lot more time than a properly functioning incremental backup program. I will have a look at Krusader, thanks.

Re the accounts; my intention is to operate from the user (non-admin) account as much as possible. That's where any downloaded or created files are most likely to be, and needing backing up. So I am still curious to know whether using that account might have something to do with the Backup program failure. I'm guessing that whatever the .tmp file was meant to be, its " non-existence" is related to not being able to be seen due to permissions issues. I would like to play around with a backup program a bit more, but am reluctant until I understand more about what might get broken and how.

Cheers

[kiraitachi]

Hi I use this for backups of partitions or disks, works for windows and Linux, It saves me lots of time when im debugging or clonning to other computers the same image.

Here is the tutorial:

http://xmodulo.com/2013/08/how-to-ba...lonezilla.html

The tool is called Clonezilla. Its a similar aproach to Acronis True Image, you might heard of it maybe.

[frankus333]

Thanks Kiraitachi

Yes, I will probably use clonezilla to image my system partition to be able to recover from disasters.

However, it seems cumbersome to have to reboot into live a USB to do regular data backups.

I am wondering about the reliability and stability of Kup, which seems like it might work well with Kubuntu:

http://kde-apps.org/content/show.php...content=147465

and would value feedback from anyone who has used it recently.

My main concern is still the permissions issue, and what precautions I may need to take to guard against another mixup while using whatever backup system from the limited user account, backing up data stored on a separate NTFS data partition, to an external NTFS HDD.

[kiraitachi]

Kup looks very polished for regular backups, Clonezilla is more for a stable image backup where you want to drop in several computers or just to make a clone of your kubuntu OS with everything working perfect and customized.

Anyway for regular backups I would just do a CRONTAB entry to run a backup script at a time, that looks pretty easy to acomplished...even I have at my raspberry Pi a script for backups.

Let me share the code is pretty easy:

echo "Hi tell me what folder you want to backup (full path)";
read backup;
echo "Ok now tell me where you want to save it (full path)";
read dest;

sudo mkdir "$dest"/backup;
sudo cp -r "$backup" "$dest"/backup;

echo "Backup done";
ls "$dest";


fecha=$(date +"%F");
sudo touch "$dest/backup/Backup from $fecha";

just save that code into a file called backup.sh and give full permissions with chmod 777 then just run:

sudo sh backup.sh

Enjoy

[SteveRiley]

I see that Lucky Backup and Back in Time seem to rate well, but are not supported by Canonical.

The Ubuntu repositories contain four components:

• Main - Officially supported software.
• Restricted - Supported software that is not available under a completely free license.
• Universe - Community maintained software, i.e. not officially supported software.
• Multiverse - Software that is not free.

Lucky Backup and Back in Time come from the Universe component. This should not be a deterrent for you -- the programs are very good at what they do, and well supported by the community. Official Canonical support is only important for enterprises that purchase technical support from Canonical. In these cases, Canonical provides support only for packages that come from the Main and Restricted components.

[frankus333]

Thanks Kiraitachi:


I am not comfortable enough with terminal use to set up backups from a command line, so will keep your suggestions on the back burner for now.


I also want an ongoing, regular process that allows me to manually increment the backups, so they don't take forever each time, which my temporary solution of a simple copy and paste of the entire data partition contents to the backup folder now takes.


I don't know enough about coding, but I can't see how your script does repeated incremental backups, unless that is what automatically happens each time it is run after the first time?


Thanks SteveRiley:


The reliability and community support for those programs is reassuring to hear, and I will look at them when I am confident that I know how to prevent permissions issues or whatever else it was from messing anything up again. Clarifying that risk, and appropriate precautions to take, is my main concern.


To collate and clarify from my first and third posts in this thread:


“So, I think my question is whether there are any considerations (and what they might be) re working in an everyday user level account instead of the administrator account, when using a backup program such as Ubuntu Backup for [regular, incremental,] manual backups of my [NTFS] data partition [to an external NTFS HDD]. Some of the Home folders of the limited account would be included in the backups, to ensure any recent downloads or files saved prior to sending them to the data partition are not missed.”


“So I am still curious to know whether using [the limited] account might have something to do with the Backup program failure. I'm guessing that whatever the .tmp file was meant to be, its " non-existence" is related to not being able to be [created or] seen due to permissions issues. I would like to play around with a backup program a bit more, but am reluctant until I understand more about what might get broken and how [-and how to prevent that].”


Cheers

[kiraitachi]

Thanks Kiraitachi:


I am not comfortable enough with terminal use to set up backups from a command line, so will keep your suggestions on the back burner for now.


I also want an ongoing, regular process that allows me to manually increment the backups, so they don't take forever each time, which my temporary solution of a simple copy and paste of the entire data partition contents to the backup folder now takes.


I don't know enough about coding, but I can't see how your script does repeated incremental backups, unless that is what automatically happens each time it is run after the first time?

Cheers

True Hahaha didn't realize the incremental backup until you said, my script just does a whole backup of directory given, It was made for my small raspberry pi hehe. So may take more time with useless copy paste of data that is already backup, maybe would me more optimal a script with rsync...

Here some info you sure would find usefull, since I think rsync is the best aproach.

http://en.wikipedia.org/wiki/Rsync

rsync is a utility software and network protocol for Unix-like systems (with a port to Microsoft Windows) that synchronizes files and directories from one location to another while minimizing data transfer by using delta encoding when appropriate. It also has the option to provide encrypted transfer by use of SSH. SSL encrypted transfer can be done via Stunnel wrapping. Quoting the official website: "rsync is a file transfer program for Unix systems. rsync uses the 'rsync algorithm' which provides a very fast method for bringing remote files into sync." A feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction, eliminating the message latency overhead inherent in transmitting a large number of small messages. rsync can copy or display directory contents and copy files, optionally using compression and recursion.

http://www.howtogeek.com/135533/how-...data-on-linux/

[gldvorak]

I have been using BackInTime to back up my entire system daily. It does an incremental backup using rsync so very little disk space is used. It seems to be upgraded quite often and the few times that I had a question it was usually answered by the lead developer.

I do have problems but that is when I ignore the suggestions and go my own way. I can hardly blame the app for those problems. The most recent was my attempt to include a couple of root folders that the app strongly suggests that one not back up. Well, I learned why they did not want you to try!

So, I highly recommend BackInTime and I can help with the set up. There are two versions, one runs as root and I use that to back up folders that have root permissions. The other that does not run as root I use to back up my Home directory. When it comes up with errors it is because a folder is mistakenly marked as root and I fix that. NOTE: I have not mentioned Windows. I do have Windows 8.1 on this machine and I have booted into it a couple of times but have not done so for ages.

Strange, strange. I finished typing and went up to check for spelling errors and garbled sentences. Sure enough there were some strange chunks of characters. I ignored those at first and started fixing spelling errors. Before I knew it I was at the bottom and had not encountered the garbled area. I had similar occurrences earlier today, but in that case I had to retype the garbled area. Don't understand!

[kiraitachi]

I have been using BackInTime to back up my entire system daily. It does an incremental backup using rsync so very little disk space is used. It seems to be upgraded quite often and the few times that I had a question it was usually answered by the lead developer.

I do have problems but that is when I ignore the suggestions and go my own way. I can hardly blame the app for those problems. The most recent was my attempt to include a couple of root folders that the app strongly suggests that one not back up. Well, I learned why they did not want you to try!

So, I highly recommend BackInTime and I can help with the set up. There are two versions, one runs as root and I use that to back up folders that have root permissions. The other that does not run as root I use to back up my Home directory. When it comes up with errors it is because a folder is mistakenly marked as root and I fix that. NOTE: I have not mentioned Windows. I do have Windows 8.1 on this machine and I have booted into it a couple of times but have not done so for ages.

True just looked backintime looks like a nice app with both gnome and kde frontends for rsync, you should give a shot to his suggestion

[frankus333]

Thanks gldvorak and kiraitachi for your interest and suggestions.

When I gain sufficient understanding of the permissions or whatever risks as outlined, I look forward to exploring these.

[frankus333]

G'day folks

I have been unable to find or receive any clear, useful information on the permissions question, so have resorted to experimentation with Back In Time (root).

After a couple of false starts and blind alleys, I now have the software set up on three machines, in the administrative account, and run it in (root) mode. As long as I am careful to mount all the required drives via Dolphin, I have been able to create incremental backups to an external NTFS HDD on each machine, which include the shared NTFS data drive, and the Desktop, Documents and Downloads folders in both the administrative and everyday user accounts, for both the LM17 and Kubuntu 14.04 installations on the machine. (I am not using any encryption on the drives.)

All these potential data repositories are saved to backup in one operation for each machine. Back In Time seems relatively forgiving of mistakes, and I have been able to adjust the backup files (by taking another snapshot) to correct instances when I didn't have a particular drive/folder mounted at the time.

I am aware that the Home folders/files for the non-booted installation are probably read only as far as Back In Time is concerned (so possibly can't be restored via the program), but I am hoping they would be able to be recovered manually from the external drive if needed. I will experiment some more to test this. [EDIT: Seems Back In Time is able to deal with the permissions issues to allow restoring to the non-booted installation's drive, at least the administrative account on that drive; very impressive!]

I plan to run the backups manually on a weekly basis, and see how it goes for each machine.

[frankus333]

G'day folks

I am prepared to mark this as solved, as the program seems to be working on all three machines, backing up all relevant data stores on both accounts on both operating systems, in one operation, on each machine. I have to boot into the administrative account on one of the OSes to do it, but I can live with that.

Thanks for the interest and suggestions.

Cheers

[gldvorak]

On my system (Kubuntu 14.04) two versions of backintime are provided. One has user access and the other has root access. If you are running windows, I have no idea.

So when I am backing up folders that have root access I use the second version and never log in except my usual user account.

George

[frankus333]

Thanks gldvorak

I don't store any data on my Windows OSes partitions; everything of value goes to the shared data partition used by all OSes on each machine. I only use Win7 partition (on one laptop) to access iTunes and MyDrive, for my iPod/iPhone and TomTom respectively, as nothing in Linux manages these devices properly (unfortunately).

The Home/ Documents, Downloads and Desktop folders on both the administrator and the limited user accounts, on both LM17 KDE and Kubuntu 14.04 OSes, plus the shared data partition, are what gets backed up on each machine. This is run on each machine from the LM17 KDE administrator account using the (root) version of BackInTime; everything gets backed up in one operation for each machine.

I can't run BackInTime from the limited user account because this account does not have sudoers permission, which I don't want to change, as it is one of the main security reasons I use a limited user account for everyday computing.

Cheers

Frank