full root backup

[vsreeser]

I want to find an easy way to make an image of the root system automatically. I can do it manually with clonezilla. Can't get that to boot from the iso, but that's another problem, I suppose. I want an image so if I blow it up, I can just reimage the drive and be back up quickly. A restore of a clonezilla image currently takes less than 7 minutes. About 15 to make it including booting from cd. Might sound like I'm being lazy, but computers are supposed to make our lives easier. Of course, we were told in the 80s of a paperless future. If anything, there is more paper now!!!

[oshunluvr]

Would you have to reboot to use dd? I guess the issue if you didn't would be a file changing during the image would result in corruption. dd cloning has the limitation of being restore-able only to an identically sized partition or you lose usable space.

The true best solution (here I go again ) is to convert to btrfs. Then you can make snapshots and/or "send" a clone of your install to another drive or computer.

Snapshots protect you from upgrade bugs or unintended deletions and the send/receive features allow full cloning without even logging out, much less rebooting.

[doctordruidphd]

I do this routinely with rsync. The problem with using dd is that you are backing up blank space.

If you are booting with another system, or a live cd, then it's simple:

sudo rsync -av /where/you/are/backingup/from /where/you/are/bacingup/to

For example, if I boot into my LTS system, and the system I want to back up is mounted as /media/stable and I want to put it on a removable disk mounted as /media/backupvault then I would use the following command:

sudo rsync -av /media/stable /media/backupvault

This will create a directory called stable on the backupvault device and put the whole system in it.

Now if you want to backup from a running system, you can still do it, but you need to set up an exclusion file so that rsync doesn't wind up recursively backing up itself.

Sample exclusion file:

###################
# Include
+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero
+ /media/*

# Exclude
- /home/greenman/.gvfs/
- /dev/*
- /media/*/*
- /mnt/*
- /proc/*
- /run/*
- /srv/*
- /sys/*
- /tmp/*
- /var/run/*
- /var/tmp/*
- lost+found/


################

Save this somewhere (for example, /usr/local/sbin) as rbackup.list, then you would do:

sudo rsync -av --exclude-from=/usr/local/sbin/rbackup.list / /media/backupvault/stable


This will create the stable directory on the device mounted as /media/backupvault.

I routinely use this method not only for backups, but also to copy from one partition to another before doing version upgrades.

Note that the rsync command has an extensive man page that is actually helpful and comprehensive.

[GreyGeek]

Of course I second oshunluvr's solution!
Btrfs makes FS maintenance a breeze.

[vsreeser]

Would you have to reboot to use dd? I guess the issue if you didn't would be a file changing during the image would result in corruption. dd cloning has the limitation of being restore-able only to an identically sized partition or you lose usable space.

The true best solution (here I go again ) is to convert to btrfs. Then you can make snapshots and/or "send" a clone of your install to another drive or computer.

Snapshots protect you from upgrade bugs or unintended deletions and the send/receive features allow full cloning without even logging out, much less rebooting.

This is what I am looking for. A way to get a clone and restore it if un upgrade blows it up. Or i do it myself with tinkering. /data, /home/ and vms are on separate drives. The drive I want to clone is 500Gb, and I have a partition on another drive set up to receive it. I'l need to go look around and find out how to convert I guess, unless someone has a tutorial to post here. Any performance gain/loss?

[oshunluvr]

Real world performance is roughly the same IME. VM dynamic virtual drives don't work on COW filesystems so you have to leave them on the separate partition/drive. Also swap should remain on a partition (vs. a swap file).

The partition conversion can be done by btrfs-convert OPTIONS <device>

The only options are:
-d Disable data checksum.
-i Ignore xattrs and ACLs.
-n Disable packing of small files.
-r Roll back to ext2fs.
-l <LABEL> set filesystem label during conversion.
-L use label from the converted filesystem.

The problem you're going to run into with a direct conversion using btrfs-convert is it converts the whole parition into a top level filesystem- no subvolumes. Unfortunately snapshots and by extension send/receive functions are only usable on subvolumes. The default btrfs (K)Ubuntu install puts / into a subvolume labeled @ and /home into @home. From there it's super easy to snapshot and backup. I just don't see an easy way to convert with rebooting into another install (liveUSB/CD would work) or just doing a new install to a new btrfs partition.

You could take these steps with your current setup:

1) Convert the partitions to btrfs on all the drives except the VM drive.
2) Create @home, @data subvolumes on the drives holding these files.
3) Copy /home and /data to their respective new subvolumes using cp --reflink=always to vastly speed copying the files (the files have to be on the same btrfs filesystem as the subvolumes to do this).
4) Remount /home to @home and /data to @data.
5) Remount the root volumes of these two drives somewhere else and delete the now duplicated files from the root filesystem leaving the subvolumes behind.

You would then have two mounted subvolumes - one each for @home and one for @data. I'd make a read-only snapshot of the subvolumes before deleting the root level files just to be safe. btrfs filesystems can be mounted at the root level or at the subvolume level or both simultaneously so doing this will allow you to purge the root level files without deleting the subvolumes.

You could then either do the same for your install partition (but you'd have to boot to a live image or another install to copy the files) and edit your grub.cfg and fstab to include the subvol= option or just do a new install to the empty partition selecting btrfs at the install time and then send/receive the @home and @data subvolumes to the new partition.

Either way it's not going to be an instant process. I'm sorry there's not an easier way.

Further explanation of the mounting of btrfs: You can mount the top level or a subvolume of a filesystem. For example, the partition /dev/sdb2 holds a btrfs filesystem. Within that filesystem are the subvolumes @, @home, and @data. @ contains your OS and all its files and folders, @home contains all your home files, and @data the extra data you have.

If you mount using this command:

sudo mount /dev/sdb2 /mnt/drive

you have mounted the top level filesystem so a listing of /mnt/drive would look like this:

@  @home  @data

A listing of any of these would show their contents.

If you then mounted /dev/sdb2 using these two commands:

sudo /dev/sdb2 -o subvol=@home /home
sudo /dev/sdb2 -o subvol=@data /data

you would see your home files in /home but also in /mnt/drive/@/home and in /mnt/drive/@home. Likewise data is in /data and /mnt/drive/@/data and /mnt/drive/@data.

In normal operation, you would have three nearly identical mount lines in fstab, one for each subvolume, with the only differences being the name of the subvolume itself, like so:

UUID=6812caf0-14bb-4dcf-aab3-3104f297c6a0 /      btrfs   rw,relatime,space_cache,compress=lzo,subvol=@
UUID=6812caf0-14bb-4dcf-aab3-3104f297c6a0 /home  btrfs   rw,relatime,space_cache,compress=lzo,subvol=@home 
UUID=6812caf0-14bb-4dcf-aab3-3104f297c6a0 /data  btrfs   rw,relatime,space_cache,compress=lzo,subvol=@data

You would then only see the files in their mount locations like normal. If you wanted to make a snapshot or a backup of one of these subvolumes, you would then need to mount the top level filesystem, make your snapshots and backups, and unmount it when you're done. I always just mount the top level at boot too so I can do my snapshots a few steps quicker.

One of the beauties of having all your data in subvolumes on the same filesystem is they now all have access to the free space - no partitions required. My media server has about 3TB of files all in separate subvolumes (11 in all) on a single 6TB drive - all one filesystem. For backups, I use three 2xTB drives and the send/receive functions to make full and incremental backups when I feel it necessary. I am writing a script for cron to do it all automatically in the near future.

[GreyGeek]

VM virtual drives don't work on COW filesystems so you have to leave them on the separate partition/drive

True, but if you create fixed sizes virtual drives instead of dynamic sizes drives they work fine on Btrfs.

[oshunluvr]

True, but if you create fixed sizes virtual drives instead of dynamic sizes drives they work fine on Btrfs.

True (post edited). In this case, since vsreeser already has a separate partition for the VMs so it seems simpler to leave that part as is...