Anyone try Timeshift yet?

[oshunluvr]

https://teejeetech.in/timeshift/

[TWPonKubuntu]

Yes, I have it installed. One word of advice to users, go into Settings->location and pick a partition which has enough room for the snapshot files. They can be large. I tried the first time using my smaller /sda partition and immediately ran out of space. I moved the location to /sdb (/home) where I have the majority of my working space available.

[GreyGeek]

https://teejeetech.in/timeshift/

Remember to delete all snapshots before uninstalling TimeShift. Otherwise the snapshots continue to occupy space on your system. To delete all snapshots, run the application, select all snapshots from the list (CTRL+A) and click the Delete button on the toolbar. This will delete all snapshots and remove the /timeshift folder in the root directory.

If you used the installer to install Timeshift, you can remove the installed files with following command:
sudo timeshift-uninstall
.

[TWPonKubuntu]

I had no problem with the application files being in my smaller partition. It was the large snapshot footprint, which was filling all my partition space. Fixed that by moving the location setting.

[Chopstick]

Hey, so this may be a bit late to this thread, but it relates to Timeshift.
I've recently (Feb. '21) reinstalled my system with BTRFS and LUKS full disk encryption, following this guide:
https://mutschler.eu/linux/install-guides/ubuntu-btrfs/
As recommended in the guide, I also installed Timeshift as well as timeshift-autosnap-apt and grub-btrfs.
I have not used Timeshift to restore anything, but it is creating regular snapshots and had a nice GUI

That being said, I am a bit disappointed that eventhough it supports a nice tiered backup scheme, it can only back up to a single location, which at the moment is on the same physical drive as my system and home. I consider this less than ideal. I realize that for frequent snapshots it is much faster on the same partition due to CoW and grub-btrf would probably only work that way, right?

But what I really need the backup for is when I loose the laptop or the drive fails. Is there a way to do both, local snapshots and external backups, with Timeshift, or should I just use something different? Or is there a way I could just mirror the timeshift-btrfs folder on an external drive? I am worried that if I do that, I will end up with only incremental changes, but not the base copy... which would make it pretty useless for a complete system restore.
Thoughts? What strategy do you use?

[GreyGeek]

TimeShift has two glaring problems:
1) It cannot store snapshots on remote locations
2) IF you plan to uninstall TimeShift you MUST first delete all your snapshots before you uninstall it. Otherwise, it will bork your system. I experienced that personally.

Since making a snapshot is so easy, and the send & receive commands allow incremental backups, I wrote my own backup script to do that. Now, all I do to make a local and incremented remote snapshot is to run "sudo /make_snapshot.sh" in a Konsole. I've been too lazy to add it to the menu. Also, I cut my snapshot overhead and workload in half by merging the @home subvolume into @ and commenting out the line in /etc/fstab the mounts @home at /home. Thus, I only have to make one snapshot of @ to capture my entire system. Last night, making a local snapshot of @ as @202103122023 and then doing an incremental backup to my remote HD took all of about 10-15 seconds. I have my script set to keep 5 snapshots, adding a new one and dropping the oldest one every day. The script is here. Modifying it for both @ and @home is something you'd have to do.

Oshunluver addressed the task of sending snapshots to remote servers using ssh.

And, here is a method for remote backups using BTRFS.

EDIT:
For clarity, I do not send my snapshots to remote drives. I have two additional drives mounted in my laptop (3 in total), and I back up to the 2nd and 3rd. I also have several external USB drives which I mount at /backup and send incremental snapshots to them. Regardless, my total backup snapshot time rarely exceeds 2 minutes. The time is mostly less than 30 seconds.

[Chopstick]

Thanks for the summary - sounds like you have a good system!

Regarding Timeshift: is seems to store its snapshots in '/run/timeshift/backup/timeshift-btrfs/snapshots', so would it be possible to just copy/send these snapshots to an external drive every now and then?
Similarly, if you uninstall Timeshift, can you not just delete these snapshots?

I guess I am not really confident in my understanding of how snapshots and incremental backups work under the hood. Is it like Git, where you essentially need to copy the entire history of diffs since your last complete (checked-out) version of the repo and said checked-out base to get a complete copy, so that if you only copy the diffs without the base, you are screwed?
Or can you just copy a snapshot, even if it was an incremental one, and the filesystem will make sure that you end up with a complete copy on the volume that you are sending to (and possibly copying everything, if there are no previous increments present)?

[GreyGeek]

...
Regarding Timeshift: is seems to store its snapshots in '/run/timeshift/backup/timeshift-btrfs/snapshots', so would it be possible to just copy/send these snapshots to an external drive every now and then?
Similarly, if you uninstall Timeshift, can you not just delete these snapshots?

It is recommended that you use TimeShift to delete the snapshots it creates. @ and @home both appear under /run/timeshift/backups while also appearing under /mnt if you mount your btrfs drive to /mnt. I deleted TimeShift first and then attempted to delete the snapshots under /run/.,... causing my @ and @home to be deleted. Only then did I discover the caveat that TimeShift must delete its own snapshots before it is deleted.

I guess I am not really confident in my understanding of how snapshots and incremental backups work under the hood. Is it like Git, where you essentially need to copy the entire history of diffs since your last complete (checked-out) version of the repo and said checked-out base to get a complete copy, so that if you only copy the diffs without the base, you are screwed?
Or can you just copy a snapshot, even if it was an incremental one, and the filesystem will make sure that you end up with a complete copy on the volume that you are sending to (and possibly copying everything, if there are no previous increments present)?

The "repo" (repository) has no bearing on making or handling BTRFS snapshots. And, "diffs", i.e., the "incremental" part of send & receive, is handled entirely by the send & receive command. Keep in mind that while the receive part of the command usually expects to be operating on a BTRFS subvolume, it is possible to send your system as an ASCII stream, using the "-f" switch to an HD that is formatted using some other file system, like EXT4 or FAT32, etc.... Using the send & receive command in reverse using "-f" converts the ASCII data back to a snapshot.

Generally, one should use the "-r" (read only) switch when creating a snapshot. That is because snapshots used for backup are generally sent to remote drives. One can mv or copy a rw snapshot anywhere under its <ROOT_FS> system but only an ro snapshot can be sent to remote locations using send & receive, i.e., one made using the "-r" switch.

The command
btrfs send -p /mnt/snapshots/@mm01yyyy /mnt/snapshots/@mm02yyyy | btrfs receive /backup

uses @mm01yyyy as the parent snapshot and a copy of that snapshot is expected to already be on /backup from a previous send & receive. The receive command sees mm01yyyy and looks for it on /backup. Using it, receive then takes what send gives it ... the difference between @mm01yyyy and @mm02yyyy ... and adds it to a copy of @mm01yyyy, creating @mm02yyyy on /backup. Ditto for each successive day. A few minutes ago I ran my backup script. It took exactly 30 seconds to make @202103132115 on /mnt/snapshots/ and then use that snapshot and the one made yesterday to create a diff stream which was sent by send & receive to /backup, my remotely mounted drive.

Keep in mind that while using -r during the snapshot command to make a ro snapshot, if one has to restore using an ro snapshot it must first be converted to a rw snapshot using the command:
btrfs property set -ts /path/to/snapshot ro falseChange "false" to true to make a snapshot read only. Read only snapshots freezes the state of the system at the moment it was created. If you do not use an incremental backup and just send & receive the entire snapshot of your system to a remote drive it can take 15 to 30 minutes or more, depending on how big your system is, to send that snapshot. Before I packed @home into @ it took me about 10 min to send @ and about 15 minutes to send @home. My <ROOT-FS> subvolume size is about 130GB. If I didn't use incremental backups instead of taking 30 seconds to send my @ it would take about 30 minutes. However, you can continue to use your system while that snapshot is being sent, if you want to.

When first created, snapshots are essentially empty and composed mainly of meta data. As you use your system and modify a file a copy of it before any changes is written to all relevant snapshots. Older snapshots began filling up and eating your drive space. While one could theoretically take a snapshot every minute of their system, the BTRFS developers recommend not creating more than a dozen or so snapshots for each subvolume. And, at some point, began deleting the oldest snapshots as the newest ones are created. Fstrim run about once a month or so discards unused blocks.. Fstrim.timer fires it off.

:~$ systemctl status fstrim
● fstrim.service - Discard unused blocks on filesystems from /etc/fstab
Loaded: loaded (/lib/systemd/system/fstrim.service; static; vendor preset: enabled)
Active: inactive (dead)
TriggeredBy: ● fstrim.timer
Docs: man:fstrim(8)

[jlittle]

But what I really need the backup for is when I loose the laptop or the drive fails. Is there a way to do both, local snapshots and external backups, with Timeshift, or should I just use something different?

Definitely use something different, at least in addition to Timeshift.

Timeshift for Linux is an application that provides functionality similar to the System Restore feature in Windows and the Time Machine tool in Mac OS...
It is designed to protect only system files and settings. User files such as documents, pictures and music are excluded.

<ramble mode, ignorable>
I do backups manually, as well as running "snapper", which takes snapshots regularly, by default every two hours. Snapper has improved a lot since I started using it. (When Greygeek tried it out, IIRC, it filled up his drive and he swore off it.) IIUC Snapper is installed by default on openSUSE, so I imagine there's new user advice and set up recommendations in the openSUSE wiki and forums.

IMO doing at least some backups yourself is a good idea, because it helps you understand what can be done. The fact that your snapshots are sitting there all the time, and can be explored using dolphin (or your favourite file manager), enables many liberating uses. Delete some stuff by accident? No need to find a backup, you've already got it (if it's not too new). Reverting a system to a snapshot (say, after a bad update) can be done quickly (in seconds) by renaming files and a reboot, so being familiar with the file names helps. </ramble>

[rdonnelly]

I use Luckybackup for my /home, USB thumb drives, and Vbox machines.
Also have 2 hard drives, I connect one at a time, and clone and swap on a monthly basis.
Just installed Timekeep, you can never have enough protection.

[GreyGeek]

Definitely use something different, at least in addition to Timeshift.


<ramble mode, ignorable>
I do backups manually, as well as running "snapper", which takes snapshots regularly, by default every two hours. Snapper has improved a lot since I started using it. (When Greygeek tried it out, IIRC, it filled up his drive and he swore off it.) IIUC Snapper is installed by default on openSUSE, so I imagine there's new user advice and set up recommendations in the openSUSE wiki and forums.

IMO doing at least some backups yourself is a good idea, because it helps you understand what can be done. The fact that your snapshots are sitting there all the time, and can be explored using dolphin (or your favourite file manager), enables many liberating uses. Delete some stuff by accident? No need to find a backup, you've already got it (if it's not too new). Reverting a system to a snapshot (say, after a bad update) can be done quickly (in seconds) by renaming files and a reboot, so being familiar with the file names helps. </ramble>

I did swear off of it. OpenSUSE has made regular improvements and it is much better now. One must create config files for both root and home. Root is run with sudo and the home backup does not require sudo. One neat feature is the link to apt that automatically creates a snapshot when it is run so that one can roll back if the install/remove/update/upgrade barfs a lung. If the modifications work as expected one can delete the peremptory snapshot. That's a nice feature but it is easily duplicated manually with a simple sudo snapshot command.

HOWEVER, snapper still has one major, glaring fault, IMO. It stores the root snapshots under / and the home snapshots under /home. These snapshots are accessible ONLY if you can boot into your system since they are inside @ or @home. IF @ doesn't load then you are out of luck and better have a copy of @ on another HD so you can go through the process of mounting your HD from a LiveUSB and then overwriting @ and @home with your backup versions of @ and @home.

I save my snapshots OUTSIDE @ or @home. By mounting your HD to /mnt
mount /dev/disk/by-uuid/ce2b5741-c01e-4b3d-b6ba-401ad7f7fcdf /mnt
Using vdir /mnt you will see:
/mnt
/mnt/@
/mnt/@home

You can create /mnt/snapshots and store your snapshots in that folder:
btrfs su snapshot -r /mnt/@ /mnt/snapshots/@202103141231

IF /mnt/@ (which is your live, running system!) becomes corrupt and won't boot then you can still mount the drive using a LiveUSB kubuntu (it contains btrfs) because while @ may be corrupt the drive will still mount. Mounted, you can use mv and btrfs to change @202103141231 to a rw snapshots. Then you can
mv /mnt/@ /mnt/@old
The system will continue to run even though @ is now @old. And then
btrfs subvol snapshot /mnt/snapshots/@202103141231 /mnt/@
Notice the lack of "-r" in the above command. @ will be rw. Ditto for @home if necessary. Never mix up your @ and @home snapshots. I.E., don't use snapshots of @ and @home taken at different times.

Now shut down the LiveUSB and reboot. Your system will start up.

[kubicle]

HOWEVER, snapper still has one major, glaring fault, IMO. It stores the root snapshots under / and the home snapshots under /home.

You can have a separate snaphots subvolume and have that mounted to /.snapshots, if you wish.
https://wiki.archlinux.org/index.php...esystem_layout

[Chopstick]

Thanks for the info, guys! I think I understand the process much better now!

@GreyGeek your script was very helpful to me in figuring out how this works! The answer to my question above is essentially, that within a volume BTRFS will not duplicate files between snapshots unless they are modified, and it will keep files around as long as any snapshot references them. So, if we do incremental backups on an external HD we can delete the first or any number of snapshots, and that will not compromise the integrity of other snapshots. I tested this myself.
I have another question for your script that comes in the next post.

@jlittle I agree, doing this yourself a couple of times is important, to understand the process - I did that, and now I am ready to move on to a more convenient solution. Initially I was actually under the impression that Time Machine was a full backup solution and not just system restore, so I assumed Timeshift could do that, too.

What do you guys think of btrbk:
https://work-work.work/blog/2018/12/...kup-btrfs.html
E.g. as opposed to snapper? It seems quite simple.

As for Timeshift itself: I guess it is still useful, but my initial thought of just mirroring/send-receive the snapshots Timeshift makes, wont work, since those do not seem to be read-only, which appears to be necessary for send-receive.
So, since I need to make a separate set of snapshots, it may not make sense to keep my @home in it as well, especially since I also have timeshift-autosnap-apt installed and I feel like having @home included is slowing down apt...

Do you guys use timeshift-autosnap-apt and grub-btrfs at all?

[Chopstick]

HOWEVER, snapper still has one major, glaring fault, IMO. It stores the root snapshots under / and the home snapshots under /home. These snapshots are accessible ONLY if you can boot into your system since they are inside @ or @home. IF @ doesn't load then you are out of luck and better have a copy of @ on another HD so you can go through the process of mounting your HD from a LiveUSB and then overwriting @ and @home with your backup versions of @ and @home.

This was actually the question I had to your script, GreyGeek: why do you mount to /mnt/@ before you make the snapshot? Why does making a snapshot to a subfolder and then send-receiving it to an external HD not work to restore your system? Would that work for @home?
I am a bit confused why everybody seems to mount the subvolumes first, eventhough they are already mounted (since the system is running).

Also, is "@" just a convention to name subvolumes? When I created snapshots of my home, I just put them into /Snapshots and called them home_* ... no "@" - is there a problem with that? (I will probably use the @ in the future, just for visual clarity, but I am wondering where this is coming from.)

[GreyGeek]

This was actually the question I had to your script, GreyGeek: why do you mount to /mnt/@ before you make the snapshot? Why does making a snapshot to a subfolder and then send-receiving it to an external HD not work to restore your system? Would that work for @home?

My mount command is
mount /dev/disk/by-uuid/ce2b5741-c01e-4b3d-b6ba-401ad7f7fcdf /mnt
so I am not specifically mounting @. /mnt becomes the <ROOT_FS> (root filesystem) when I mount the disk to /mnt. One can mount @ or @home specifically to a mount point, as shown in my (and your) /etc/fstab file:

[FONT=monospace][FONT=monospace][COLOR=#000000]UUID=ce2b5741-c01e-4b3d-b6ba-401ad7f7fcdf /               btrfs   defaults,subvol=@ 0       1 [/COLOR]
# /home was on /dev/sda1 during installation 
#UUID=ce2b5741-c01e-4b3d-b6ba-401ad7f7fcdf /home           btrfs   defaults,subvol=@home 0       2
[/FONT][/FONT]

Since I copied the contents of @home into /mnt/@/home I no longer need to mount @home, so I commented it out and my fstab only mounts @ as "/". Notice that in the past when I did mount @home it was being mounted as /home in fstab.

During installation the HD is formatted with BTRFS and the HD becomes the root filesystem, <ROOT_FS>. It is the TOP of the BTRFS binary tree, the parent subvolume The first two child subvolumes created during the install are @ and @home. The naming convention @ and @home were arbitrarily chosen by Canonical. They could be named anything. You could, for example, make a rw snapshot of @ and call it Chopstick:
btrfs su snapshot /mnt/@ /mnt/Chopstick

(notice that since "-r" was not used Chopstiks is read-write)
then open /etc/fstab and change subvol=@ to subvol=Chopstick and save it. Then reboot.
When you open a Konsole and do "vdir /" you will still see the usual hierarchical structure, with "/" at the top. Mount your HD to /mnt and you will see /mnt, /mnt/Chopstick and /mnt/@home. You will also see /mnt/@ but you can delete @
btrfs subvol delete -C /mnt/@
and your system will continue to work fine, even after rebooting again, because @ is no longer being used.

You could add another child leaf to the parent leaf, <ROOT_FS> by creating, say, @data.
btrfs subvol create -c /mnt/@data
(the -c commits the operation)
Now, <ROOT_FS> has three child leaves under it.
Create a place to mount it to. Under your home account add a subdirectory, say, Data.
Then, in /etc/fstab, bind @data to /home/chopstick/Data

[FONT=monospace][FONT=monospace]UUID=yourhduuid /home/chopstick/Data    btrfs   defaults,subvol=@data 0       3[/FONT][/FONT]

which is AFTER the stanza in fstab that mounts @home to /home.
Now you have a directory under your home account in which you can store data which is, for all practical purposes, separate from either / or /home/chopstick. When you create a snapshot of it:
btrfs subvol snapshot -r /mnt/@data /backup/snapshots/@data2021031513455
that snapshot does not include anything in / or /home/chopstick, just what is in /home/chopstick/Data.

I am a bit confused why everybody seems to mount the subvolumes first, even though they are already mounted (since the system is running).

Notice that in fstab what is being mounted is @ and @home, not the <ROOT_FS>. Ergo, from within / or /home you have no access to <ROOT_FS>. because you cannot "cd .." any higher while at /. By mounting the HD to /mnt, and not a subvolume leaf, you gain access to the <ROOT_FS> as /mnt. You are free to work on it as you please. If you use mv to move @ to @old:
mv /mnt/@ /mnt/@old
the system continues to run normally. If you delete @ your system will hang and you will have to use a LiveUSB to recover using a previous snapshot.

You can make a rw snapshot of some previously created read-only snapshot, say @2021031245 and name it @ and make it read-write by not using "-r":
btrfs su snapshot /mnt/snapshots/@2021031245 /mnt/@ (notice the missing "-r")
Then you can umount /mnt and reboot. You have successfully rolled @ back to @2021031245

Also, is "@" just a convention to name subvolumes? When I created snapshots of my home, I just put them into /Snapshots and called them home_* ... no "@" - is there a problem with that? (I will probably use the @ in the future, just for visual clarity, but I am wondering where this is coming from.)

Yes, the use of "@" is just a Canonical naming convention. The only caveat is:

The btrfs-tools command ''set-default'' will break Ubuntu's layout

Since Ubuntu is set up to always keep the top of the btrfs tree as the default mounting subvolume it will break when using the btrfs-tools command set-default, since this command is specifically designed to change the default mounting subvolume.

The mount options for / and /home described above relies on the fact that the corresponding subvolumes @ and @home can be located below the default mounting subvolume, and if set-default is used, this is no longer the case.

If you have accidentally used set-default and want to revert, you can do the following

sudo mount /dev/sdX# /mnt
sudo btrfs subvolume set-default 5 /mnt

since the id 5 is a permanent alias for the top of the btrfs tree.

The "default mounting subvolume", i.e., the "top of the tree", is the <ROOT_FS>, i.e., the BTRFS formatted HD.

BTW, never use /dev/sdX# to mount an HD. Use the unique UUID designation, which can be obtained by using "blkid" in a Konsole. Why? Because if you mount or remove a drive the system can switch the /dev/sdX designation of a drive without notification. The UUID was created to avoid that. When I was trying out BTRFS's raid on two HDs they were /dev/sda and /dev/sdb. I then removed my CDROM and put in an HDCADDY, into which I plugged in a 3rd HD. While perusing the file structure I noticed that my raid was now composed of /dev/sda and /dev/sdc. and the 3rd drive was now /dev/sdb. After I switched the raid to two SINGLETON's I noticed that the /dev/sdX designations remained the same, and that is how they are to this day, three years later.