I have very recently installed Kubuntu 18.04 on a 500gb HD, with no Windows or other extraneous objects. I have made my first snapshot following instructions but have no idea what I am doing. What I need is a list of btrfs instructions for taking snapshots on a regular basis. Plus I'm a little confused--what happens to those snapshots? Where do they reside? How much memory is involved? If I am to get the most benefit out of my new file system I have a lot to learn.
Announcement
Collapse
No announcement yet.
I have btrfs and now what?
Collapse
This topic is closed.
X
X
-
Show us the snapshot command you used.
You have two subvolumes: /@ and /@home
Did you make snapshots of both, one after the other?
There is a reason for making both snapshots at the same time. Application installations can write to both root and home, leaving config and other files in both places. (Apps run as AppImages are totally located within their image file). To back up one without the other, or to restore one without the other would/could disconnect an app from its supporting files.
The man pages tells it all for Btrfs. You can use "man btrfs > btrfs.txt" to create a text file of Btrfs commands. Here are the other btrfs commands. Use "man xxxx" for them.
SEE ALSO
mkfs.btrfs(8), ionice(1), btrfs-balance(8), btrfs-check(8), btrfs-device(8), btrfs-filesystem(8),
btrfs-inspect-internal(8), btrfs-property(8), btrfs-qgroup(8), btrfs-quota(8), btrfs-receive(8),
btrfs-replace(8), btrfs-rescue(8), btrfs-restore(8), btrfs-scrub(8), btrfs-send(8), btrfs-subvolume(8),
Here are my steps to making backups, and restoring from backups. (Note that the first time I mounted /mnt as root I created /mnt/snapshots as a folder to store my backup snapshots in. That folder is not accessible from within the user's home account without sudo'ing and then mounting the drive as /mnt, so it can't be accidentally changed or deleted.)
Making a backup snapshot:
Open a Konsole
sudo -i
mount /dev/disk/by-uuid/xxxxxxxxxxxxxxx /backup/ (If remote storage is needed)
mount /dev/disk/by-uuid/xxxxxxxxxxxxxxy /mnt
btrfs su snapshot -r /mnt/@ /mnt/snapshots/@YYYYMMDD
btrfs su snapshot -r /mnt/@home /mnt/snapshots/@homeYYYYMMDD
If backups are all that is wanted jump to the “Exit” steps
Sending snapshots to a remote storage device
btrfs send /mnt/snapshots/@YYYYMMDD | btrfs receive /backup/
btrfs send /mnt/snapshots/@homeYYYYMMDD | btrfs receive /backup/
sync
Unmount backup and mnt
umount /backup (If backup was mounted)
umount /mnt
Rolling back to a specific snapshot:
Open a Konsole and issue
sudo -i
mount /dev/disk/by-uuid/xxxxxxxxxxxxxxxxxx /mnt
mv /mnt/@ /mnt/@old (the system will continue to use @ as @old)
sync
mv /mnt/@home /mnt/@homeold (the system will continue to use @home as @homeold)
sync
btrfs subvol snapshot /mnt/snapshots/@homeYYYYMMDD /mnt/@home
sync
btrfs subvol snapshot /mnt/snapshots/@YYYYMMDD /mnt/@
sync
umount /backup
umount /mnt
Exit root and Konsole
exit
exit
Reboot the computer (If rolling back, otherwise it is unnecessary)
Reboot
If reboot is successful then mount /mnt and:
btrfs subvol delete -c /mnt/@homeold"
btrfs subvol delete -C /mnt/@old
or, mv them to /mnt/snapshots with new names.
One important point: always use UUID's to mount the partitions to /mnt or /backup or what ever. The reason is that what /dev/sdX points to can change as one adds or removes storage devices but UUID's are ALWAYS bound to a specific storage device. My UUID's are:
Code::~$ vdir /dev/disk/by-uuid/ total 0 lrwxrwxrwx 1 root root 10 Jun 3 08:44 [B]17f4fe91-5cbc-46f6-9577-10aa173ac5f6[/B] -> ../../sdb1 lrwxrwxrwx 1 root root 10 Jun 3 08:44 [B]47a4794f-b168-4c61-9d0c-cc22f4880116[/B] -> ../../sda1 lrwxrwxrwx 1 root root 10 Jun 3 08:44 [B]b3131abd-c58d-4f32-9270-41815b72b203[/B] -> ../../sdc1
In root you can assign those UUID's to env variables, say "hda" and "hdb", etc., then append them to the appropriate commands as needed.
EDIT:
I noticed that Oshunluver posted a comment to your question. As you can tell, his approach to making snapshots is different from mine. I don't modify where @ and @home are located and I don't modify fstab. He shows you how flexible a Btrfs system can be. Be advised that Oshunluver knows Btrfs better than I do. My approach will work with minimum modification to the system, just the addition of the snapshots folder to /mnt.
Later we'll discuss using the "-f" switch in the btrfs send command, and perhaps sending to storage devices on remote Internet servers.Last edited by GreyGeek; Jun 03, 2018, 08:34 AM."A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
– John F. Kennedy, February 26, 1962.
- Top
- Bottom
-
Originally posted by oldgeek View PostI have very recently installed Kubuntu 18.04 on a 500gb HD, with no Windows or other extraneous objects. I have made my first snapshot following instructions but have no idea what I am doing. What I need is a list of btrfs instructions for taking snapshots on a regular basis. Plus I'm a little confused--what happens to those snapshots? Where do they reside? How much memory is involved? If I am to get the most benefit out of my new file system I have a lot to learn.
**Pro Tip**: all btrfs commands can be shortened to their least common characters. Instead of "btrfs subvolume snapshot" you can type "btrfs su sn".
The purpose of a snapshots are twofold:
1. You can revert to an earlier state;For example, you decide to try out a new program or upgrade you system with a ton of new package updates. Before you install or upgrade, you take a snapshot of @ and name it @save. You install or upgrade and all hell breaks loose as the program installation or upgrade wreaks havoc on your system. No problem, just go to /subol and rename @ to @bad and rename your snapshot @save to @ and reboot. You're right back where you started. This can be done with your @home too.
How often you take a snapshot and how long to keep them will depend on your activity level. If you rarely make changes and don't update your system daily, there probably no need to snapshot daily. It's a good practice to snapshot before any package upgrades just in case the results are bad, and it takes almost no space (more on that below).
2. You can make backups.BTRFS lets you copy an entire subvolume by either "sending" it to another btrfs file system or making a backup file that can be restored later if needed. The difference between a snapshot and a backup is the snapshot must remain on the same file system that it was taken on. A snapshot becomes a backup when you move it off the source file system.
Backups begin with a read-only snapshot, which is done by adding "-r" to the snapshot command, so the above snapshot command becomes "sudo btrfs su sn -r /subvol/@ /subvol/@save". Read-only status is required for a backup to prevent data corruption. Remember, all of this and most btrfs commands can be done will the file system is mounted and still in use.
For example, if you have two drives in your system that both contain a btrfs file system. You can take a read-only snapshot of your home on drive A and send it to drive B. Assuming your drive B btrfs file system is mounted as "/backups" the commands to do this are:
sudo -i
btrfs su sn -r /subvol/@home /subvol/@home_backup1
btrfs send /subvol/@home_backup1 | btrfs receive /backups/
exit
You now have a full exact copy of @home on drive B. You can also "send" and "receive" from a file which is useful of you want to backup to a thumb drive, but don't want to format it with btrfs.
To your other questions: A snapshot takes zero drive space (I assume drive space is what you meant by "memory") initially. As you make changes to the source subvolume (the one you took a snapshot of), the previous state of that subvolume is "saved" in the snapshot and it grows accordingly. If you delete a file from the source subvolume, it still exists in the snapshot. If you add a file to the source subvolume, it does not appear in the snapshot. If you edit a file in the source subvolume, the copy of the file in the snapshot is not edited.
Here's a test for you to experience some of this in action. Open konsole and do these commands (substitute your btrfs partition device name for sda2);
sudo mkdir /subvol
sudo mount /dev/sda2 /subvol
ls /subvol
sudo btrfs su sn /subvol/@ /subvol/@snap1
ls /subvol
Assuming all went correctly, this should reveal the root level of your btrfs file system in /subvol. The first "ls" command should show "@ @home" and the second "@ @home @snap1".
Now let's edit fstab as I suggested above. Hit "ALT-F2" which will open the krunner bar at the top of your screen. In it type "kate /etc/fstab" and this will open your fstab file in the editor. It will have a lines that look something like these:
Code:[FONT=monospace][COLOR=#000000]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@ 0 1 [/COLOR][/FONT][COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@home 0 2 [/FONT][/COLOR][COLOR=#000000][FONT=monospace]UUID=[/FONT][/COLOR][FONT=monospace][COLOR=#000000]68bf458c-1133-4f4c-8b60-1d4ad15db901 [/COLOR][/FONT][COLOR=#000000][FONT=monospace] / swap sw 0 0[/FONT][/COLOR][FONT=monospace][COLOR=#000000] [/COLOR][/FONT]
Code:[FONT=monospace][COLOR=#000000]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@ 0 1 [/COLOR][/FONT][COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@home 0 2 [/FONT][/COLOR][COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@ 0 1[/FONT][/COLOR][COLOR=#000000][FONT=monospace] [/FONT][/COLOR][COLOR=#000000][FONT=monospace]UUID=[/FONT][/COLOR][FONT=monospace][COLOR=#000000]68bf458c-1133-4f4c-8b60-1d4ad15db901 [/COLOR][/FONT][COLOR=#000000][FONT=monospace] / swap sw 0 0[/FONT][/COLOR][FONT=monospace][COLOR=#000000] [/COLOR][/FONT]
Code:[FONT=monospace][COLOR=#000000]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@ 0 1 [/COLOR][/FONT][COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults,subvol=@home 0 2 [/FONT][/COLOR][COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs defaults 0 0[/FONT][/COLOR][COLOR=#000000][FONT=monospace] [/FONT][/COLOR][COLOR=#000000][FONT=monospace]UUID=[/FONT][/COLOR][FONT=monospace][COLOR=#000000]68bf458c-1133-4f4c-8b60-1d4ad15db901 [/COLOR][/FONT][COLOR=#000000][FONT=monospace] / swap sw 0 0[/FONT][/COLOR]
cat /etc/fstab
This will print out the fstab file and show you the edits you just did. Now type this:
cat /subvol/@snap1/etc/fstab
This will show you fstab again, but your edits will not be there. The snapshot saved the original file.
Of course, you can use any folder name you like if /subvol isn't to your liking. While you're editing fstab, I suggest you add these options to your btrfs mounts instead of defaults:
Code:[FONT=monospace][COLOR=#000000]rw,noatime,space_cache,compress=lzo,autodefrag[/COLOR][/FONT]
Code:[COLOR=#000000][FONT=monospace]UUID=e5e1be14-a10e-4188-b511-a610bea9e3a0 / btrfs [/FONT][/COLOR][COLOR=#000000][FONT=monospace]rw,noatime,space_cache,compress=lzo,autodefrag[/FONT][/COLOR][COLOR=#000000][FONT=monospace],subvol=@ 0 1[/FONT][/COLOR]
- Top
- Bottom
Comment
-
Originally posted by oldgeek View PostThanks for the advice. Now I need to try things out and get familiar with the commands."A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
– John F. Kennedy, February 26, 1962.
- Top
- Bottom
Comment
-
For me a very useful btrfs command is... cd.
In oshunluvr's example, where he's mounted the top of the btrfs volume on "/subvol" and made a snapshot of the root subvolume @ into @snap1, let's say on Sunday, then on MondayCode:$ cd /subvol/@snap1/etc $ ls
Regards, John Little
- Top
- Bottom
Comment
-
Originally posted by jlittle View PostFor me a very useful btrfs command is... cd.
In oshunluvr's example, where he's mounted the top of the btrfs volume on "/subvol" and made a snapshot of the root subvolume @ into @snap1, let's say on Sunday, then on MondayCode:$ cd /subvol/@snap1/etc $ ls
- Top
- Bottom
Comment
-
Originally posted by jlittle View Post... Note that snapshots every hour would not use much more space; only differences are recorded.
Btrfs has a mechanism for incremental backups.
This is a description for the OP and as a source document within KFN:
************************************************** *************
Initial Bootstrapping
We will need to create a read-only snapshot of the volume that serves as the reference for the first backup. I will call this subvolume BACKUP. The subvolume is read-only because "btrfs send" requires read-only subvolumes to operate on. NB: there is currently an issue that the snapshots to be used with "btrfs send" must be physically on the disk, or you may receive a "stale NFS file handle" error. This is accomplished by "sync" after the snapshot:
btrfs subvolume snapshot -r /home /home/BACKUP
sync
Once created, we can distribute the initial copy into existing directory or subvolume /backup/home.
(Note: to take a snapshot of /home the user has to be root. The snapshot "BACKUP" is stored inside /home as /home/BACKUP. I do not recommend this because if, for some reason, /home becomes inaccessible then so /home/BACKUP is inaccessible. How, then, can it be used to restore /home?)
The choice of "BACKUP" as a name is optional. All that is required is that the destination subvolume be an external Btrfs pool, say on another mounted drive, or another server, or in the cloud. Remember, a snapshot is not truly a "backup" if it is NOT located on another HD and at another site. Keeping all your snapshots on your main drive is worthless if that drive dies. Having copies on a 2nd or 3rd internal HD allows for recovery from a damaged drive. Having snapshots on a removable USB HD allows one to move the snapshot off site, so if the laptop is stolen or destroyed the data is still safe and the user's computing environment can be easily restored. I have three HD's mounted inside my Acer V3-771G laptop. I mount sda1 as /mnt (the <ROOT_FS>), sdb1 as /media and sdc1 as /backup. I also have a USB WD 350GB USB Passport drive that is off site. It contains a snapshot of everything I am unwilling to lose. All the rest I can reconstruct. IF your total system size is less than, say, 128GB then a USB stick will suffice, in fact, a couple of them. Note: you cannot us cp to copy a snapshot from one HD to another. That is what "btrfs send & receive" are for.
The subvolume appears as /backup/home/BACKUP:
btrfs send /home/BACKUP | btrfs receive /backup/home
Code:[B]btrfs send[/B] [-ve] [-p [I]<parent>[/I]] [-c [I]<clone-src>[/I]] [-f [I]<outfile>[/I]] [I]<subvol>[/I] [[I]<subvol>[/I]...] -e if sending multiple subvolumes at once, use the new format and omit the [I]end cmd[/I] marker in the stream separating the subvolumes -p [I]<parent>[/I] send an incremental stream from [I]parent[/I] to [I]subvol[/I] -c [I]<clone-src>[/I] use this snapshot as a clone source for an incremental send (multiple allowed) -f [I]<outfile>[/I] output is normally written to standard outout so it can be eg. piped to receive, use this option to write it to a file (on the destination)
Bootstrapping is now done. The subvolume /home/BACKUP is kept around to serve as local reference for the data that has been backed up, and it is needed for constructing the incremental backup for the next step.
Incremental Operation
During incremental backup, we make a new snapshot:
btrfs subvolume snapshot -r /home /home/BACKUP-new
sync
We can now send the difference between the old and new backup to the backup volume:
btrfs send -p /home/BACKUP /home/BACKUP-new | btrfs receive /backup/home
Once this command completes, we should have these 4 subvolumes:
/home/BACKUP,
/home/BACKUP-new,
/backup/home/BACKUP and
/backup/home/BACKUP-new.
We will now need to migrate the new backup as the old one, and do something for the old one. We could keep it around, maybe timestamped with the date of that backup, or just straight out delete it. Here, I am deleting it:
btrfs subvolume delete /home/BACKUP
mv /home/BACKUP-new /home/BACKUP
btrfs subvolume delete /backup/home/BACKUP
mv /backup/home/BACKUP-new /backup/home/BACKUP
(NOTICE: while you can use mv to move or rename a snapshot you CANNOT use rm to delete a snapshot!)
But for instance, if you did want to keep a history of backups, perhaps you would snapshot one of the snapshot directories with something like:
btrfs subvolume snapshot -r /backup/home/BACKUP /backup/home.$(date +%Y-%m-%d)
An addition note: this tutorial refers to a single snapshot taken of @home and stored at /backup/home.
In Ubuntu/Kubuntu/Neon there are two base subvolumes:
@
@home
which are mounted in fstab as / and /home.
You can see them by mounting your system drive to /mnt as root, and doing a directory listing.
vdir /mnt
@
@home
If you create a subdirectory there, say "snapshots", the listing would look like:
@
@home
snapshots
That subdirectory is where I store all my snapshots.
btrfs subvolume snapshot -r /mnt/@ /mnt/snapshots/@_$(date +%Y-%m-%d)
btrfs subvolume snapshot -r /mnt/@home /mnt/snapshots/@home_$(date +%Y-%m-%d)
They are not under @ or @home. They are under /mnt, the <ROOT_FS> level. To be inaccessible the HD itself would have to fail.
I send my archival snapshots to my sdc HD, which is mounted as /backup
btrfs send /mnt/snapshots/@_YYYYMMDD | btrfs receive /backup
btrfs send /mnt/snapshots/@home_YYYYMMDD | btrfs receive /backup
Because many apps store files in both / and /home it is advisable to take snapshots of both @ and @home. Since snapshots take a fraction of a second they would be separated in time by only a few seconds as the second command is typed in or recalled from history. Thus, they are essentially at the "same time".
This concludes the incremental backup step.
************************************************** ***********
All incremental backup really does is create successive snapshots that contain what the base snapshot contains plus any recent changes since that backup. The previous increment becomes the current base for a new increment. IF done on a daily basis, or at noon and evening if data changes are heavy, then rolling back can be very selective. However, the incremental changes are added to the base, so the send & receive command still takes an amount of time proportional to the size of the snapshot. More data, more time. Fortunately, while the send & receive is being done the system can continue to be used with little degradation of performance. And, at no time is the system required to shut down, so production does not have to cease while backing up is being done.
OpenSUSE which, unlike Ubuntu, defaults to Btrfs, creates several initial system subvolumes during installation. They include:
linux-s7mu:~ # btrfs subvolume list /
ID 257 gen 33 top level 5 path @
ID 258 gen 62 top level 257 path @/.snapshots
ID 259 gen 155 top level 258 path @/.snapshots/1/snapshot
ID 260 gen 61 top level 257 path @/boot/grub2/i386-pc
ID 261 gen 16 top level 257 path @/boot/grub2/x86_64-efi
ID 262 gen 43 top level 257 path @/opt
ID 263 gen 47 top level 257 path @/srv
ID 264 gen 154 top level 257 path @/tmp
ID 265 gen 66 top level 257 path @/usr/local
ID 266 gen 155 top level 257 path @/var/cache
ID 267 gen 36 top level 257 path @/var/crash
ID 268 gen 24 top level 257 path @/var/lib/libvirt/images
ID 269 gen 65 top level 257 path @/var/lib/machines
ID 270 gen 25 top level 257 path @/var/lib/mailman
ID 271 gen 27 top level 257 path @/var/lib/mariadb
ID 272 gen 28 top level 257 path @/var/lib/mysql
ID 273 gen 28 top level 257 path @/var/lib/named
ID 274 gen 30 top level 257 path @/var/lib/pgsql
ID 275 gen 155 top level 257 path @/var/log
ID 276 gen 36 top level 257 path @/var/opt
ID 277 gen 155 top level 257 path @/var/spool
ID 278 gen 154 top level 257 path @/var/tmp
ID 283 gen 61 top level 258 path @/.snapshots/2/snapshot
With Ubuntu/Kubuntu/Neon there are just two top level subvolumes: @ and @home. Note one thing: if you mount your system on /mnt and browse through /mnt/@ you will see "/home" listed inside. Browsing into /home you will observe that it is EMPTY! That is because @home isn't "inside" @, it is a separate subvolume and "/home" is just a place holder in @.
IF I were to create the subvolume @data at /mnt/@data and then mount it in fstab as /data, then it would appear in @ as /data also, but would be empty. @data is at the <ROOT_FS> level. Every subvolume is separate from the others and needs to be snapshotted and backed up separately.Last edited by GreyGeek; Jun 04, 2018, 10:59 AM."A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
– John F. Kennedy, February 26, 1962.
- Top
- Bottom
Comment
-
Well, like I used to tell my Physics students: "Everything is easy once you know how!""A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
– John F. Kennedy, February 26, 1962.
- Top
- Bottom
Comment
-
Originally posted by SpecialEd View PostHaving trouble with the Btrfs man pages:
So, do you have the btrfs-progs package? Or did you get btrfs some other way? Maybe you're not using an Ubuntu-derived distro, where the packaging is different?
If /usr/share/man/man8/btrfs.8.gz is there, I would check out the man set up. Are there any environment variables affecting man? (They mostly start with "MAN".) Some alias or function for man?
Anyway, btrfs wiki has the manpages generated from git, you can view them there.Regards, John Little
- Top
- Bottom
Comment
Comment