Comparison of Ext4 and Btrfs

[oshunluvr]

Yeah, 1 chunk isn't much. I'm not coming up with other ideas at the moment - except maybe deleting the backup subvols and doing the send|receive again to see if you get different results.

[oshunluvr]

I did a test here to check my theory on the snapshot thing. I sent snapshot 1 and it consumed about 12GB, then snapshot 2 sent and used about the same. Then I deleted snapshot 2 and re-sent it as an incremental backup. That time it consumed about 300MB.

[oshunluvr]

The only thing that makes sense is there's a nested subvol/snapshot in there somewhere.

[GreyGeek]

The only thing that makes sense is there's a nested subvol/snapshot in there somewhere.

You saw the entire file hierarchy in the vdir listing! I like your idea of deleting the snapshots. Tomorrow I'm going to delete all of them on both drives and redo them, after I balance the chunks.

[mr_raider]

OK, did a test and some research. If you choose manual partitioning and btrfs, the system does indeed create a swapfile. However, btrfs currently cannot properly handle swapfiles so although it's created, it's not being used.

btrfs basically ignores it:

[FONT=monospace][COLOR=#54FF54][B]stuart@zestyVM[/B][/COLOR][COLOR=#000000]:[/COLOR][COLOR=#5454FF][B]~[/B][/COLOR][COLOR=#000000]$ free -m[/COLOR]
            total        used        free      shared  buff/cache   available
Mem:          1997         782         118           4        1096        1050 
Swap:            0           0           0
[/FONT]

There was a bug filed during alpha testing (I think) regarding this.

So currently; If you're proficient enough to want use btrfs, you either do it by manually partitioning and creating a swap partition, or use a different non-btrfs partition to hold the swap file, or don't use swap.

That would explain why my btrfs install in VM kept throwing memory errors and crashing. Good to know.

Sent from my ZTE A2017U using Tapatalk

[GreyGeek]

I deleted the snapshot subvolume and all the backups to get a clean /dev/sdb1 (/backup) Btrfs installation. Then I did a balance and a scrub.
After that I created new snapshot ro backups of @ and @home on /mnt and did a send & receive to /backup/snapshots.

Here is the usage data:

:~# btrfs fi usage /mnt
Overall:
Device size: 691.19GiB
Device allocated: 95.07GiB
Device unallocated: 596.12GiB
Device missing: 0.00B
Used: 92.44GiB
Free (estimated): 597.63GiB (min: 299.57GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 155.95MiB (used: 0.00B)

Data,single: Size:92.01GiB, Used:90.50GiB
/dev/sda1 92.01GiB

Metadata,DUP: Size:1.50GiB, Used:995.28MiB
/dev/sda1 3.00GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
/dev/sda1 64.00MiB

Unallocated:
/dev/sda1 596.12GiB


root@jerry-Aspire-V3-771:~# btrfs fi usage /backup
Overall:
Device size: 698.63GiB
Device allocated: 176.07GiB
Device unallocated: 522.56GiB
Device missing: 0.00B
Used: 174.51GiB
Free (estimated): 523.44GiB (min: 262.16GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 261.34MiB (used: 0.00B)

Data,single: Size:173.01GiB, Used:172.13GiB
/dev/sdb1 173.01GiB

Metadata,DUP: Size:1.50GiB, Used:1.19GiB
/dev/sdb1 3.00GiB

System,DUP: Size:32.00MiB, Used:48.00KiB
/dev/sdb1 64.00MiB

Unallocated:
/dev/sdb1 522.56GiB

Notice that /backup no longer has a copy of my home account directory. /mnt contains both the OS (@, @home) AND FOUR backup snapshots, two each of @ and @home. /mnt has only the four backup snapshots. Yet, /backup has only 523 Gb of free space and /mnt has 598 Gb. /backup is missing 75Gb of unaccounted for space.

My assumption was that the ro snapshot @home_bkup-2017-04-13, which I send & received to /backup/snapshots contains the full contents of my /home/jerry directory, unlike the ro @home_bkup-2014-04-13 on /mnt.

[oshunluvr]

So, just for grins, open the /backup folder and navigate to the install backup there, then open the /mnt folder inside the backup of @ and see if anything is in there.

[GreyGeek]

So, just for grins, open the /backup folder and navigate to the install backup there, then open the /mnt folder inside the backup of @ and see if anything is in there.

Mmm... clever idea!
However, there is no "@" or "@home" on /backup, as the mc listing in a post above shows.

[FONT=monospace][COLOR=#000000]:/backup/snapshots/@_bkup-2017-04-13# vdir[/COLOR]
total 16
drwxr-xr-x 1 root root    0 Mar 29 11:16 backup
drwxr-xr-x 1 root root 2550 Mar 28 21:58 bin
drwxr-xr-x 1 root root  680 Mar 27 14:25 boot
drwxr-xr-x 1 root root    0 Mar 24 14:45 cdrom
drwxr-xr-x 1 root root  894 Mar 16 10:18 dev
drwxr-xr-x 1 root root 4358 Apr 12 18:45 etc
drwxr-xr-x 1 root root    0 Mar 24 14:44 home
lrwxrwxrwx 1 root root   32 Mar 26 13:16 initrd.img -> boot/initrd.img-4.8.0-42-generic
lrwxrwxrwx 1 root root   32 Mar 24 15:41 initrd.img.old -> boot/initrd.img-4.4.0-67-generic
drwxr-xr-x 1 root root  538 Mar 28 21:58 lib
drwxr-xr-x 1 root root 1304 Mar 27 14:22 lib32
drwxr-xr-x 1 root root  112 Mar 28 21:58 lib64
drwxr-xr-x 1 root root   18 Apr 13 14:16 media
drwxr-xr-x 1 root root    0 Apr 11 22:32 mnt
drwxr-xr-x 1 root root   52 Mar 28 21:59 opt
drwxr-xr-x 1 root root    0 Apr 12  2016 proc
drwx------ 1 root root  202 Apr  5 19:06 root
drwxr-xr-x 1 root root  162 Mar 16 10:19 run
drwxr-xr-x 1 root root 3980 Mar 31 20:28 sbin
drwxr-xr-x 1 root root    0 Feb 24 07:24 snap
drwxr-xr-x 1 root root    0 Mar 16 10:05 srv
drwxr-xr-x 1 root root    0 Feb  5  2016 sys
drwxrwxrwt 1 root root  992 Apr 13 14:35 tmp
drwxr-xr-x 1 root root   80 Mar 27 14:22 usr
drwxr-xr-x 1 root root  108 Apr  3 15:34 var
lrwxrwxrwx 1 root root   29 Mar 26 13:16 vmlinuz -> boot/vmlinuz-4.8.0-42-generic
lrwxrwxrwx 1 root root   29 Mar 24 15:41 vmlinuz.old -> boot/vmlinuz-4.4.0-67-generic

[/FONT][FONT=monospace]
root@jerry-Aspire-V3-771:/backup/snapshots/@_bkup-2017-04-13# vdir /mnt
total 0
[/FONT]

but I don't see anything there.

[oshunluvr]

try

vdir /backup/snapshots/@_bkup-2017-04-13/backup

should be empty.

[GreyGeek]

try

vdir /backup/snapshots/@_bkup-2017-04-13/backup

should be empty.

It is.

[oshunluvr]

Well Jerry, I'm stumped

[GreyGeek]

Me too.
Going to continue to explore. I haven't installed natime yet, but if I'm going to do a lot of directory and file size comparisons I will have to do that.

[GreyGeek]

I just saw your find essay on Btrfs for noobs and you reminded me of something I did out of habit when I formatted my /dev/sdb drive -- I created a partition and didn't need to!

I'm going to blow away the partitions and give Btrfs /dev/sdb not /dev/sdb1, then redo the archival backups.
Thanks for the memory jog!

[oshunluvr]

LOL, ok glad it did some good. You weren't the target of that peice, but glad you found a tidbit in there.

Most of my desktop drives are partitioned for boot purposes but the server didn't need more than three bootable drives.

For awhile I didn't boot to my ssd's, using a stand alone grub partition on a hard drive instead. Last time I re-org'd my desktop storage the old hard drives on my desktop had reached 60,000+ power-on hours so I moved the grub partition to the ssd's.

[GreyGeek]

I finished the transformation. Here is a brief account.
I opened KPartition and deleted the sdb1 partition.
From within KPartition I could NOT format just /dev/sdb alone without creating sdb1, a partition.
So, I closed KPartition and opened a root Konsole.
Without attempting to mount /dev/sdb to /backup I issued
mkfs.btrfs /dev/sdb
It gave me an error, stating that /dev/sdb had a DOS partition on it.
So, I issued
mkfs.btrfs -f /dev/sdb
and it worked.

Then I mounted /dev/sdb to /backup
mount /dev/sdb /backup
and I also mounted /dev/sda1 to /mnt
mount /dev/sda1 /mnt

I then proceeded to delete the backup snapshots from /mnt and created fresh ro backups.

[FONT=monospace][B][COLOR=#000000]vdir /mnt/snapshots/[/COLOR][/B]
total 0
drwxr-xr-x 1 root root 256 Apr  8 16:53 @_bkup-2017-04-08
drwxr-xr-x 1 root root 256 Apr  8 16:53 @_bkup-2017-04-13
drwxr-xr-x 1 root root  10 Apr  8 16:54 @home_bkup-2017-04-13

root@jerry-Aspire-V3-771:~# [B]btrfs su delete -c /mnt/snapshots/@_bkup-2017-04-08 [/B]    
Delete subvolume (commit): '/mnt/snapshots/@_bkup-2017-04-08'

root@jerry-Aspire-V3-771:~# vdir /mnt/snapshots/                                
total 0
drwxr-xr-x 1 root root 256 Apr  8 16:53 @_bkup-2017-04-13
drwxr-xr-x 1 root root  10 Apr  8 16:54 @home_bkup-2017-04-13

root@jerry-Aspire-V3-771:~# [B]btrfs su delete -c /mnt/snapshots/@_bkup-2017-04-13[/B]
Delete subvolume (commit): '/mnt/snapshots/@_bkup-2017-04-13'

root@jerry-Aspire-V3-771:~#[B] btrfs su delete -c /mnt/snapshots/@home_bkup-2017-04-13[/B]
Delete subvolume (commit): '/mnt/snapshots/@home_bkup-2017-04-13'

root@jerry-Aspire-V3-771:~# [B]vdir /mnt/snapshots/[/B]
total 0

[/FONT]

Now create the new snapshots of @ and @home:

[FONT=monospace][COLOR=#000000]:~# [B]btrfs su snapshot -r /mnt/@ /mnt/snapshots/@_bkup-2017-04-14[/B][/COLOR]
Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@_bkup-2017-04-14'

root@jerry-Aspire-V3-771:~#[B] btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home_bkup-2017-04-14[/B]
Create a readonly snapshot of '/mnt/@home' in '/mnt/snapshots/@home_bkup-2017-04-14'

root@jerry-Aspire-V3-771:~# [B]vdir /mnt/snapshots/[/B]
total 0
drwxr-xr-x 1 root root 256 Apr  8 16:53 @_bkup-2017-04-14
drwxr-xr-x 1 root root  10 Apr  8 16:54 @home_bkup-2017-04-14
[/FONT]

Then I created a subvolume on /backup called "snapshots"

root@jerry-Aspire-V3-771:~# vdir /backup/
total 0
root@jerry-Aspire-V3-771:~# [B]btrfs su create /backup/snapshots[/B]
Create subvolume '/backup/snapshots'

Then send & receive the ro snapshots on /mnt/snapshots to /backup/snapshots

root@jerry-Aspire-V3-771:~# [B]btrfs send /mnt/snapshots/@_bkup-2017-04-14 | btrfs receive /backup/snapshots[/B]
At subvol /mnt/snapshots/@_bkup-2017-04-14
At subvol @_bkup-2017-04-14

root@jerry-Aspire-V3-771:~# [B]btrfs send /mnt/snapshots/@home_bkup-2017-04-14 | btrfs receive /backup/snapshots[/B]
At subvol /mnt/snapshots/@home_bkup-2017-04-14
At subvol @home_bkup-2017-04-14

Now, let's check the usage of the HD space:

[FONT=monospace][COLOR=#000000]:~# [B]btrfs fi usage /backup/[/B][/COLOR]
Overall:
    Device size:                 698.64GiB
    Device allocated:             88.02GiB
    Device unallocated:          610.61GiB
    Device missing:                  0.00B
    Used:                         86.63GiB
   [B] Free (estimated):            611.19GiB[/B]      (min: 305.88GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              130.73MiB      (used: 0.00B)

Data,single: Size:86.01GiB, Used:85.43GiB
   /dev/sdb       86.01GiB

Metadata,DUP: Size:1.00GiB, Used:615.12MiB
   /dev/sdb        2.00GiB

System,DUP: Size:8.00MiB, Used:16.00KiB
   /dev/sdb       16.00MiB

Unallocated:
   /dev/sdb      610.61GiB


root@jerry-Aspire-V3-771:~# [B]btrfs fi usage /mnt
Overall:[/B]
    Device size:                 691.19GiB
    Device allocated:             95.07GiB
    Device unallocated:          596.12GiB
    Device missing:                  0.00B
    Used:                         88.05GiB
    [B]Free (estimated):            601.16GiB[/B]      (min: 303.11GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              148.78MiB      (used: 0.00B)

Data,single: Size:92.01GiB, Used:86.96GiB
   /dev/sda1      92.01GiB

Metadata,DUP: Size:1.50GiB, Used:557.42MiB
   /dev/sda1       3.00GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
   /dev/sda1      64.00MiB

Unallocated:
   /dev/sda1     596.12GiB
[/FONT]

Now, THAT free space makes sense.
I have no clue as to what the "dos" partition was, since I deleted the only partition on the HD, sda1.