Announcement

**oshunluvr** · Feb 12, 2011, 01:10 PM

Re: BTRFS Testing and use

@dibl:

Today I created a directory with 24,850 files in 520 sub-folders, total size 3.8GiB. I used it to re-run the "Small Test" case. (The "large file" test is what it is -- no need to re-do that one). Here are the results:

BTRFS Copy Speed Test

Without space_cache and compress options

Large File From To Time
1 ext4 btrfs 1:33
2 btrfs ext4 3:54

Small Files
1 ext4 btrfs 1:40
2 btrfs ext4 2:43

With space_cache and compress options

Large File From To Time
1 ext4 btrfs 1:52
2 btrfs ext4 3:48

Small Files
1 ext4 btrfs 1:26
2 btrfs ext4 2:27

Now it is possible to discern a significant performance difference. I re-ran the "small files" test twice -- I think this is real.

**dibl** · Feb 12, 2011, 01:39 PM

Re: BTRFS Testing and use

Thanks oshunluvr.

The "large file" for this test is a single 16GiB video, .mpg format. As stated above, I created a directory of 24,850 files in 520 subfolders, total size of 3.8GiB, for the "small files" test. You can do the math -- these are all very small html and .png files. The drives involved are very fast already -- the ext4 filesystem is on a PCI-e bus SSD, and the btrfs fileystem is on a pair of WD1002FAEX SATA 6GiB/s drives on a Marvell 88SE9123 SATA-3 controller.

My tentative conclusion is that btrfs file operations won't be noticeably faster on transfers of very large files, when the space_cache and compress options are enabled. However, transfers of many small files appear to be roughly 10% faster on a btrfs filesystem with those two options enabled (2.6.37 or later kernel required, and the mount options don't take effect until you reboot the system).

I don't have the ability to run transfers between two separate btrfs filesystems -- I only have the single 2-disk btrfs filesystem. Btrfs is still in development, and we know from experience that "tweaking" the code for performance is normally the last developmental activity prior to release for general use, so it probably is not as fast today as it ultimately will be.

Truly geek news .... 8)

**oshunluvr** · Mar 28, 2011, 08:13 PM

Re: BTRFS Testing and use

Ok - I finally got my testing done. Please remember this is very unscientific.

For my testbed I used my daily desktop Kubuntu 10.04 with 2.6.32-30-generic kernel. This is important to note because there are significant upgrades to BTRFS with the .38 kernel and the utilities changed - more options available that speed things up. SInce I'm using .32 I don't get those upgrades.

For partitions I used 2 sets of 4x6gb partitions on 4 exactly the same hard drives. For reiserfs and for ext4 I first create RAID0 devices using mdadm. For btrfs I used the btrfs-raid option.

For all partitions I used only the default options - no special tweaks during creation or mounting. This was mostly a function of my time required to research all the choices.

I used a single 3.0gb large avi file for the large file test and for the many files test I used a folder containing 13.2GB 7740 files 93 subfolders - mostly jpg's. I ran a timed copy command in a terminal, didn't use the computer for anything else during the tests, deleted the target after each copy (so overwrite lag isn't a factor) and ran each test six times - throwing out the largest of each set and then averaging the remaining five.

All timings came out with real, user, and sys times

Large Many
file test files test
reiserfs 13.415 real 186.944
0.057 user 0.247
10.657 sys 75.300

ext4 11.254 real 291.344
0.030 user 0.287
6.443 sys 45.160

btrfs 14.891 real 160.022
0.037 user 0.247
4.703 sys 36.330

So clearly ext4 is better on the large file but remember no special settings were used and ext4 has plenty of options. One thing I noticed was the CPU time of reiserfs was way higher than the others so even though it was a bit faster in partical use, it slower other tasks down considerably. btrfs won out handily on the many files test.

I am preparing to make the change to 11.04 and along with that I am moving to btrfs. Even if the results were different reiserfs is practically dead and ext4 is a temporary fs - btrfs is the future...

**dibl** · Mar 28, 2011, 08:55 PM

Re: BTRFS Testing and use

Good job -- thanks for posting it!

**toad** · Mar 29, 2011, 12:48 AM

Re: BTRFS Testing and use

Interesting discussion. The only thing that keeps me away from btrfs is the lack of an fsck tool. Aren't you worried that there may be a problem?

Having said that, from what I read by those using btrfs already it is a blinder.

**dibl** · Mar 29, 2011, 06:13 AM

Re: BTRFS Testing and use

As it says in the wiki "btrfs is stable on a stable machine" -- my experience indicates that is true. My installation (pair of WD1000s) is on a big tower that is running on a UPS, and I shut it down when I'm gone from the house all day. As long as you don't suffer a hard crash, and keep all data backed up, I don't think anything bad is likely to happen. I wouldn' t advise btrfs on a laptop, for example.

**oshunluvr** · Mar 29, 2011, 06:55 AM

Re: BTRFS Testing and use

Supposedly the fsck tool is "almost ready." I the meantime having a backup will suffice. With the cost of hard drives, there is no reason to not have space for a backup.

I agree with dibl - my system is similar to his although I leave my computer running 24/7.

I really can't remember the last time I needed to run a filesystem check that wasn't related to my own actions.

**toad** · Mar 29, 2011, 07:07 AM

Re: BTRFS Testing and use

Originally posted by oshunluvr

Supposedly the fsck tool is "almost ready."

Ah, I didn't know (checked some time last week). I'll keep an eye on it.

**oshunluvr** · Mar 31, 2011, 07:20 AM

Re: BTRFS Testing and use

I've been trying to install the Natty Daily Current to a btrfs raid0 4 partition with no luck.

It occurs to me that although btrfs has the RAID0,1,10 options that the downside of RAID0 might outweigh any upside.

From what I can find no one has reported anything re. using RAID0 vs. regular btrfs setup.

I think that things like snapshots and adding/subtraction devices I might be better off not selecting RAID0 and sticking to LZO compression.

It appears this will give me better performance than the 4 device RAID0 resierfs I'm replacing anyway.

Opinions?

**dibl** · Mar 31, 2011, 08:49 AM

Re: BTRFS Testing and use

I installed it with default values, based on this from the "multiple devices" section of the btrfs wiki:

By default, metadata will be mirrored across two devices and data will be striped across all of the devices present.

That sounded like a reasonable configuration for my main data storage, which amounts to some 450GB. So, addressing my pair of WD1002FAEX drives, I did something like

Code:

sudo mkfs.btrfs /dev/sdd /dev/sde

You have to remember that btrfs is really a different concept than what we are use to thinking about, when we say "RAID": https://btrfs.wiki.kernel.org/index.php/SysadminGuide

So bearing in mind how it actually works, when I check my btrfs filesystem RAID status, it shows the "data" and "metadata" just as per the wiki:

Code:

root@aptosidbox:/home/don# btrfs filesystem df /mnt/DATA
Data, RAID0: total=484.00GB, used=445.07GB
Data: total=8.00MB, used=52.00KB
System, RAID1: total=8.00MB, used=40.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.62GB, used=634.77MB
Metadata: total=8.00MB, used=0.00

I dunno whether that sheds any light on your question ....

Also, AFAIK, Grub still won't boot a btrfs filesystem, so /boot will have to be elsewhere on ext2/3/4.

**oshunluvr** · Apr 01, 2011, 07:49 AM

Re: BTRFS Testing and use

There really isn't much commentary regarding the RAID options for btrfs. It appears the only difference is how the metadata is treated, which again may not be worth the bother.

Roger on the /boot. Since I've been using RAID for a couple of years I almost do that by reflex anyway... https://launchpad.net/btrfs-grub2

**oshunluvr** · May 01, 2011, 10:00 AM

Re: BTRFS Testing and use New Install: 11.04 using btrfs

I did two installs for testing.

First I installed to a single partition using the ALT installer disk. This worked fine although it was incredibly slow. I really don't know how long it took because I left, but I know it was more than two hours.

I then attempted to use the "device add" function of btrfs - no joy. I got a general ERROR and no other info. I attempted this both while using the partition and when not using it (from the second install). At this point I would say this function doesn't work. An interesting thing that happened suiring this install was since I did not specify a separate /home partition, the installer automatically created a subvolume for / and one for /home with a separate mount line in fstab. This allows you to easily backup and/or make a "snapshot" of your /home subvolume! What a great function!

I did the second install from the LiveCD version to a brtfs four partition array using the "-m raid0" switch which spreads metadata (rather than duplicating) across all disks. This time I also used a separate /home partition created with the metadata on all devices also across four partitions.

This install also created subvolumes for root and home although they are on their own partitions. Again, genius - since you can later add more subvolumes if you desire.

I haven't tweaked mount options yet or done any speed testing but I believe the btrfs project is on the right path and will be THE disk format of choice for all distros in very short order.

**oshunluvr** · Aug 08, 2011, 11:33 PM

Re: BTRFS Testing and use

Time for a small update: I've been using btrfs as my primary filesystem for five "devices" since June. So far (knocking on wood...) no issues at all.

All data backed up, of course.

This week I decided to remove one of the four hard drives on my desktop computer to use it in a DVR. So I have had to delete three partitions from three btrfs devices in order to clear this drive of data.

I did the first delete from a rarely used install partition - no issues.

As I type I am deleting a partition from my /home btrfs device. This device is 240gb total in size 60gb X 4 partitions. I am doing this while logged in and running from my home which is on this device! Amazing. I can tell the system is busy because of an occasional lag, but it's brief and not often.

Next will be removing the last partition from my install which I will also do while running from it. As a long-time hard drive abuser I am totally in love with this filesystem. I can't wait until it's the default.

**dibl** · Aug 09, 2011, 07:10 AM

Re: BTRFS Testing and use

Thanks for the update. Here, my btrfs filesystem has been 100% free of any cause for concern -- nada. Earlier this summer I started recording my collection of vinyl record albums, so my filesystem usage now looks like this:

Code:

root@aptosidbox:/home/don# btrfs filesystem df /mnt/DATA
Data, RAID0: total=762.00GB, used=638.96GB
Data: total=8.00MB, used=7.73MB
System, RAID1: total=8.00MB, used=64.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=11.88GB, used=1.16GB
Metadata: total=8.00MB, used=0.00

I haven't looked lately to see if there is any news about the fsck tool -- have you seen anything about that?

BTW, with the increase in amount of data, backup has become a little more challenging. I decided to try a 7z compressed tar backup, which worked perfectly after I figured out how to do it, but it took 36 hours due to the heavy compression options that I used.

With a lot of music and video content, which does not compress much, I think maybe I'll just have to spring for more/bigger backup drives and stick with "cp" as my backup tool.

Announcement

BTRFS Testing and use

BTRFS Testing and use

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment