fsck.btrfs in MM? Or not?

[fermier]

Greetings,

Could someone kindly please unconfuse me...?

I see several statements (from what used to be reputable sources... sigh) that fsck.btrfs does not exist for MM and therefore if you have a btrfs system crash there is no way to recover. (I know, I know that btrfs should still be considered as experimental and I know there will be those out there that will tell you that they have been using it for a long time without any mishap. This is not something I would like to hear as an answer to my questions... :-) ) However, when I look in the btrfstools package for MM I see the entry for this program. So, does it exist or doesn't it? And while I am asking questions, does it really have the functionality at the moment to repair a damaged btrfs?

I have a very limited MM installation for the moment and I am not in a position to verify this, which explains my dum questions...

l8r

[dibl]

Re: fsck.btrfs in MM? Or not?

I've been seriously thinking about an experimental installation of MM on a btrfs partition, just to see how it goes, and whether I can observe differences from my ext4 installation. If no one else has the answer to your question, I'll probably know it on the weekend.

[fermier]

Re: fsck.btrfs in MM? Or not?

I've been seriously thinking about an experimental installation of MM on a btrfs partition, just to see how it goes, and whether I can observe differences from my ext4 installation. If no one else has the answer to your question, I'll probably know it on the weekend.

Thanks that would be considered most kind of you. I hope you would be able to simulate a crash as well to know whether any errors can be fixed and the system can be recovered.

There are several reports from others using btrfs and MM and they seem to be satisfied. Most were disappointed with not perceiving any increase in speed compared with ext4. I think the main attraction is really the flexibility using btrfs (at least for me). My idea was to use it in Raid1 mode and just adding more disks to the array as I need them. The documentation doesn't say that they need to be of the same size nor what happens if they are not, i.e. like the case of the existing software Raid1 where the smallest disk is the common denominator and the larger disks end up as wasted space. Is there a hidden "gotcha" involved in btrfs as well when using Raid1...?

For Raid0 this question does not seem relevant for I understood that everything is just striped together irrespective of their individual capacities. Maybe I should just give Raid10 a try for that matter as then I do not need to wonder...

I shall be awaiting with eager anticipation on what you find...

l8r

[dibl]

Re: fsck.btrfs in MM? Or not?

A little digging turned up this at: https://help.ubuntu.com/community/btrfs

"Stability"

Btrfs does not yet have a fsck tool that can fix errors. While Btrfs is stable, it is currently possible to corrupt a filesystem irrecoverably if your machine crashes or loses power. http://www.mail-archive.com/linux-bt.../msg05749.html

"The main concern against BTRFS is not stability, but the lack of a functional fsck utility." http://www.mail-archive.com/linux-bt.../msg06277.html

Chris Mason on August 26, 2010: We're still actively developing it. I don't have a release date planned yet but we should have betas coming out over the next few months. http://www.mail-archive.com/linux-bt.../msg05898.html

Canonical confirms btrfs won't happen until 11.04

robbie.w: I deferred this for completion in Natty, as we won't be able to complete all the remainin work for Maverick. https://blueprints.launchpad.net/ubu...-btrfs-support

So, I dunno. I guess I can install it and run it, but it doesn't sound like pulling the power cord to torture-test it would be a good idea, until they have released a fsck tool that can fix it. :P

[Snowhog]

Re: fsck.btrfs in MM? Or not?

"Stability"
While Btrfs is stable, it is currently possible to corrupt a filesystem irrecoverably if your machine crashes or loses power.

Which is the number one reason NOT to use such a file system; unless you truely want to live on the edge, and want to get pushed over it!

[fermier]

Re: fsck.btrfs in MM? Or not?

A little digging turned up this at: https://help.ubuntu.com/community/btrfs

"Stability"

Btrfs does not yet have a fsck tool that can fix errors. While Btrfs is stable, it is currently possible to corrupt a filesystem irrecoverably if your machine crashes or loses power. http://www.mail-archive.com/linux-bt.../msg05749.html

"The main concern against BTRFS is not stability, but the lack of a functional fsck utility." http://www.mail-archive.com/linux-bt.../msg06277.html

Chris Mason on August 26, 2010: We're still actively developing it. I don't have a release date planned yet but we should have betas coming out over the next few months. http://www.mail-archive.com/linux-bt.../msg05898.html

Canonical confirms btrfs won't happen until 11.04

robbie.w: I deferred this for completion in Natty, as we won't be able to complete all the remainin work for Maverick. https://blueprints.launchpad.net/ubu...-btrfs-support

So, I dunno. I guess I can install it and run it, but it doesn't sound like pulling the power cord to torture-test it would be a good idea, until they have released a fsck tool that can fix it. :P

Do I detect that I have succeeded in sowing confusion rather than in reaping reason...? Now you are starting to understand my question and confusion. I also read this reference and others, yet, when I look in the btrfstools package one sees that there is a check program. Very confusing indeed.

Ouf, I did not want to go in to the arguments to using or not using this FS and certainly not into discussing its stability and reliability for use on systems that need to be stable. I thought I made that clear. Nevertheless, thanks Snowhog for reminding us. I was talking about experimenting and I think so was Dibl, though he seems to have run scared... :-) Oh well, I guess I would need to reconfigure my inflexible system into something where I can actually test this in the manner that I think of eventually using it. ...

[dibl]

Re: fsck.btrfs in MM? Or not?

I was talking about experimenting and I think so was Dibl, though he seems to have run scared... :-)

What -- me scared (dibl swigs a whiskey and cracks knuckles)? Nah -- 10.10 goes on a btrfs partition tomorrow!


If there's a btrfs fsck utility on the Live CD, I'll run it and see what happens. I'm just saying, the first thing I do after I've got my Conky running on it is probably not gonna be to pull the power cord and see what happens. I'll save that maneuver for at least a week.

[fermier]

Re: fsck.btrfs in MM? Or not?

I'll save that maneuver for at least a week.

Mmmm, scrounging up courage or the right level of whiskey? :-)

Anyway, to explain the confusion regarding running brtfs on multiple disks/partitions of different sizes and using a "raid1" behaviour:

http://kerneltrap.org/mailarchive/li...6830903/thread

[dibl]

Re: fsck.btrfs in MM? Or not?

My intention was to simply change the existing partition from ext4 to btrfs and re-install Kubuntu 10.10. Using the Alternate installation CD, I soon learned that selecting "btrfs" leads to discovery of the requirement for a separate /boot partition to use it.

So, I booted my handy Parted Magic Live USB stick, and changed the first of my four primary partitions to an "extended" type, and then within that 32GB of space, I made a 256MB ext4 partition, and let the rest of it stay unallocated (Parted Magic ver. 4.4 does not offer btrfs as a filesystem type).

Back to the Kubuntu 10.10 Alternate CD, and this time I was able to step through the installation, selecting the little partition for /boot and the large one for the rest of the system, with a btrfs filesystem type. I watched as the installer did the necessary hardware and networking checks, decided that I need 794 files, and proceeded to download and install the system. It took 47 minutes from booting the Alternate CD to the first boot of the new system.

So, up came Plymouth and a 1280x1024 display, two virtual desktops, using the free nouveau driver for my GTX260. It looked fine, although of course there was no ability to use 3D or other advanced desktop effects. It came up very quietly -- a check in System Settings > Multimedia confirmed that it did not automatically find my Intel HDA integrated audio, or configure Phonon to use it.

I was pleased to see how many of my commonly-used packages were automatically installed, including Open Office.org and ALSA. The first order of business was to install the first round of the rest of the needed (by me) packages:

muon, gimp, conky, hddtemp, hwinfo, kinfocenter, nvclock, nano, chromium-browser, and maybe one or two more that I can't remember, and I've got my basic stuff, with which to start setting up the rest of my desktop environment.

So, I opened Konsole and ran updates -- there were not a lot, so I assume the Alternate install CD has been updated since the original 10.10 release, because I had observed many updates on my prior 10.10 system, since the formal version release several weeks ago.

I opened System > Additional Drivers and let it find my Nvidia card. I clicked "Accept" and waited -- it took over 5 minutes, even on my broadband cable connection, to download and configure the ver. 260.19.06 proprietary driver, which of course includes removing the nouveau driver and updating the initramfs. But it all worked as designed, which is good to see -- jockey-kde has always been a little "iffy" on my hardware, so I'm glad to see it work exactly as expected.

With the proprietary driver installed, a reboot was needed. As expected, the Plymouth screen now looked awful -- fragmented across my Samsung monitor, with Ubuntu brown spots here and there peeking through the blue. It's easily fixable -- see #2.d. on my FAQs -- I'll do that later. I opened nvidia-settings with "kdesudo", changed the deskop resolution to 1600x1200 and applied it, and used the "Save to X Configuration" button to write an /etc/X11/xorg.conf file. I looked at the file and it looks very normal for running my GTX260.

Then I opened System Settings > Window Behavior and changed the number of virtual desktops to 4, then went back and opened System Settings > Desktop Effects and made sure the cube effect and compositing were enabled, clicked "Apply" and voila I've got the *buntu version of a desktop cube (I still prefer the compiz version, but not enough to bother with it at this stage of setting up my work environment).

Installation of paman, padevchooser, and pavucontrol was needed to let me access my audio chip. Sure enough, in pavucontrol I found the output device "analog speakers" muted. With it unmuted, System Settings > Phonon > "Test" produced the expected tones.

On to the question of the day -- what about that btrfs performance? Just before I started the new installation, in my previous Kubuntu 10.10 system which was installed on an ext4 filesystem, I did the following little manuever:

- open a Konsole window
- copy a particular directory that lives on another hard disk drive, in an ext4 partition, to my home folder, using the "cp -r" command. This particular directory is actually the input and output files for a large web site. It contains 3,502 files in 93 sub-folders and is 404.5MiB in size (according to Dolphin).
- from the time I hit "Enter" to the time I got the prompt back, was 4 seconds according to the clock on the taskbar.
- then I deleted it with the "rm -rf" command, watching the clock, but I got the prompt back immediately, so whatever time was used in the background to actually delete everything was not visible to me.

So, with my new Kubuntu 10.10 on my new btrfs filesystem, I repeated the test.

- copy time was ... 25 seconds!
- delete time was 1 second.

I was so shocked at the copy time that I changed to a test location on an ext4 partition that is not on the drive where the target directory is, and repeated the copy test from ext4 to ext4 filesystems. Again it was less than 4 seconds. Again the complete deletion took 1 second.

Hmmmmm -- I dunno what to make of it. Perhaps there are special efficiencies when copying from one ext4 filesystem to another, that are not available when copying from ext4 to btrfs. I'll guess we'll have to continue to monitor the features and characteristics of btrfs, with regard to filesystem operations. I have not yet begun to explore the "pooled filesystem space" feature -- I would need more btrfs partitions to do that. Perhaps another day ....

That's what I know about Kubuntu on btrfs, so far. Haven't looked very far into how I'll fsck it, yet, but that's coming up soon.

EDIT: I used the Kubuntu 10.10 Live CD, but I couldn't find a partition manager on it (I know there should be one somewhere), so I installed GParted in the Livd CD session (ver. 0.6.2), opened it and browsed to the btrfs partition, highlighted it and ran "check", and all indications are that it worked correctly (see attached screenshot). I also noticed that GParted does not show that there is any space used in the btrfs partition -- I think there are still some details that need to be worked out, on that score. Also note that GParted as yet has no ability to make the btrfs filesystem -- you have to install the btrfs-tools package, and use the mkfs.btrfs command (see the manual).

After setting the Medibuntu repo in my sources, and installing all my multimedia and non-free font needs, I checked with df to see what the filesystem usage looks like. Here it is:

don@meerkat:~$ df -h
Filesystem      Size Used Avail Use% Mounted on
/dev/sda6       32G 4.1G  28G 13% /

ta-da!

[fermier]

Re: fsck.btrfs in MM? Or not?

EDIT: opened it and browsed to the btrfs partition, highlighted it and ran "check", and all indications are that it worked correctly (see attached screenshot). I also noticed that GParted does not show that there is any space used in the btrfs partition -- I think there are still some details that need to be worked out, on that score. Also note that GParted as yet has no ability to make the btrfs filesystem -- you have to install the btrfs-tools package, and use the mkfs.btrfs command (see the manual).

Thanks for this confirmation and also reporting on your other experiences. So the check does exist. I wonder then why so many sources say it doesn't... As far as booting is concerned, some say that Grub2 has been patched to be able to deal with booting a btrfs, but this was not my experience.

Concerning your long/slow copy, I must admit that seems rather suspicious. One may need to investigate. What about copying the same from one btrfs to another btrfs partition? I guess this would be a fair comparison, because copying from ext4 to btrfs may involve some CPU cycles... I guess now you are telling me that I need to configure my old system to use multiple disk systems so that I can test the different Raid mode configurations... Something to keep me out of mischief next weekend... ;-)

[dibl]

Re: fsck.btrfs in MM? Or not?

What about copying the same from one btrfs to another btrfs partition? I guess this would be a fair comparison, because copying from ext4 to btrfs may involve some CPU cycles...

Great minds think alike! Yesterday I backed up data and converted a partition on a different drive to btrfs, planning a re-run of the same test. So, this morning I ran it three times, and our hunches were right.

-- copy time was between 2 and 3 seconds -- I'd need a stopwatch to nail it exactly.
-- delete time was less than 1 second.

I guess now you are telling me that I need to configure my old system to use multiple disk systems so that I can test the different Raid mode configurations... Something to keep me out of mischief next weekend... ;-)

Yes, that's what you should do, and share the results and how to do it.

BTW, in case you didn't absorb it from your research, when you make a btrfs partition (either via installing the OS or using mkfs.btrfs), upon inspection you will learn that it has two UUIDs. For example, after my weekend adventures, my three hard drives now look like this:

don@meerkat:~$ sudo blkid
/dev/sda2: LABEL="MUSIC" UUID="1e908a65-7ddb-4a1f-9913-d5a82ddb3137" TYPE="ext4" 
/dev/sda3: LABEL="DOCS&PIX" UUID="bef7fc54-d120-4869-920c-edc64a214bae" TYPE="ext4" 
/dev/sda4: LABEL="VIDEOS" UUID="932854dc-d16b-4612-9bd7-37345d1def09" TYPE="ext4" 
/dev/sda5: UUID="ec17a5f9-d3b5-4332-97c5-af3074aa0904" TYPE="ext4" 
/dev/sda6: UUID="8cfacbec-2b8a-43e0-b499-7aaf721a4330" UUID_SUB="0ef90973-f1b1-43a1-94ed-c3e1e79636e6" TYPE="btrfs" 
/dev/sdb1: LABEL="DOCSPIXBAK" UUID="2b7ce29a-4550-4c6f-9171-8406d9688da7" TYPE="jfs" 
/dev/sdb2: LABEL="MUSICBAK" UUID="b69c6415-23e4-4e96-9f72-17d2d7736b93" TYPE="jfs" 
/dev/sdb3: LABEL="BTRFSTEST" UUID="0c423529-961e-4526-8016-c14da6f1a863" UUID_SUB="0e0252d9-b2ff-445c-9d61-3685c5ab411c" TYPE="btrfs" 
/dev/sdc1: LABEL="SIDUX" UUID="dfd5d21c-b8d7-4dd7-8d44-ac3146008354" TYPE="jfs" 
/dev/sdc2: UUID="8f03cf64-35dd-4d7f-b055-2d42aa291028" TYPE="swap" 
/dev/sdc3: LABEL="WHATEVER" UUID="a1934295-3ff7-4700-901a-637513a74cfa" TYPE="ext4"

So, when you want to set up a btrfs partition to mount in /etc/fstab, you need to use the first UUID. The second one is the default "sub-volume", about which I still have everything to learn.

[Snowhog]

Re: fsck.btrfs in MM? Or not?

The second one is the default "sub-volume", about which I still have everything to learn.

What is btrfs?

A subvolume is like a directory - it has a name, there's nothing on it when it is created, and it can hold files and other directories. There's at least one subvolume in every Btrfs filesystem, the "default" subvolume.

"Subvolumes (separate internal filesystem roots)"

Btrfs - Multiple Device Support

[dibl]

Re: fsck.btrfs in MM? Or not?

Yeah, that's "book learning". I'm talking about "hands-on figuring it out by doing it" learning.

[Snowhog]

Re: fsck.btrfs in MM? Or not?

Oh, the 'bumps, bruises, and scrapes' method of learn'n.

[dibl]

Re: fsck.btrfs in MM? Or not?

Yes -- skinned knuckles is the only way I've ever been able to learn anything useful. :P