root on ZFS

[rjvbb]

Has anyone yet managed to run a(n) (K)Ubuntu 14.04 system off a ZFS root? The default GRUB utilities (*) don't handle a ZFS root pool properly (grub-probe fails to return the appropriate device) and the GRUB utilities in the zfsonlinux ppa stop with Raring.

I've had partial success using the GRUB utilities from the zfsonlinux Debian repository, only to find that the boot process stalls sometime the boot message about the mounting of encrypted disks (I have none). At that point, Ctrl-Alt-Del no longer allows a proper reboot: the reboot is started but gets stuck at "waiting for state".

*) grub-pc grub-pc-bin grub-common grub2-common

[oshunluvr]

AFAIK no one here uses ZFS. Several of us use BTRFS.

[rjvbb]

My test install, the one I cloned onto a ZFS pool is indeed on btrfs. Without wanting to start a flame war, I prefer ZFS; esp. the fact it can give redundancy on a single-disk pool (copies=2). OTOH btrfs's autodefrag is nice, but not a replacement for the increased protection through redundancy ... and the fact that ZFS is available on other platforms too. I doubt btrfs will ever be ported beyond Linux.

[oshunluvr]

No flame war required here. I looked into ZFS a while ago. But since I'm a 95% linux user and don't need or want Solaris (although I played with it a bit and have needed it in the past), BSD, or OS X, I could see no reason to struggle with making ZFS work in a linux environment. Since Ubuntu doesn't support it, it requires outside support - I just didn't think the effort was worth it since btrfs does 80-90% of what ZFS does and has (maybe had - I haven't looked at ZFS in a couple years) more features for desktop users. Personally, I don't see the benefit to having multiple copies of a file on a single disk, but it's easy enough to clone subvolumes. I must not get what use you have for that feature.

Most of the users here aren't using advanced disc layouts or doing high-end servers or production work. Most of the encounters I have here are laptop users trying to keep their "daily drivers" going. My earlier comment was just to point out that it's unlikely you'll get any response with ZFS questions here. Kubuntu isn't exactly the go-to distro for super-techie types.

[oshunluvr]

Since we're discussing it, from the ZFS wiki (emphasis added):

There are several reasons as to why it is better to rely solely on ZFS by using several independent disks and RAID-Z or mirroring. For example, a ZFS volume with RAID-0 volumes even with "copies=2" can be failure prone, as the RAID-0 volumes will fail in the event of any disk failures. Thus, storing data on RAID-0 with a ZFS volume and "copies=2" enabled doesn't increase data reliability, instead, it reduces it

I'm curious as to how/what you're using copies=2 for? Doesn't RAID1 accomplish the same thing?

[rjvbb]

copies=2 (or even 3) is what gives protection through redundancy when you only have a single drive - just as most desktop (= laptop...) users would have. I'm not really sure why it'd reduce reliability on RAID-0.
COW, checksumming and the associated features of filesystems like ZFS and btrfs are of little use against bitrot when you only have a single copy of each file, and thus cannot restore a file when corruption is detected. RAID1 cannot do that either, regardless of whether you use ZFS on top of it or another filesystem. Of course copies=N won't protect against complete disk failure, but (spinning) disks rarely go at once without warning. The only way to get something similar with btrfs is to cut up a single disk (or install partition) in 2 equal partitions, and do mirroring between those - but then you lose the possibility to control redundancy on a dataset level. Example: my /tmp is a dataset (subvol in btrfs terminology) with copies=1, with checksumming and syncing disabled, as a means to speed up operations as much as possible in a directory that's always in flux anyway. Other datasets that hold large collections of files (say for building kernels) have copies=1 to reduce disk use.

There's been a good article on next-generation filesystems on Ars Technica a short while ago, which points out these things (even though it focuses on btrfs and server-like set-ups with multiple drives).

I haven't played a lot with btrfs yet (because it's linux-specific) but until now I don't know of a way to let all subvolumes mount when mounting the parent ("pool"; and in the proper place when mounting to an alternate mountpoint), as is the case with ZFS. I do think that ZFS's lz4 compression is more effective though ... and the fact that you can create a dataset with its specific mountpoint that's basically anywhere in the tree without having to edit fstab pleases me a lot.

You're right though that I probably ought to have asked in an Ubuntu forum — the best tutorial how to install a mainstream Linux distro to a ZFS root is written for Ubuntu after all...

[oshunluvr]

The reason for the assumed loss in reliability is probably no more than the usual one when using RAID0 for any purpose - 2 drives = twice as likely to fail. I assume when you say bit rot you mean data decay. I see your point there - I've never experienced data decay that I'm aware of nor do I know anyone who has - but most data in my world (I run a simulator network) is refreshed often enough that I doubt it has time to occur. I suppose if you're stuck with a single device for storage having copies=2 might be useful. In my world, I never have only one device for storage.

Some of the other possibilities of ZFS you refer to are interesting, but of little use to average users. I doubt the speed increases are noticeable during average use (desktop/laptop daily use, not production environments) and backups are commonplace these days with the low cost of additional or cloud storage for backups. Besides, scans of Grandma's photos and a couple 100 CD rips aren't worth that much labor IMO. I guess from my viewpoint the superior benefits of ZFS aren't justified by the extra work vs. just selecting btrfs at install time.

Still, it be fun to create a fully fine tuned ZFS system just to see how it differs. Now that SSDs are cheaper the $1 MB level, fancy partitioning schemes and exotic file systems are in less demand. My main home PC has 2x256GB Samsung 840 Pros running btrfs in RAID0 data/ RAID1 meta mode and then backed up to a platter drive. It's so fast that I have no reason to explore further improvements.

Does the copies=2 setup autodetect data decay and fix it without sysop intervention? That would be massive. I hadn't really considered the mounting differences. I don't think editing fstab once every time I do an install is that much of a bother. I probably use 30% of what btrfs has to offer and still it's head and shoulders above ext4.

I wasn't aware that anyone Ubuntu had written that tutorial, so maybe you will find someone over there. I do think the main Ubuntu forum is not a great one. Maybe you can find the author(s) of the tutorial on IRC somewhere. When I suggested looking else for info, I really only meant that I cruise this forum daily and no one has ever mentioned ZFS ever. It's likely, short you and me and less than a handful of others - no one has even heard of ZFS. Heck, BTRFS is in the kernel and almost no one knows about or uses it.

[rjvbb]

I never said that that tutorial was published on the Ubuntu forum ... https://github.com/zfsonlinux/pkg-zf...oot-Filesystem

You're right, bitrot/data decay aren't that common in my experience either, no matter what the author of the Ars article seems to claim. But yes, the whole goal of a COW filesystem with checksumming and redundancy is to detect and correct errors due to flipped bits, failing sectors or whatever transparently. Btrfs will do that too, but you'd need to have configured a mirror, which is of course also the typical use case for ZFS.

I didn't mean to suggest that ZFS is faster, though. It probably isn't, except maybe for specific cases like removing a large directory tree with sync=disabled.

I find myself more and more creating a new dataset when I start playing with building a new kernel, or things like trying pkgsrc. That's why I mentioned editing fstab: when I'm done I can just destroy the dataset (faster than removing a big directory tree), and I don't have to remember to clean up fstab. It's not a big thing, just a nice li'l bonus I appreciate.

As to extra work: that's only because the installers don't support it It was really very easy to get an LMDE install on ZFS: I just cloned an install that had all the necessary packages installed from an ext4 partition onto a ZFS pool, nothing special there.

[oshunluvr]

Agreed re. installers and cloning. Basically the same thing I do when a distro doesn't "do" btrfs, lol. Good to know you won't be trolling at Ubuntu forums - It's been a long time since I found any love over there I can see your point about dataset destruction - just not one of my common functions. Re. the speed thing I was thinking about having /tmp and others in a RAID0 or single data mode and "real" data in RAID1 or copies=2 mode. Makes sense it would be somewhat faster than a full disc array.

Anyway - good luck! Post back when you have success. You never know who might be lurking