ZFS on laptop or desktop

[freenastier]

I really love the redundancy, snapshot and error correcting features ZFS offers. Now I was wondering if I could use it also as a filesystem on my desktop or laptop.
I do understand that I loose many of the qualities ZFS offers due to hardware restrictions, such as only 1 hard drive and the lack of ECC memory, but I am primarily interested in the snapshot / replication possibilities.

For example if I had ZFS on my laptop, then I could make snapshots of my important data and send them to a data-set on my Freenas server.
Then for example if I screw up my laptop during a workshop I could simply rollback to a working snapshot. If I loose data, then it is still available as a bakup on the Freenas server hopefully.

Is this a good idea or is ZFS not a good candidate to be used like that?

 

[Jailer]

I have a spare FreeBSD desktop with a ZFS root installation. TrueOS also uses ZFS if you don't want to roll your own.

This should probably be moved to off topic.

 

[freenastier]

Sorry for posting in the wrong thread. I thought this was a 'general question'. My apologies.

I noticed TrueOS is also a BSD flavour. Would ZFS on Linux also be an option or is the risk of running into compatibility issues too big?
I have more experience with Linux myself and often need to run Linux.

 

[Jailer]

Sorry for posting in the wrong thread.

No worries, I just threw that in there in case a mod happens to see this thread.

Would ZFS on Linux also be an option or is the risk of running into compatibility issues too big?

That I can't answer as I've never used ZFS on Linux. I'm not sure if ZFS on Linux supports a root ZFS installation. I know it didn't in the past but that may have changed.

 

[Ericloewe]

ZFS on Linux is compatible. There's always the issue of not every system supporting all new feature flags immediately, so the bleeding edge is indeed bloody. However, you can easily create a pool with just the feature flags that are supported across the OS versions you need.

That said, ZFS is not a first-class citizen on Linux, unlike FreeBSD and Illumos. It's really a better experience in those OSes. For FreeBSD, TrueOS is pretty nice and really makes use of ZFS.

 

[emk2203]

I really love the redundancy, snapshot and error correcting features ZFS offers. Now I was wondering if I could use it also as a filesystem on my desktop or laptop.
I do understand that I loose many of the qualities ZFS offers due to hardware restrictions, such as only 1 hard drive and the lack of ECC memory, but I am primarily interested in the snapshot / replication possibilities.

For example if I had ZFS on my laptop, then I could make snapshots of my important data and send them to a data-set on my Freenas server.
Then for example if I screw up my laptop during a workshop I could simply rollback to a working snapshot. If I loose data, then it is still available as a backup on the Freenas server hopefully.

Is this a good idea or is ZFS not a good candidate to be used like that?

The latest zfs-on-linux has feature parity with the zfs version used in FreeNAS and FreeBSD, at least since version 0.7.5. All feature flags are supported. A current Ubuntu 18.04LTS has a recent enough zfs version, and you could install it with zfs-on-root.

I am not using this setup currently, but I have used it in the past and was quite happy with it. You have the software option from Linux together with zfs.

 

[Arwen]

Well, I have ALL my Linux computers using ZFS with both Mirrored root pool, and simple dumping ground striped, (no redundancy). All 3 use Gentoo Linux which supports ZFS pretty well, (but is not for the less technically savy people). I can and do boot off a ZFS root pool. That gives me snapshots, clones and alternate boot environments for upgrades that can be backed out.

Here is the break down;

• Desktop with 64GB SATA DOM & 500GB spinner, with 64GB from each for root pool mirroring
  
• Laptop with 1TB SATA, 2 x 30GB partitions for Mirrored root pool, and the remaining as a simple stripe
• Miniture media server with 1TB mSATA & 2TB spinner, with 25GB from each for root pool mirroring

Of course on the laptop, mirroring onto the same device is both slower and does not protect against device failure. But DOES protect against block failures and bit rot.

For the last 10 years or so, I have been using alternate boot environments so that I can backout failed updates;

1. First, I simply used 2 same sized partitions on a single disk. Before an update, I cloned the working OS to the alternate partition. Then booted off that alternate to make sure it would work. I then performed the update. If the update went south, I simply rebooted and selected the next lower Grub entry.
2. Second generation of ABEs used BTRFS and snapshots. This was clumsey in the early days, since named datasets could not be used in Grub for the Linux command line. I had to extract the BTRFS dataset number and use that. They have since fixed that.
3. About 2013 ZFS on Linux stabilized, (even more stabilization came in 2015). I started using ZFS on Linux, and never looked back. BTRFS is dead to me since they NEVER stabilized things like RAID-5/6, and have quite a few places where dataloss can occur.

Last, I now use ZFS snapshots on my home directories on each. Two sets;

• Hourly snapshots, for 1 day, (24 snapshots that over-write the oldest)
• Daily snapshots, but taken every hour to get the most current time the computer was up. Only 7 snapshots.

I even used them just 2 hours ago for a quick restore of something I over-wrote, (user error :).

There are some Linux distros that support ZFS on Linux, but if I recall correctly their is only one distro at present that has a GUI installer for ZFS on root. The rest require manual work. It's as @Ericloewe says, ZFS on Linux is not really a first class citizen. That said, it's VERY stable and QUITE usable. We even have dataset encryption coming sooner on Linux than the other *nixes.

 

[Ericloewe]

Of course on the laptop, mirroring onto the same device is both slower and does not protect against device failure. But DOES protect against block failures and bit rot.

Why not copies = 2? I expect that to be safer, besides being faster.

 

[KrisBee]

I think comments like zfs uses whole disks, ECC RAM and lots of it need considered qualification if people are not to be discouraged from using zfs on desktops and laptops. For example, which of Arwen’s systems uses ECC RAM? Do they require any more or less RAM than say an ext4 based system in order to perform at around the same level? But licensing makes zfs on root for Linux generally a DIY effort and a stumbling block.

There are some Linux distros that support ZFS on Linux, but if I recall correctly their is only one distro at present that has a GUI installer for ZFS on root. The rest require manual work. It's as @Ericloewe says, ZFS on Linux is not really a first class citizen. That said, it's VERY stable and QUITE usable. We even have dataset encryption coming sooner on Linux than the other *nixes.

If @Arwen is thinking of Antergos, as a recent Linux Journal article showed, the installer does not create a truly redundant zfs mirror as /boot is placed on a separate ext4 partition and grub only on one drive. The PROXMOX installer can create a redundant zfs mirror with root on zfs and grub installed on both drives. PROXMOX have fully embraced zfs which makes perfect sense for their specialist debian based virtualisation platform. But for widespread use of zfs on linux desktops and laptops the use case is not so compelling. Yet my ideal linux would install zfs on root, have easily administered alternative boot environments, no systemd and no pulseuadio.

If TrueOS hadn't decided to reinvent the PC-BSD wheel and wasn't such a moving target I might consider it. GhostBSD is a possible alternative, but really it seems to boil down to BSD + DIY config of a desktop environment and living without skype and adobe flash.

 

[danb35]

I think comments like zfs uses whole disks, ECC RAM and lots of it need considered qualification

Nobody knowledgeable is making such broad (and incorrect) statements.

PROXMOX have fully embraced zfs

Well, kind of fully. It's fully supported in their installer, which will (as you say) do root on ZFS in whatever ZFS configuration you want (even if they like the traditional RAID terminology over proper ZFS terminology). They do have the handy pve-zsync script. But other than that, you're on your own. Expand a pool? CLI. Replace a disk? CLI--and you'd better hope you get it partitioned correctly, GRUB installed correctly, etc. Scrubs? CLI. SMART tests? CLI.

 

[KrisBee]

Proxmox is not a NAS product, so yes you'd need the CLI for the kind of zfs admin you've mentioned. Is that much different to say a FreeBSD desktop install for example with one pair of mirrored drives for root and other drives for data?

Nobody knowledgeable is making such broad (and incorrect) statements

Hopefully not, but these are common misconceptions that zfs advocates have to repeatedly counter. Having read the negative attitudes to using zfs on consumer grade hardware without ECC RAM a prospective user might feel rightly nervous about adopting zfs on their desktop.

But the OP had asked about of zfs on a desktop/laptop - "Is this a good idea or is ZFS not a good candidate to be used like that?" @Arwen has shown some possibilities and benefits, but not any gotchas or downsides. Laptop users breaking free of Windows can typically hit problems with EFI booting, hardware compatibility, getting functions keys working, suspend/resume and graphic drivers. Is zfs the right choice for latest shiny gaming rig?

 

[Arwen]

Why not copies = 2? I expect that to be safer, besides being faster.

There is a rumor that some SSDs will de-duplicate indentical blocks. So, I try and prevent that by 2 different partitions. Will my scheme work? Don't know.

Plus, ZFS is smart enough to write 2 copies far apart, EXCEPT if the SSD re-maps them to be together, in the same easure block.

Laptop speed was perfectly fine until the last 2 months, Meltdown. Need I say more?

 

[danb35]

Hopefully not, but these are common misconceptions that zfs advocates have to repeatedly counter.

...largely (unfortunately) due to overzealous advocates on this very forum, some of whom (inadvertently, no doubt) perpetuating FUD like the Scrub of Death(tm). Also due to the fact that FreeNAS is memory-hungry. Sure, ZFS will use all your RAM for caching if you let it, but it isn't ZFS's fault that FreeNAS used to fall over, die, and eat your data if it didn't have at least 8 GB of RAM (this behavior seems to have improved in recent years, thankfully, but we've also been a lot more insistent on adequate RAM).

The issue with consumer-grade hardware isn't, really, that it's a bad choice for ZFS; it's that (1) it's often simply crappy hardware (e.g., Realtek NIC); (2) it has features that servers don't need (e.g., audio), while lacking features that servers often do need (e.g., IPMI); and (3) hardware compatibility issues with FreeBSD/Linux.

Proxmox is not a NAS product, so yes you'd need the CLI for the kind of zfs admin you've mentioned.

"Oranges are round, so yes the sky is blue." Proxmox uses storage. It primarily operates using a web GUI. It's perfectly sensible that storage management should be part of that web GUI. It isn't (for ZFS or anything else), but that doesn't mean it doesn't belong there.

Is zfs the right choice for latest shiny gaming rig?

I'd say ZFS is a good choice for any hardware that can run a suitable OS well. Since "a suitable OS" excludes Windows, and "latest shiny gaming rig" may have hardware compatibility issues with Linux (not to mention FreeBSD), it's going to be very hardware-specific.

 

[Arwen]

@KrisBee, none of my Linux computers have ECC memory, though I would prefer such. Next, here is the amount of memory;

1. Desktop - 8GB
2. Media server - 16GB
3. Laptop - 2GB

The laptop sometimes has trouble compiling large packages of software with 2 or 3 threads. For those, I have to reset my compile parameters to 1 thread. (Gentoo is a source distro...) And potentially un-mirror my swap, so I can double it's size.

Next, for recovery purposes, I use separate /boot and swap. They get mirrored via standard Linux MD-RAID. Clumsy, but it works reasonably well. Even lets me hibernate my desktop and laptop.

 

[Arwen]

..
But the OP had asked about of zfs on a desktop/laptop - "Is this a good idea or is ZFS not a good candidate to be used like that?" @Arwen has shown some possibilities and benefits, but not any gotchas or downsides.
...

Ask, and you shall receive. Downsides and gotchas of ZFS on Linux;

• Can't update pool to use latest features until I can get a recorvery image that supports those features. So, no new compression or checksum algorythms.
• Installation is clumsy, but part of that is using Gentoo
• Took me a while to get a dataset / filesystem layout I liked
• All kernel updates require me to re-build the ZFS packages, before I create the initial RAM disk
• Grub 2 is clumsy as *ell, forcing me to manually create my own entries for ZFS alternate boot environments.
• YAGG - Yet Another Grub Gripe. ZFS can be a bit of a moving target, with new features being released several times a year. It can take a long time for Grub to get both updated and down to the users who want those features.

Probably a few more.

 

[KrisBee]

@Arwen thanks for those additional hardware details and the frank appraisal of the downsides as you see them. One of which is the obvious need to plan for zfs aware rescue media. Your list tends to feed my own prejudice that apart from zfs on Linux for data drives, I can't see myself committing to zfs on root in Linux outside of a non critical Proxmox install at home, or purely as a learning experiment.

@danb35 Maybe those storage elements will appear under the proxmox web GUI in future releases, but I think it is expected that proxmox users are proficient at the CLI, are familiar with their admin guide and the current limitations of the web interface. Don't the priorities of the developers just reflect their commercial target market?

 

[danb35]

Don't the priorities of the developers just reflect their commercial target market?

Well, I assume those priorities reflect their perception of their target market (whether that perception is accurate is a separate question, I guess). But it just seems incomplete--if you're going to GUI-fy the thing, GUI-fy the whole thing, don't do it halfway. And the web interface for Proxmox is quite a mature feature, not something that was just released and is rapidly evolving.

 

[KrisBee]

As a casual home user of proxmox, I'm not bothered, it does what I want. Serious production users with multi-node Ceph setups etc. may have a different opinion.

One horrible hack I've not tried is putting x11 and something like xfce on proxmox as a quick and dirty way of get zfs on root linux. Proxmox installer doesn't support EFI , but I wouldn't need it on any real hard I had.

 

[emk2203]

@Arwen thanks for those additional hardware details and the frank appraisal of the downsides as you see them. One of which is the obvious need to plan for zfs aware rescue media.

If you want to have zfs aware rescue media, you can use Parted Magic. It's zfs aware and always quite recent.

 

[Arwen]

@emk2203, interesting product, but it costs money.
Cheaper to simply open a bug in Gentoo to request an update of the boot image.