Freenas as home infrastructure core

[stevezemlicka]

I'm just getting over a long relationship with qnap after she broke my heart with data loss. It was odd because it was only partial and it wasn't any root folders but several folders were missing contents after a pool resize (which should have been non-destructive).

Anyway, I'm looking at Freenas. My last experience with it was I think circa 2009 which was relatively good but limited to NAS functionality. I've noticed there are quite a few more features than I remember and I'd like to leverage jails and virtualization. I'm close to deciding on Freenas (have my server burning in with a preliminary install of it) but before I commit, have a few questions that I'm hoping some out there can help me with.

1. I'm nervous about booting from a USB drive (that just isn't something that's overly common with my enterprise background). I've done some OSes on CF/SD but I see this isn't necessarily recommended from other iX forum threads. I have a proliant server with a SD and an internal USB. Any concerns with installing to both usb flash and sd for resiliency. I have limited drive bays so this would seem to make sense.
2. The word on the street is that Freenas updates openzfs beyond the standard BSD release. Is this true? While I do plan on repurposing my qnap as a backup appliance for freenas and then replicating to cloud as tertiary, it seems, shall we say, not prudent for freenas to implement openzfs that's not supported by BSD.
3. I came across a reddit thread where many people were pointing out issues with stability as it relates to Freenas Jails and some even indicate that they do their own scripted updates for Jails like Plex. While I'm an OK IT guy, I mainly work on Windows Servers (sympathy accepted) and linux. I'm not sure I have the time to invest in building my own support structure just to keep things from breaking?
4. I've used ESXi, Hyper-V, and XenServer but never bhyve. How is this from a hypervisor standpoint (particularly around stability for other server OSes)? Does it allow thin provisioning disks (I think I read that ProxMox doesn't)?

Any tips, tricks, or other helpful topics are welcome. Thanks for taking the time to read and for any input you may have.

 

[danb35]

I'm nervous about booting from a USB drive

You should be, and that really hasn't been the recommended boot media for several years--a small SSD is a better choice. You don't really need a drive bay for it either, but you'd obviously need a SATA port. If you don't have a free SATA port, you could look for a small USB-attached SSD instead.

issues with stability as it relates to Freenas Jails

I don't recall hearing of issues with stability of jails, but if you're using the plugins, they aren't updated very regularly. Most of the active people around here would recommend installing the software yourself in a jail (using someone's script if needed or desired) rather than using a plugin.

As to using it as a Hypervisor, I haven't done this--I use Proxmox for my virtualization needs (which works quite well).

 

[Yorick]

1) if you want to keep SATA for drives, you can do an m.2 2242 ssd with usb enclosure. I’ll find you a suggested combo. Or a small SATA ssd, I’ll edit and link you to a thread that discusses models.
2) OpenZFS 2.0 combines the ZoL and ZoF code base. Ix drives the BSD portion of that. Expect that TrueNAS Core, BSD and Linux will all be able to use each other’s ZFS pools. That’s a big change. As OpenZFS 2.0 is not released yet, expect there’ll be a transition period.
3) I am not aware of any stability issues with BSD jails, if stability means how well they work in production. I’ve seen reports of jails breaking during major version upgrades, and requiring some attention so their networking worked again. That never happened to me, and, your mileage will vary. I use “boring” jails - base jail, vnet, static IP (dhcp is fine too if one must), no NAT shenanigans and the most exciting thing I might do is change the devfs rules for Plex.
The Plex plugin is now as up to date as the pkg repository because there is no separate repository for the plugin any more. That said I use a jail created by Danb35’s script so I can play with Plex hardware transcode. Nothing special to update Plex, just a pkg upgrade -y.
4) bhyve is a mixed bag. There are reports of Ubuntu having trouble when configured with several vCPUs, though I don’t know whether FreeBSD 12.1 resolved that. It might have. VirtIO for storage and networking is a must and the “secret sauce” for performance and stability.
PCI passthrough is now supported, and works, but passing through a GPU isn’t supported and in my testing is not stable.
I don’t consider bhyve to be as solid as other hypervisor options.
I’d put the exact guests you want through their paces on TrueNAS Core 12.0 RC1, keeping in mind there is an open bug with bhyve losing the plot when a guest is restarted . That bug aside, run your desired guests in their desired configuration for a few weeks and see whether they stay stable.

TrueNAS as a virtualization platform will massively improve with TrueNAS Scale, enabled by the work talked about in 2). And, that’s 12 to 18 months out if you want to be conservative about it. There’s not even an alpha yet. Still, the prospect of a Linux-based TrueNAS should be interesting to anyone who wants to do more than casual virtualization.

 

[stevezemlicka]

Great info. I suppose a bit more info from me would be useful here. I already have the server and disks (proliant dl380 g7, 2x 1tb ssd, 4x 4tb ssd with plans to add a das enclosure for 3.5" spinning disks). So the initial challenge is how do I install while still still allowing me to leverage the two 1TBs which will likely be mirrored. Unfortunately the usb inside is vertical limiting the length of the drive. I do have a 256gb 2230 nvme but cannot find a USB to name that is short enough to fit (and I don't really want anything sticking out). I guess I could cut one because it seems the pcb is just for structure for the standard length nvme but I don't really want to mess with all that.

I really wish freenas offered some partitioning options during setup but maybe that's for the best. I'm starting to think that if my goal is to be sort of hyperconverged (to use marketing slang) maybe I should install proxmox or similar and then install freenas to a vm and pass through the data disks for raw access in freenas. Obviously freenas wouldn't be portable if i setup a 2nd proxmox server unless i moved the disks to a das/San but that's a challenge for future me.

Has anyone setup freenas inside xenserver/proxmox and passed through the data disks or would that end up being a bit of an experiment here?