Hi
I'm thinking of change from my current storage system setup. For reference, I'll include my full server setups as this will set the context. Apologies, I think this might be a long post!
I have a 3 node vSphere cluster with vCenter and HA/DRS/vMotion enabled. I utilise Starwind vSAN (free version) with directIO enabled to give each Windows VM local storage which is then shared via iSCSI to all 3 machines. The vSAN is a mirrored array setup to ensure that I can keep VMs running in the case of a single host failure.
On one of these hosts I presently run an xpenology VM build using DSM 6.1, with 3 vCores and 6GB ram assigned. This host is an X8DTE-f system with quad core westmere xeon and 32GB DDR3 rdimms running in a 16 bay SM chassis (12 assigned to the Xpenology VM, 4 to the vSAN VM) and 2 LSI 9211-8i cards in passthrough. The Xpenology system is purely used as general house storage, VM backups and stuff. It uses docker to then backup the storage array to crashplan.
The storage is setup as SHR-2 (2 disk parity, hybrid raid) using 5 disks presently, 3 WD Red 3TBs and 2 enterprise Seagate 2TB drives, with the drive format as BTRFS.
I'm always concerned Xpenology/DSM may just fail, an update may kill everything given that its unsupported and Synology like to change security implementations every time they update. I'm also concerned about BTRFS in general and given DSM uses MDADM rather than the BTRFS raid setup, it probably won't really protect from bitrot... So, I keep looking at FreeNAS and wondering if I can really use it. I'll make it clear here that I do not intend to run FreeNAS as VM storage for the vSphere hosts, the VMs are handled through vSAN... FreeNAS would be replacing Xpenology as general storage.
So I have a number of themes I want to ask about!
1) Drive setup / vdevs etc.
As mentioned above I use a hybrid raid to allow for mixed drive usage.. My understanding is that FreeNAS doesn't support this approach, is this ever going to come or just wishful thinking?
As I want to keep drive replacement costs to a minimum as I can't simply afford to replace 12 drives everytime I want to increase capacity, and you can't expand a vdev once created, I was thinking of using RaidZ1 with 4 drives per vDev, which would allow me to have up to 3 vDevs to play and expand storage as necessary. Does this make sense? How much stress would a RaidZ1 be under when rebuilding from a drive failure and what is the consensus around risk of vdev failing as I understand once a vdev fails, the Zpool is also gone?
In simple terms:
vDev1 (RaidZ1) vDev2 (RaidZ1) vDev3 (RaidZ1)
Drive 1 Drive 5 Drive 9
Drive 2 Drive 6 Drive 10
Drive 3 Drive 7 Drive 11
Drive 4 Drive 8 Drive 12
I would get 3 disk parity across 3 vdevs?
2) Apps etc.
I currently have installed the following:
Hyper backup - backs up docker config and containers to my onedrive so that if I get failure of the machine I can get these dockers backup quickly and grab my data from crashplan.
Docker - Crashplan, TVheadend - as described by their names!
Download Station - very good download program, bittorrent etc.
Emby/Plex Server - no description needed!
What does FreeNAS have in the way of these? Can I use Docker as I am familiar with this? Is there something equivalent to download station?
I've got Westmere CPUs so these have more virtualisation extensions that the Nahalem chips, so presume no issues running docker?
3) FreeNAS as a VM with the same config as Xpenology
As I utilise an alternative way of providing HA access to VMs, my purpose around FreeNAS is the same as Xpenology... General storage but this time in a supported platform with better data protection... Are there any issues with running FreeNAS as a VM? Can I run jails and docker images with FreeNAS as a VM? Anything I need to be aware off?
I'll probably end up migrating from VM to baremetal but that would happen once I move the servers to more low powered units.
Thanks for your time in reading this far too long post and possibly ill-formated too!
Any thoughts or comments are really appreciated.
Cheers
Chris
I'm thinking of change from my current storage system setup. For reference, I'll include my full server setups as this will set the context. Apologies, I think this might be a long post!
I have a 3 node vSphere cluster with vCenter and HA/DRS/vMotion enabled. I utilise Starwind vSAN (free version) with directIO enabled to give each Windows VM local storage which is then shared via iSCSI to all 3 machines. The vSAN is a mirrored array setup to ensure that I can keep VMs running in the case of a single host failure.
On one of these hosts I presently run an xpenology VM build using DSM 6.1, with 3 vCores and 6GB ram assigned. This host is an X8DTE-f system with quad core westmere xeon and 32GB DDR3 rdimms running in a 16 bay SM chassis (12 assigned to the Xpenology VM, 4 to the vSAN VM) and 2 LSI 9211-8i cards in passthrough. The Xpenology system is purely used as general house storage, VM backups and stuff. It uses docker to then backup the storage array to crashplan.
The storage is setup as SHR-2 (2 disk parity, hybrid raid) using 5 disks presently, 3 WD Red 3TBs and 2 enterprise Seagate 2TB drives, with the drive format as BTRFS.
I'm always concerned Xpenology/DSM may just fail, an update may kill everything given that its unsupported and Synology like to change security implementations every time they update. I'm also concerned about BTRFS in general and given DSM uses MDADM rather than the BTRFS raid setup, it probably won't really protect from bitrot... So, I keep looking at FreeNAS and wondering if I can really use it. I'll make it clear here that I do not intend to run FreeNAS as VM storage for the vSphere hosts, the VMs are handled through vSAN... FreeNAS would be replacing Xpenology as general storage.
So I have a number of themes I want to ask about!
1) Drive setup / vdevs etc.
As mentioned above I use a hybrid raid to allow for mixed drive usage.. My understanding is that FreeNAS doesn't support this approach, is this ever going to come or just wishful thinking?
As I want to keep drive replacement costs to a minimum as I can't simply afford to replace 12 drives everytime I want to increase capacity, and you can't expand a vdev once created, I was thinking of using RaidZ1 with 4 drives per vDev, which would allow me to have up to 3 vDevs to play and expand storage as necessary. Does this make sense? How much stress would a RaidZ1 be under when rebuilding from a drive failure and what is the consensus around risk of vdev failing as I understand once a vdev fails, the Zpool is also gone?
In simple terms:
vDev1 (RaidZ1) vDev2 (RaidZ1) vDev3 (RaidZ1)
Drive 1 Drive 5 Drive 9
Drive 2 Drive 6 Drive 10
Drive 3 Drive 7 Drive 11
Drive 4 Drive 8 Drive 12
I would get 3 disk parity across 3 vdevs?
2) Apps etc.
I currently have installed the following:
Hyper backup - backs up docker config and containers to my onedrive so that if I get failure of the machine I can get these dockers backup quickly and grab my data from crashplan.
Docker - Crashplan, TVheadend - as described by their names!
Download Station - very good download program, bittorrent etc.
Emby/Plex Server - no description needed!
What does FreeNAS have in the way of these? Can I use Docker as I am familiar with this? Is there something equivalent to download station?
I've got Westmere CPUs so these have more virtualisation extensions that the Nahalem chips, so presume no issues running docker?
3) FreeNAS as a VM with the same config as Xpenology
As I utilise an alternative way of providing HA access to VMs, my purpose around FreeNAS is the same as Xpenology... General storage but this time in a supported platform with better data protection... Are there any issues with running FreeNAS as a VM? Can I run jails and docker images with FreeNAS as a VM? Anything I need to be aware off?
I'll probably end up migrating from VM to baremetal but that would happen once I move the servers to more low powered units.
Thanks for your time in reading this far too long post and possibly ill-formated too!
Any thoughts or comments are really appreciated.
Cheers
Chris
Last edited: