jcl123
Dabbler
- Joined
- Jul 6, 2019
- Messages
- 23
Hello all,
OK, I have a bunch of hardware to build some new servers, I was using a QNAP for years until it died. I have been struggling to decide between FreeNAS and Windows with Storage spaces for a while, but I am a bit fearful of ReFS until maybe it has a few more years and iterations under it's belt. FreeNAS and ZFS seem like a much more stable option for important data.
At the same time, I am not an experienced Linux/UNIX/etc. guy, and honestly I find Samba annoying for a bunch of reasons. It always annoyed me on the QNAP about the translation between the network shares and file system, differences in administration behavior, and a long list of other quirks with SMB, I much prefer to use Server 2019.
So, wondering what would happen if instead I just present an iSCSI share to a Windows VM? More complex, but potentially the best of both worlds?
I am building FreeNAS as a VM on VMWare ESXi anyways (with pass-thru), so at the same time I could also present iSCSI for VMFS while I am at it.
So, I am wondering what people think of this idea?
Obviously the hardware matters, so if ppl are in favor of this idea, I could use some advice on how specifically to implement it.
What I have is as follows, I have been trying to follow the guides and recommendations as closely as possible.
I am doing two servers and then backing up or replicating between them (need to decide the best way to do that)
There will be a 3rd copy at another site or in the cloud such as Backblaze (TBD)
Hardware:
- Two Supermicro motherboards with integrated 12Gig LSI HBAs (IT mode)
- One CPU is 16-core and the other is 12-core
- 128GB of RAM each, all ECC RDIMMs (using 16GB DIMMs)
- A pair of Intel 10Gig NICs
- Disks
- I have many 4TB 7200RPM Seagate Enterprise SAS drives for data
- I have 7x WD 10TB SATA drives ("shucked" from USB drives) (where data currently resides on Windows machine)
- I have a bunch of 200Gig SAS eSSDs with PLP, these are SLC 10DWPD high endurance (Samsung)
- I have a bunch of 400Gig SAS eSSDs, also with PLP, these are TLC but are also 10DWPD high endurance (Seagate)
- I have 4x eNVMe drives, these are Intel TLC but only medium endurance, I think 2-3DWPD
- I have one 16-bay Supermicro case with 2x platinum power supplies as well as a UPS
- The other case is a normal tower case, but it can handle 12 3.5" disks plus a couple more 2.5" SSDs
- I did convert my QNAP into a JBOD with a nice Intel expander, so I could do another 10x drives with that on either machine
Some other supporting information:
- I have 20TB of data currently, growth has slowed down allot recently, so 30-40TB available is plenty
- Pictures, home videos (4K and VR/360), and DVD/BD ISOs, and backups of other computers (tax records and documents don't amount to much)
- I care mostly about data security, and then maybe power consumption and noise
- These are in the basement where it is cold, and I have tuned the fans for low noise already
- I don't plan on creating a whole bunch of high-performance VMs if any at all
- Performance is important, but data is being served over gigabit to a dozen or so computers
- It may not seem like it, but I do like simple, I do work as a Windows / VMWare admin/engineer
Questions:
- If you are just doing normal file sharing, most recommendations say don't bother with ARC/ZIL on SSDs unless you are doing iSCSI
- But, if I am doing iSCSI maybe I do need them, although if it is mainly for a file share, maybe a corner case?
- When you get into that, I see allot of controversy over the size and configuration of SSDs used for ARC/ZIL
- I am leaning towards using the SAS SSDs rather than the NVMe's because of the higher endurance
- I am wondering how much RAM to give FreeNAS, seems pretty hungry, maybe 32-64GB?
- I am seeing that you don't want to create a VDEV larger than 8-12 or so drives
Hoping some people with allot of track time with these things can point me in the right direction.
Thanks in advance
-JCL
OK, I have a bunch of hardware to build some new servers, I was using a QNAP for years until it died. I have been struggling to decide between FreeNAS and Windows with Storage spaces for a while, but I am a bit fearful of ReFS until maybe it has a few more years and iterations under it's belt. FreeNAS and ZFS seem like a much more stable option for important data.
At the same time, I am not an experienced Linux/UNIX/etc. guy, and honestly I find Samba annoying for a bunch of reasons. It always annoyed me on the QNAP about the translation between the network shares and file system, differences in administration behavior, and a long list of other quirks with SMB, I much prefer to use Server 2019.
So, wondering what would happen if instead I just present an iSCSI share to a Windows VM? More complex, but potentially the best of both worlds?
I am building FreeNAS as a VM on VMWare ESXi anyways (with pass-thru), so at the same time I could also present iSCSI for VMFS while I am at it.
So, I am wondering what people think of this idea?
Obviously the hardware matters, so if ppl are in favor of this idea, I could use some advice on how specifically to implement it.
What I have is as follows, I have been trying to follow the guides and recommendations as closely as possible.
I am doing two servers and then backing up or replicating between them (need to decide the best way to do that)
There will be a 3rd copy at another site or in the cloud such as Backblaze (TBD)
Hardware:
- Two Supermicro motherboards with integrated 12Gig LSI HBAs (IT mode)
- One CPU is 16-core and the other is 12-core
- 128GB of RAM each, all ECC RDIMMs (using 16GB DIMMs)
- A pair of Intel 10Gig NICs
- Disks
- I have many 4TB 7200RPM Seagate Enterprise SAS drives for data
- I have 7x WD 10TB SATA drives ("shucked" from USB drives) (where data currently resides on Windows machine)
- I have a bunch of 200Gig SAS eSSDs with PLP, these are SLC 10DWPD high endurance (Samsung)
- I have a bunch of 400Gig SAS eSSDs, also with PLP, these are TLC but are also 10DWPD high endurance (Seagate)
- I have 4x eNVMe drives, these are Intel TLC but only medium endurance, I think 2-3DWPD
- I have one 16-bay Supermicro case with 2x platinum power supplies as well as a UPS
- The other case is a normal tower case, but it can handle 12 3.5" disks plus a couple more 2.5" SSDs
- I did convert my QNAP into a JBOD with a nice Intel expander, so I could do another 10x drives with that on either machine
Some other supporting information:
- I have 20TB of data currently, growth has slowed down allot recently, so 30-40TB available is plenty
- Pictures, home videos (4K and VR/360), and DVD/BD ISOs, and backups of other computers (tax records and documents don't amount to much)
- I care mostly about data security, and then maybe power consumption and noise
- These are in the basement where it is cold, and I have tuned the fans for low noise already
- I don't plan on creating a whole bunch of high-performance VMs if any at all
- Performance is important, but data is being served over gigabit to a dozen or so computers
- It may not seem like it, but I do like simple, I do work as a Windows / VMWare admin/engineer
Questions:
- If you are just doing normal file sharing, most recommendations say don't bother with ARC/ZIL on SSDs unless you are doing iSCSI
- But, if I am doing iSCSI maybe I do need them, although if it is mainly for a file share, maybe a corner case?
- When you get into that, I see allot of controversy over the size and configuration of SSDs used for ARC/ZIL
- I am leaning towards using the SAS SSDs rather than the NVMe's because of the higher endurance
- I am wondering how much RAM to give FreeNAS, seems pretty hungry, maybe 32-64GB?
- I am seeing that you don't want to create a VDEV larger than 8-12 or so drives
Hoping some people with allot of track time with these things can point me in the right direction.
Thanks in advance
-JCL