Chris Wieringa
Cadet
- Joined
- Mar 29, 2016
- Messages
- 2
First time builder here, been lurking and researching for 2-3 weeks now. I'm looking at building a high-performance FreeNAS storage box to use as primary storage for 5 XenServer hypervisors. This build would be replacing some old Dell MD3000i SAS 3.0Gbps arrays (60 spindles total) running iSCSI, with a new FreeNAS system of >35TB of usable space running NFS. I hoping to get 5 years of worry-free usage on this platform.
Currently I have 2 VM file servers, and 1 database server that do non-trivial amounts of IOPS, and a whole lot (40-60) VMs that are idle 98% of the time. Additionally, nightly incremental host-based backups are performed of important VMs and full backups on weekends.
I figure total IOPS of my current Dell arrays is less than 12000, and am looking to rely on FreeNAS's read cache / ZIL write-cache to bring up IOPS while bringing down the total number of disks. A Slog may be considered in the future if RAM doesn't suffice for read-cache. A ZIL would be used because of NFS for attaching the disk to the XenServers.
I'm planning on upgrading networking to 10Gbps for this build, and would hope to connect the FreeNAS via 20Gbps bond. Servers may get upgraded to 10Gbps, or may continue to use a 2 or 4Gbps bond.
After reading the forums, I thought I would run the build past some knowledgeable folks before moving on with it.
(Build via Thinkmate: http://www.thinkmate.com/system/superstorage-server-5028r-e1cr12l/138752 )
Base System: Supermicro SuperStorage Server 5028R-E1CR12L
CPU: (1) Intel Quad-core E5-1620v3 3.50Ghz
Memory: 256GB - (8) 32GB PC4-17000 2133Mhz DDR4 ECC Registered (running at effective 1866Mhz)
System Disks: (2) 128GB Micron M600 SATA 6.0Gbps SSD
Storage Disks: (12) 6.0TB SAS 3.0 12.0Gb/s 7200RPM 3.5" Hitachi Ultrastar HE8 (512e)
ZIL: Intel SSD DC P3600 Series 400GB PCIe 3.0 x4 NVMe Solid State or Intel 750 400GB PCIe 3.0 x4 NVMe Solid State (overprovisioned)
Controller: Integrated LSI 3008
A couple of questions I had that I can't find clear answers for:
Currently I have 2 VM file servers, and 1 database server that do non-trivial amounts of IOPS, and a whole lot (40-60) VMs that are idle 98% of the time. Additionally, nightly incremental host-based backups are performed of important VMs and full backups on weekends.
I figure total IOPS of my current Dell arrays is less than 12000, and am looking to rely on FreeNAS's read cache / ZIL write-cache to bring up IOPS while bringing down the total number of disks. A Slog may be considered in the future if RAM doesn't suffice for read-cache. A ZIL would be used because of NFS for attaching the disk to the XenServers.
I'm planning on upgrading networking to 10Gbps for this build, and would hope to connect the FreeNAS via 20Gbps bond. Servers may get upgraded to 10Gbps, or may continue to use a 2 or 4Gbps bond.
After reading the forums, I thought I would run the build past some knowledgeable folks before moving on with it.
(Build via Thinkmate: http://www.thinkmate.com/system/superstorage-server-5028r-e1cr12l/138752 )
Base System: Supermicro SuperStorage Server 5028R-E1CR12L
CPU: (1) Intel Quad-core E5-1620v3 3.50Ghz
Memory: 256GB - (8) 32GB PC4-17000 2133Mhz DDR4 ECC Registered (running at effective 1866Mhz)
System Disks: (2) 128GB Micron M600 SATA 6.0Gbps SSD
Storage Disks: (12) 6.0TB SAS 3.0 12.0Gb/s 7200RPM 3.5" Hitachi Ultrastar HE8 (512e)
ZIL: Intel SSD DC P3600 Series 400GB PCIe 3.0 x4 NVMe Solid State or Intel 750 400GB PCIe 3.0 x4 NVMe Solid State (overprovisioned)
Controller: Integrated LSI 3008
A couple of questions I had that I can't find clear answers for:
- What is the state of support for LSI 3008? While as of mid-2015 it was "don't do it - wait", but I can't find any statements on it with 9.10. Is it considered stable in FreeNAS 9.10? Or are there other 12G SAS controllers that would be better?
- Would I get the performance with FreeNAS (with ZIL, 256GB RAM read-cache) that I need when shrinking the spindles from 60 to 12?
- What would be the best way to configure my disks for performance and space? Two RAIDZ2 vdevs (4disks6TB2vdevs = 48TB, usable up to 80% 38.4TB), six mirrored vdevs (6TB * 6vdevs = 36TB, usable up to 80% 28.8TB)? I'm assuming a 12-disk RAIDZ3 would be too CPU intensive and beyond recommending to even consider.
- Even with massive over-provisioning, would the Intel P3600 or 750 not have enough endurance for a ZIL? I could go with a P3700 if necessary, but I figure it would be mostly wasted as a ZIL.
- Considering a 20Gbps network bond, what size ZIL should I shoot for?
- And last, is a single quad-core 3.5Ghz CPU fast enough or should I redesign with a dual-socket system?