NobleKangaroo
Dabbler
- Joined
- Apr 18, 2017
- Messages
- 18
Looking to make a TrueNAS server to replace the current server. Plans are as follows:
R730xd 2.5" SFF
CPU: 2x E5-2690v4 or 2x E5-2640v3(4x or 2x, respectively, more compute power than current NAS)
RAM: 256GB ECC DDR4 - 8x 32GB 2Rx4 PC4-2400T-R
HBA: H730P or HBA330
NICs: 2x MCX312A-XCBT 10Gbps dual SFP+ (4 links in LACP) (I've used these for years and they're solid)
Disks:
Additional info:
Some questions I have:
Thank you!
R730xd 2.5" SFF
CPU: 2x E5-2690v4 or 2x E5-2640v3(4x or 2x, respectively, more compute power than current NAS)
RAM: 256GB ECC DDR4 - 8x 32GB 2Rx4 PC4-2400T-R
HBA: H730P or HBA330
NICs: 2x MCX312A-XCBT 10Gbps dual SFP+ (4 links in LACP) (I've used these for years and they're solid)
Disks:
- Front plane: 24x 1.2TB 10k 2.5 Samsung SAS (fast VM storage)
- Mid plane: 4x 18TB 5400 3.5 Ironwolf Pro SATA (slow NAS and media storage)
- Rear bays: 2x 200GB 2.5 Kingston SSD (mirrored boot drives)
Pools:- fast (VMware VM files): 3x RAID-Z2 8x 1.2TB 10k 2.5 Samsung SAS
- slow (backups / media): RAID-Z1 4x 18TB 5400 3.5 Seagate Ironwolf Pro SATA
- boot (TrueNAS boot): mirror 2x 200GB 2.5 Kingston SSD
Capacity: (before file systems)- 21.6TB (18x 1.2TB) of 10k SAS storage, double-disk parity on each volume
- 54TB (3x 18TB) of 7200 SATA storage, single-disk parity
Additional info:
The pool setup currently in use is:
- (red) 6x 10TB WD Elements SATA in RAID-Z2 (backups / media) / 36TiB usable
- (gold) 4x 4TB WD Gold SATA in RAID-Z1 (VMware VM files) / 10TiB usable
- (purple) 2x 8TB WD Purple SATA in RAID-1 (Blue Iris NVR) / 8TiB usable
After moving to a new server, this frees up the old server. The plan is to repurpose it for offsite backups:
- RAID-Z3 12x 10TB WD Elements SATA in RAID-Z2 (offsite backups) / 81TiB usable
- The offsite server will have a little more than enough space to host a full 1:1 copy of the primary server.
Some questions I have:
1. Thoughts on virtualizing TrueNAS, and passing the disks through directly to the VM?
I'm kind of leaning towards just putting it on bare metal, but it would be nice to be able to virtualize TrueNAS (and run other VMs) if the performance is relatively comparable (within maybe a 10% penalty loss) to that of running it physically. Whether I go with 2x E5-2690v4 or E5-2640v3 depends on whether or not I virtualize TrueNAS. Originally I was thinking about passing the disks through to VMware but if that causes issues, I'm going to back off and just install TrueNAS on bare metal. If the consensus isn't one way or the other, I'll absolutely resort to doing testing of my own before a full deployment.
2. Is an H730P fine, or should I really try to get an HBA330 instead?
I've heard mixed results ("H730P is fine" / "you should get an HBA330") and honestly don't have much of a preference, outside of perhaps resell value. This is such a small non-issue, but comments would be appreciated if there's a clear difference in performance.
3. Any insights into VM performance with the above proposed setup? (pool architecture, HBAs, etc)
VMs will be served to three ESXi servers via NFS, over 4x LACP'd 10GBps connections. What I have is sufficient but during brief periods of activity, I can see noticeable performance hiccups. I think it's partially because I'm reaching higher levels of storage saturation (using 66% of the 4-drive RAID-Z1 pool currently), but also partially because I have so few drives in that pool (4 drives in RAID-Z1) that are able to share the load. Upgrading to a larger amount of smaller drives should help with this. Surely the new setup will be better than just 4x 7200 RPM SATA drives in a single RAID-Z1, but I'm looking to see some real use-case if anyone has that info.
4. I'm not necessarily averse to switching to a mirrored setup for even faster VM storage, but...
I don't have too many reservations about switching to a mirrored setup such as:
- fast (VMware VM files): 8x mirror - 3x 1.2TB 10k 2.5 Samsung SAS
- slow (backups / media): RAID-Z1 - 4x 18TB 5400 3.5 Seagate Ironwolf Pro SATA
However I've really enjoyed the additional capacity that RAID-Z2 provides, compared to mirrors, and don't know if the risk of losing both drives in the same mirror (and thus the whole pool) is worth the speed bonus. Sure, I'll have offsite backups, but that could still represent hours - if not days - of data recovery, and short-term "not yet backed up" data loss. Feel free to share your use-case and the speeds you've seen. Despite being slightly biased, I'm open to persuasion.
5. Anything else that is glaringly obvious, feel free to comment.
Please provide feedback! I'm here to learn and to build a safe, yet performant setup.
Thank you!
Last edited: