BUILD Large NAS with various recycled hardware for ESXi

[Lunchbox Larry]

Dear forum,

This is my first post here, and I've spent a good deal of time reading before making this post.

I am attempting to build a large FreeNAS system for the purpose of being a datastore for ESXi. The hardware I have at my disposal is a bit of a hodge-podge of recycled components, but at least has some potential for performance. I'd like to post the build specs here and would be very grateful if more experienced FreeNAS users could give me a sanity check and/or suggest configuration specs for this system.

Here's the hardware list:

Server:
Dell PowerEdge 810
4x Intel(R) Xeon(R) CPU X7560 @ 2.27GHz (total 32 cores, 64 threads)
64GB DDR3 ECC RAM
2x 4Gb Dual port fiberchannel HBAs (4 ports)
3x NetApp DS14-MKII Drive shelves populated with HITACHI HUA721010KLA33SX A90A (42 disks total)
4 port PCI Intel 1GB NIC (NOT using the Dell Broadcom NICs as they seem to really hate FreeBSD)

The way I currently have them set up is with redundant multipathing, so there are 4 paths to each physical disk.

Some of my questions:

1) Would this hardware be capable of supporting a few dozen VMs?
2) Is there any way of monitoring drive shelf status (power supply, fans, etc) within FreeNAS?
3) If this system would be theoretically capable of handling this type of load, what partioning and pool scheme would you recommend for so many drives?

Thank you for your consideration, and I apologize if any of my questions are redundant to those already answered on this board (I couldn't find any).

Regards,

Jamie

 

[marbus90]

1) I suppose so. If it's slow, throw RAM at the problerm.
2) no clue, I don't even know if FreeNAS supports FC initiator mode properly. There's a supermicro controller card with its own IPMI chipset to run random supermicro chassis in JBOD mode. SAS DAS storage to the FreeNAS host is certainly preferable. IIRC it's around $300-$400 for a 24bay supermicro plus $100 for said controller card (powers on the PSUs in the chassis, controls fans etc, the Expander external connection is handled separately)
3) 2way or 3way mirrorsets, striped together. Maybe 3way mirror spread across all shelves - either read off serial numbers and create manually or just plug in 3 HDDs at a time and add the new mirror vdev.
..... ^ alternatively with the 24bay supermicro solution 2way mirrorsets, don't forget spares then.

 

[Lunchbox Larry]

Thank you for the detailed reply.

Regarding question #1, I suppose I ought to have looked into what the MAX RAM is for that system. Seems like 128GB is max if I'm reading correctly.

Regarding the FC adapters, I have successfully set up an identical system configured for backup storage as an NFS box, and so far it seems to be working well (the drive access portion anyhow). Only problems I have run into are with the Broadcom NICs which will just kill nfs for no reason at all and must be bounced to resolve. And of course, I've made zero progress getting monitoring of shelf status.

Thanks for the advice regarding the mirrorsets. I will look into this more.

I of course still welcome all feedback on this.

 

[Lunchbox Larry]

1) I suppose so. If it's slow, throw RAM at the problerm.
2) no clue, I don't even know if FreeNAS supports FC initiator mode properly. There's a supermicro controller card with its own IPMI chipset to run random supermicro chassis in JBOD mode. SAS DAS storage to the FreeNAS host is certainly preferable. IIRC it's around $300-$400 for a 24bay supermicro plus $100 for said controller card (powers on the PSUs in the chassis, controls fans etc, the Expander external connection is handled separately)
3) 2way or 3way mirrorsets, striped together. Maybe 3way mirror spread across all shelves - either read off serial numbers and create manually or just plug in 3 HDDs at a time and add the new mirror vdev.
..... ^ alternatively with the 24bay supermicro solution 2way mirrorsets, don't forget spares then.

Does anyone happen to know a model number for that? I've looked all morning and I can't find anything even close to that price point. Is this a current model, or an old one?

Thanks!

 

[marbus90]

CSE-PTJBOD-CB3 control board
MCP-120-84702-0N bracket