For this system we could roll back as it is not a production box, We have 3 Host systems run a total of 12 VM's. Each VM is 50 GB with the exception of one that is 450GB and we can present storage as NFS or iSCSI.
Great, then you don't need a SLOG, which makes things a lot easier. I cannot make specific recommendation w/o numbers, but here is some fruit of thought:
To begin with, you will want a 1u system with 10 2.5' slots. 4 HDDs most likely will not provide sufficient IOPS for VMs
Get a supermicro system with at least 1 PCIe slot, 2 is better. 1 slot is for your 10G NIC and another is for your SAS HBA. SM makes MB /w SAS controller on-board as well, get one of that if you went with 1 slot chassis.
CPU doesn't really matter much, a cheap e5 v2 will do. Do not opt for the latest one because DDR4 RAM is much more expensive than DDR3. If you would ever need to go above 128G RAM, get 2 e5-26XX v2.
Start with 64G of ECC RAM, more if you want better read performance.
2 cheap SSDs for boot, used intel enterprise ones are cheap and really good.
Depending on your IOPS, get one of the following (assuming your space need is 1T):
1. 8 1T SATA HDDs in striped mirrors
2. 8 10k/15k 1T SAS HDDs in striped mirrors
3. 4 to 8 SSDs in striped mirrors, make sure total raw capacity is > 2T.
Due to the nature of COW filesystem, the write performance decreases exponentially as the pool fills, which is especially evident on HDDs. I recommend to keep the pool less than 33% full if you are using HDDs, and less than 50% for SSD.
