Hi all,
I've been working on building an ESXi cluster for home use. I'd like to build it with a home-ish budget, but with (near) enterprise-level redundancy, backups, etc. I've been mixing and matching consumer-grade hardware for 5+ years and it's time for an upgrade.
The goal here is to have a 3-node ESXi cluster for compute and use FreeNAS for media storage and VMware datastores.
For VMs, I'll be running (mostly Windows Server 2016 but perhaps Linux where appropriate):
NAS01: Dell R710, 2x Xeon L5520, 48GB RAM, 4 NICs
Much of these (for example, the actual servers themselves) are already purchased. The R710 is already up and running. But I don't have any of the virtualization set up yet, as I'm still hammering out the proper architecture. Which leads me to my questions:
Thanks!
I've been working on building an ESXi cluster for home use. I'd like to build it with a home-ish budget, but with (near) enterprise-level redundancy, backups, etc. I've been mixing and matching consumer-grade hardware for 5+ years and it's time for an upgrade.
The goal here is to have a 3-node ESXi cluster for compute and use FreeNAS for media storage and VMware datastores.
For VMs, I'll be running (mostly Windows Server 2016 but perhaps Linux where appropriate):
- Two Plex Media Servers
- Two domain controllers
- One web server for Sickbeard, CouchPotato, Headphones, XDM, HTPC Manager
- One download server (running Deluge, most likely)
- One BDR server for Veeam (VM backups)
- One application server for miscellaneous things like network monitoring, Madsonic, DDNS
- vCenter
NAS01: Dell R710, 2x Xeon L5520, 48GB RAM, 4 NICs
- FreeNAS bare metal on Crucial MX300 SSD (in DVD caddy)
- Dell PERC H200 in IT mode - 6x8TB WD Red in RAIDZ2 (for media storage from SMB share)
- Two PCIe SSD adapters - 2x275GB Crucial MX300 SSDs in mirror, for use as ESXI datastore shared via NFS
- Two 2TB HDDs in RAID1 for location of Veeam repository and miscellaneous backups (hardware configs, etc). Shared via SMB. Synced to Backblaze B2.
- Two 275GB Crucial MX300 SSDs in RAID1, for use as ESXI datastore shared via NFS
- ESXi on Samsung 840 Evo 120GB SSD in PCIe adapter
- Dell PERC H200 in IT mode - VT-d passthrough to FreeNAS VM. This is to use up the drives from my previous setup. 4x4TB WD Red in RAIDZ1 (media storage), 4x3TB WD Red in RAIDZ1 (media storage), 2x4TB mirrored enterprise drives for ESXi datastores shared via NFS
- ESXi on SSD (TBD)
- Will only host one of the Plex servers - used for low power usage and capable compute resources
- Storage TBD - can hold up to four SSDs
- ESXi on SSD (TBD)
- Will only host one of the Plex servers - used for low power usage and capable compute resources
- No datastores
Much of these (for example, the actual servers themselves) are already purchased. The R710 is already up and running. But I don't have any of the virtualization set up yet, as I'm still hammering out the proper architecture. Which leads me to my questions:
- From what I've been reading, NFS seems like the best protocol to use for VMware datastores in this use case, rather that iSCSI. Is this correct, and does anyone have any suggestions?
- I'm trying to distribute datastore storage as much as possible to avoid single points of failure. However, I'm not obsessed with full hyper-convergence. For example, VMs on the R510 will run off of the datastores on the R710 and Synology, because FreeNAS is virtualized on the R510 and I don't want to open that can of worms. Given that I'm keeping datastores on both NASes, my impression is that something like Starwind or VMware vSAN is out of the question. Is that correct?
- I'm planning on using (good quality) consumer-grade SSDs here. Will these be able to handle the load of my VMs?
- The R210 II will be running an ESXi datastore, but it won't be running FreeNAS, thus no software RAID. It seems like I need a good quality RAID card, as this will have to be hardware RAID. Any recommendations?
Thanks!