I've been struggling with hardware design for my new nas (specifically SLOG and performance optimization) so I figured I'd ask for general feedback and opinions.
It is for esxi virtual machine datastores. I run a few statistics and machine learning programs through daily data files. The data is read-only on a remote server. A copy is brought over the network to an ESXi host where a VM runs through the data, generates an output file (which will be saved on the new TrueNAS machine), the data is then discarded and a new day/week/month's data is brought in and the process repeats.
My issue is that my old ESXi environment had local storage on each ESXi host, my old machines were old and slow (ddr3 and pcie gen 2.0), I didn't record any machine/datastore/network I/O numbers, I also upgraded my network to 10GbE... So I have no clue what a VM will want to do to its datastores on this new TrueNAS machine, that will be on a 10GbE network.
The new ESXi hosts all have plenty of memory and will be much faster, but I still have no clue what the VMs will do back to their datastores on TrueNAS --or how that relates to what hardware I want for slog, do I want a "small file" or "metadata" space outside of the spinny drives? What do I want to do about the small, random I/O from the VMs going back to TrueNAS?
Or --should I simplify my life a bunch and build a dedicated, bare-metal machine for this work?
Thanks.
It is for esxi virtual machine datastores. I run a few statistics and machine learning programs through daily data files. The data is read-only on a remote server. A copy is brought over the network to an ESXi host where a VM runs through the data, generates an output file (which will be saved on the new TrueNAS machine), the data is then discarded and a new day/week/month's data is brought in and the process repeats.
My issue is that my old ESXi environment had local storage on each ESXi host, my old machines were old and slow (ddr3 and pcie gen 2.0), I didn't record any machine/datastore/network I/O numbers, I also upgraded my network to 10GbE... So I have no clue what a VM will want to do to its datastores on this new TrueNAS machine, that will be on a 10GbE network.
The new ESXi hosts all have plenty of memory and will be much faster, but I still have no clue what the VMs will do back to their datastores on TrueNAS --or how that relates to what hardware I want for slog, do I want a "small file" or "metadata" space outside of the spinny drives? What do I want to do about the small, random I/O from the VMs going back to TrueNAS?
Or --should I simplify my life a bunch and build a dedicated, bare-metal machine for this work?
Thanks.
