Sanity Check, am I really upgrading?

[Mehnock]

I've been a user of FreeNAS and now TrueNAS for at least 6 years. I currently have 2 TrueNAS Core boxes serving iSCSI datastores (2 paths per datastore/ 2 VLAN SAN) to 4 ESX hosts running 7x. In total, I have 15 VMs running Windows server (various versions) doing server stuff and 8 Windows 10 VMs for remote users.

The two SAN boxes are all SSDs (using desktop grade SSDs), one has all WD Red SA500 SSDs and the other has Crucial MX500 SSDs. I have two arrays of inexpensive SSDs (RAIS) LOL!

I am building a new box as follows and my question is if this will theoretically be an improvement.

• HP ProLiant DL380 G9 Dual E5-2670v3 256GB RAM (should I remove half the RAM, old boxes have 32 and 64Gb and they are getting 93% and 100% Arc Hit Ratios)
• HP 3PAR M6710 DAS connected via a LSI SAS9300-8E (the DAS is currently connected to one of the old boxes using a LSI SAS9207-8e
• 24 HP SAN 1.2Tb 10K drives
• One Intel DC P3700 400Gb NVMe drive for SLOG
• 3x 10Gbps ports on three VLAN as SAN to connect to my 4 ESX Hosts (Multipath to the Datastore)

I'm "downgrading" from desktop grade SSDs to server grade SAS drives. I'm gaining dual ports on the drives as the current SSDs are single port, I'm cabling both HBA ports to the DAS and they will be used plus I'm adding a SLOG that should pickup the slack.

Should this system be faster than my current systems?

 

[jgreco]

If you're getting 93%/100% ARC hits on 32 and 64GB of RAM, you might make things better by going to 128GB RAM and then adding in some L2ARC, like two 256GB NVMe SSD's. The benefit is that if you are already pretty close to your working set size, doubling the RAM will put you solidly in a zone where L2ARC is likely to pick off the occasional miss, and is large enough that ARC should be able to collect good data on MFU for L2ARC evicts. This means that reads from HDD pool would happen very rarely and things should be very fast for read.

Writes may not be as fast as with the SSD however. If you can keep a good bit of space free on the HDD's, maybe that isn't a problem.

 

[Mehnock]

Thanks @jgreco, I am planning for 60% usage on the Storage and my yearly data growth is not too high.

So you think the SLOG + the Dual port SAS drives won't be faster than my current all SSD boxes?

 

[jgreco]

Synchronous writes via SLOG are always slower than asynchronous writes. Whether or not X happens to be faster than Y is hard to say, as multiple factors come into play.

Write speeds are a function of fragmentation and occupancy. If you had a 10% full pool, that's likely to have write speeds that might be faster than SSD, but once you're up at 50% you will eventually hit a steady state that is rather slow, maybe unexpectedly so.

Index of /files/freenas

fragmentation/delphix-steady-state.png

See also

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com