10GB Considerations and Performance Expectations

Shankage

Explorer
Joined
Jun 21, 2017
Messages
79
Hi all,

I run two instances of FreeNAS/TrueNAS in my envrionment, both backed by gigabit links. I do have 10gbit switches in place and merely need to upgrade the nics on the hosts to provide a 10gbit backing. Before doing so I wanted to have a discussion on the two systems, the current performance and best ways to test, plus what their expected current and 10gbit performance should be.

The systems are both used for VM storage (which I need to improve) and media (which doesn't matter as much).

System 1
Baremetal R710

  • CPU: Dual E5520's
  • 96GB RAM
  • 1 NIC for management access, 2 nics for iSCSI and NFS
  • 6 x 6tb WD RED 5400s (mirrored vdevs)
  • FreeNAS-11.3-U5
  • No slog of L2arc
Quick test run directly on the appliance

Code:
root@SAN:~ # dd if=/dev/zero of=ddtest bs=16K count=4M
4194304+0 records in
4194304+0 records out
68719476736 bytes transferred in 206.754866 secs (332371751 bytes/sec)


System 2
R730XD with virtualized FreeNAS on ESXi7

  • CPU E5-2640 v3 (VM given 4 CPUs)
  • 64GB RAM
  • HBA passthrough to VM
  • 8 x 1tb 10k SAS drives (mirrored vdevs /w 2 x 186gb SSDs for slog)
  • FreeNAS-11.2-U8
Same test above run on this system
Code:
root@VMs-NAS[~]# dd if=/dev/zero of=ddtest bs=16K count=4M
4194304+0 records in
4194304+0 records out
68719476736 bytes transferred in 105.472846 secs (651537146 bytes/sec)


Before plunging into 10gbit with either of these systems and potentially buying SSDs for one of them, I first want to ensure I am thoroughly testing both and getting the most out of either of these systems now, understand what performance I should be seeing and ensure I will see a difference with 10gbit.

Happy for any input, additional tests, recommendations!

Thanks!
 
Top