as near as I could tell, I was unable to find any adjustable cache settings on the controller, at all... Nothing in the config menus talked about write cache etc...
FreeNAS Server:
Older MN2-LR board + process (not sure what it is)
PCIe LSI SAS2008 Controller - Connects 1, 120GB Log SSD, 1, 1.5TB Disk, and 1, 128GB cache SSD.
6 On-board SATAII ports Connect another 5, 1.5TB Disks, and the other 120GB Log disk
8 GB RAM - Max supported on this board (4x2)
Single on-board NIC dedicated to Storage traffic
Single on-board NIC deidcated to management
The test utlizes IOMeter and VMware ESX from 2 ESX Hosts, each with their own LUN and set of VM's.
ie. VMSERVER1 = 4 VM's on LUN1 and VMSERVER2 = 4 VM's on LUN2.
Each VM has 2GB of RAM. IOMeter configured as follows:
Test File Size: 7.5GB
Disk Queue Depth: 3
Write - 100%
Random - 100%
IO Transfer Size - 4K
Each test ran for 30 minutes. The first was thrown out due to caching / data building processes.
Surely some formatting will be lost below...
Code:
IOps MBps Avg. Write Response Time Bytes Written Write I/Os
IR Firmware - NFS
518.116029 2.023891 46.317555 3810344960 930260
537.77005 2.100664 44.590117 3955040256 965586
634.490623 2.478479 37.789613 4666241024 1139219
632.681712 2.471413 37.932419 4652875776 1135956
IT Firmware - NFS
674.84783 2.636124 35.561073 4963000320 1211670
629.366709 2.458464 38.108609 4628602880 1130030
661.849424 2.585349 36.258458 4867469312 1188347
658.876941 2.573738 36.413746 4845559808 1182998
IT Firmware - ISCSI
683.656827 2.670534 35.103562 5027811328 1227493
697.390624 2.724182 34.402822 5128740864 1252134
685.238942 2.676715 35.022641 5039288320 1230295
660.814984 2.581309 36.317034 4859723776 1186456
Result:
IT Firmware is slightly faster than the IR firmware, and iSCSI is slightly faster than NFS, though the differences is small.
One thing I noticed is that when using NFS, zpool iostat shows upto 80-90MB writes, while when using iSCSI it was only around 20-25MB writes, though the VM's themselves were getting about the same IO - kind of unexpected.