I've got two volumes on the same FreeNAS 11 box that use HDDs for data storage:
-- #1 is a 2 drive mirror using 5 year old WD Se 4 TB HDDs (WD Se was the predecessor to WD Red Pro)
-- #2 is a 2x2 mirror (like a RAID10) using brand new WD Gold 10 TB HDDs
Both are set up the same way with full disk zvols that are used as ESXi datastores via iSCSI.
#1 is 83% full (usually I keep this under 50%) and #2 is 8% full. #1 is significantly fragmented, #2 has almost no fragmentation as I just set it up a few days ago.
When reading large files (> 1 GB each) off both volumes, I'm finding that the 2 drive mirror is at least 25% faster than the 2x2 mirror when the reads are *not* coming from the ARC. IE, the actual disk read performance is being tested here because I'm reading seldomly accessed files. I've tested this two ways:
1) From the ESXi console running "dd" commands that output to /dev/null
2) From a Win 2016 VM that has a VMDK on each volume, copying files from each disk to a RAM disk
In both cases, I'm monitoring the disk performance on the FreeNAS box using "zpool iostat -v 1".
FWIW, the performance on the 2x2 mirror seems to go up and down quite a bit while the performance on the 2 drive mirror is more consistent / less jumpy. I've also noticed with the zpool iostat output that each drive in the 2x2 mirror is doing 70% more read ops per second than each drive in the 2 drive mirror.
Any idea why the 2x2 mirror is so slow?
(System: FreeNAS 11.0-U3 on Supermicro X9SRE-F with E5-1650v2, 128GB RAM, mirrored USB flash for boot disk, 3 pools, each pool has 1-2 SLOG SSDs)
-- #1 is a 2 drive mirror using 5 year old WD Se 4 TB HDDs (WD Se was the predecessor to WD Red Pro)
-- #2 is a 2x2 mirror (like a RAID10) using brand new WD Gold 10 TB HDDs
Both are set up the same way with full disk zvols that are used as ESXi datastores via iSCSI.
#1 is 83% full (usually I keep this under 50%) and #2 is 8% full. #1 is significantly fragmented, #2 has almost no fragmentation as I just set it up a few days ago.
When reading large files (> 1 GB each) off both volumes, I'm finding that the 2 drive mirror is at least 25% faster than the 2x2 mirror when the reads are *not* coming from the ARC. IE, the actual disk read performance is being tested here because I'm reading seldomly accessed files. I've tested this two ways:
1) From the ESXi console running "dd" commands that output to /dev/null
2) From a Win 2016 VM that has a VMDK on each volume, copying files from each disk to a RAM disk
In both cases, I'm monitoring the disk performance on the FreeNAS box using "zpool iostat -v 1".
FWIW, the performance on the 2x2 mirror seems to go up and down quite a bit while the performance on the 2 drive mirror is more consistent / less jumpy. I've also noticed with the zpool iostat output that each drive in the 2x2 mirror is doing 70% more read ops per second than each drive in the 2 drive mirror.
Any idea why the 2x2 mirror is so slow?
(System: FreeNAS 11.0-U3 on Supermicro X9SRE-F with E5-1650v2, 128GB RAM, mirrored USB flash for boot disk, 3 pools, each pool has 1-2 SLOG SSDs)