I am getting incredibly slow read speeds from magnetic drives with Truenas. Shouldn't I see better than 314-400MB/s read performance from 6vdev mirrors with a total of 12 3TB drives? I have tested different drives, The only thing that seems to not exhibit this problem are SSDs when testing with SSDs I get good read and write performance. 10gbps line rate in both directions with a single 4 drive raidz1 vdev.
I have tested different HBAs and different cables (I am using LSI 9200-16e in the pictures, but have also tested this with LSI 9206-16e) both had the same results
I have reinstalled truenas from scratch (vanilla) problem persists.
I have tested all of the drives individually with hdd sentential -- none of the drives had any issues and read and write speeds were ~140MB/s average per drive (tested all at the same time in aggregate I saw 2GB/s at peak.)
I have tested this using fio and smb. If I'm reading from ARC things are line rate 10gbps otherwise they read speeds are 1/3rd write speeds or worse.
I've created smaller pools and get about the same results read speed wise.
System specs:
dual e5-2667 CPUs
128GB ddr3 ecc memory
LSI 9200-16e
LSI 9206-16e
separate power supplies for drives (tested 2 different PSUs)
Read speeds:
Write Speeds are fine:
I have tested different HBAs and different cables (I am using LSI 9200-16e in the pictures, but have also tested this with LSI 9206-16e) both had the same results
I have reinstalled truenas from scratch (vanilla) problem persists.
I have tested all of the drives individually with hdd sentential -- none of the drives had any issues and read and write speeds were ~140MB/s average per drive (tested all at the same time in aggregate I saw 2GB/s at peak.)
I have tested this using fio and smb. If I'm reading from ARC things are line rate 10gbps otherwise they read speeds are 1/3rd write speeds or worse.
I've created smaller pools and get about the same results read speed wise.
System specs:
dual e5-2667 CPUs
128GB ddr3 ecc memory
LSI 9200-16e
LSI 9206-16e
separate power supplies for drives (tested 2 different PSUs)
Read speeds:
Write Speeds are fine: