New drives, Solnet-array-testing results. Cause for alarm?

[JohnDigital]

Hello everyone, thank you for reading. I bought 11 new 5TB drives and they have passed initial SMART testing. Now I am running solnet-array-test-v2 on them for a couple days. There isn't any pool created and these drives are not otherwise in use, the drives are identical. Two of the drives are noticeably slower than the other nine. Eight drives da0-da7 are on a M1015 in IT mode and all are testing at or above average speeds, and three are on the motherboard of a Dell 1900 PE server (16GB ECC, Dual XEON) which the two slow ones are connected ada0-ada2 (the two abnormal drives are ada0 and ada1), I would think that this is itself the issue, but the third drive connected to the motherboard is in line or above with the others connected to the LSI M1015. See results, test still running..

Code:

Array's average speed is 163.118 MB/sec per disk

Disk    Disk Size  MB/sec %ofAvg
------- ---------- ------ ------
da0      4769307MB    174    107
da1      4769307MB    157     96
da2      4769307MB    168    103
da3      4769307MB    169    103
da4      4769307MB    161     99
da5      4769307MB    174    107
da6      4769307MB    162     99
da7      4769307MB    158     97
ada0     4769307MB    159     97
ada1     4769307MB    157     97
ada2     4769307MB    155     95

Performing initial parallel array read
Wed Mar  2 11:27:12 PST 2016
The disk da0 appears to be 4769307 MB.      
Disk is reading at about 165 MB/sec        
This suggests that this pass may take around 482 minutes
                                           
                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
da0      4769307MB    174    164     95
da1      4769307MB    157    156     99
da2      4769307MB    168    162     97
da3      4769307MB    169    164     97
da4      4769307MB    161    158     98
da5      4769307MB    174    166     95
da6      4769307MB    162    159     99
da7      4769307MB    158    156     99
ada0     4769307MB    159     94     59 --SLOW--
ada1     4769307MB    157     94     60 --SLOW--
ada2     4769307MB    155    156    100

Awaiting completion: initial parallel array read
Thu Mar  3 00:58:59 PST 2016
Completed: initial parallel array read

Disk's average time is 38387 seconds per disk

Disk    Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
da0         5000981078016   34259     89 ++FAST++
da1         5000981078016   37023     96
da2         5000981078016   35288     92 ++FAST++
da3         5000981078016   35566     93
da4         5000981078016   36762     96
da5         5000981078016   34683     90 ++FAST++
da6         5000981078016   36160     94
da7         5000981078016   37055     97
ada0        5000981078016   48707    127 --SLOW--
ada1        5000981078016   48707    127 --SLOW--
ada2        5000981078016   38042     99

Performing initial parallel seek-stress array read
Thu Mar  3 00:58:59 PST 2016
The disk da0 appears to be 4769307 MB.      
Disk is reading at about 164 MB/sec        
This suggests that this pass may take around 485 minutes
                                           
                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
da0      4769307MB    174    164     94
da1      4769307MB    157    145     92
da2      4769307MB    168    157     93
da3      4769307MB    169    160     95
da4      4769307MB    161    153     95
da5      4769307MB    174    167     96
da6      4769307MB    162    150     93
da7      4769307MB    158    153     97
ada0     4769307MB    159     68     43
ada1     4769307MB    157     66     42
ada2     4769307MB    155    127     82

Awaiting completion: initial parallel seek-stress array read

Has anyone experienced this before? Is this normal or expected? I have one drive already going back on an RMA, and want to expedite that process.

 

[jgreco]

Switch the drives around and find out. It *could* be normal based on maybe something like NCQ being set differently. The best time to experiment and identify quirks is now, of course.

 

[JohnDigital]

Thanks for prompt reply. That was already going to be the plan once this parallel seek-stress test is done. I will also look into the Command Queue and see if they are different from the others.

 

[JohnDigital]

NCQ is 32 on all drives. I accidentally killed my terminal, so I switched the drives around and ill let it run another day or two and see what we get. Thanks.