Speed of pool limited regardless of number of disks.

[helloha]

I am testing some older disks and I ran into something peculiar. The performance of the pool is limited to about 200 MB/s. Regardless if I have 2 disks in stripe, or 4.

I have 4 identical disks. Below is the speed of the individual disk.

1 DISK - STRIPE

Code:

10GB    10737418240 bytes transferred in 79.783709 secs     (134581588 bytes/sec)
50GB    53687091200 bytes transferred in 628.541031 secs     (85415412 bytes/sec)

Here are the speeds of the disks in stripe from 2-4 disks. As you can see the performance is almost the same.

2 DISK - STRIPE

Code:

10GB    10737418240 bytes transferred in 60.563390 secs     (177292226 bytes/sec)
50GB    53687091200 bytes transferred in 342.066161 secs     (156949436 bytes/sec)
150GB    161061273600 bytes transferred in 1087.945603 secs     (148041661 bytes/sec)

3 DISK - STRIPE

Code:

10GB    10737418240 bytes transferred in 57.790275 secs     (185799743 bytes/sec)
50GB    53687091200 bytes transferred in 341.488987 secs     (157214707 bytes/sec)

4 DISK - STRIPE

Code:

10GB    10737418240 bytes transferred in 56.669852 secs     (189473201 bytes/sec)
50GB    53687091200 bytes transferred in 348.917533 secs     (153867565 bytes/sec)

These are the iostat readings:

Code:

WD-500GB                                        4.49G   916G      0  1.52K      0   194M
  gptid/40218c6a-60a0-11e5-ab69-001f2954783c  2.24G   458G      0    782      0  97.8M
  gptid/c58673a9-60fc-11e5-b636-001f2954783c  2.24G   458G      0    772      0  96.6M

WD-500GB                                       2.00G  1.35T      0  1.57K      0   198M
  gptid/264cd0c9-6103-11e5-b636-001f2954783c   683M   459G      0    534      0  65.8M
  gptid/26b5cb62-6103-11e5-b636-001f2954783c   683M   459G      0    535      0  66.0M
  gptid/271ab28f-6103-11e5-b636-001f2954783c   684M   459G      0    535      0  65.8M

WD-500GB                                        1.59G  1.80T      0  1.55K      0   196M
  gptid/274ee5d9-6107-11e5-b636-001f2954783c   407M   460G      0    402      0  48.9M
  gptid/27ba59a9-6107-11e5-b636-001f2954783c   407M   460G      0    402      0  48.9M
  gptid/2821ad00-6107-11e5-b636-001f2954783c   406M   460G      0    391      0  48.9M
  gptid/288dbbb1-6107-11e5-b636-001f2954783c   406M   460G      0    391      0  48.9M

All disks are connected to the system via a Dell Perc h310. I also tested different pools in stripe and reached speeds of 500 MB/s. Also have an SSD on a SATA2 port that reaches 230MB/s.

Anyone know what could be causing this?

Cheers!
K.

 

[SweetAndLow]

Your ssd was 230 bits per second or bytes per second? Also provide your hardware and freenas version.

 

[helloha]

Harware is the same as in my autograph, and yes megabyte.

Supermicro X7DWN+ - Dual Xeon 5130 - 64GB Ram - 8x 4TB Seagate ST4000DM000 - 2x 4TB HGST HDN724040ALE640 - 1x 256GB Crucial_CT256MX100SSD1- 6x HGST 3TB - TDK LoR TF30 USB 3.0 PMAP (boot) - 2x Dell H310 SAS/SATA Controller with H16 IT firmware - HP360T Gb NIC - Supermicro SC825 2U Chassis 920Watt Platinum PSU.

Running: FreeNAS-9.3-STABLE-201502271818

 

[helloha]

Has anyone ever encountered lower performance from using different SAS cables?

I think I narrowed the issue down to the Mini SAS to SATA cable that I'm using.

I have two Dell H310 cards with the same IT firmware.

On each card I use 2 SFF8087 connectors splitting out into 2x4 SATA connectors.

I have 2 SFF8087 cables from the Molex brand, on the connector label it says: "Molex 79576-3004". And another set from a different brand, on the connector label it says: "Controller Based CBL-00079-01-A-R Adaptec REV A" this set also has the grey sideband cable.

When I use the Molex set to go to the drives performance is as described above. It seems to be capped at 100 MB/s.

When switching to the Adaptec set with the sideband cable, performance goes up dramatically.

Anyone ever heard of such a thing? Could it be that the slow cable is missing shielding and is therefore slower?

Note: on the individual split out sata cables it all says "molex"... So I'm thinking they are the same, just branded differently.

Cheers!
K.