read speed is slow

[bones0008]

Hi,

My freenas is build on a DIY box with 64GB RAM, and AVAGO MR9361-8i RAID card (1G cache) with 24 hard drive RAID-6, space is 200TB. No cache drive and log drive. 10Gb Ethernet port.

Free nas version is 11.3-U5. Compression is off. Single volume.

I use dd command to test the read and write speed on it with 100GB file. Speed looks good.

write speed is 1825053492 bytes/sec

read speed is 1772204604 bytes/sec

(check the attachment)

But I use NFS share connect from my test box (with 10Gb Ethernet), CENTOS 7.7 NFS version 3

write speed is 1.08GB/s

read speed is 318MB/s

I don't know why the read speed is slow. Because the RAM is too small? Any one has idea about this?

 

[SweetAndLow]

Start with iperf and test going both directions. Then do a dd test over the wire both directions. See what info that gives you.

 

[sretalla]

AVAGO MR9361-8i RAID card (1G cache) with 24 hard drive RAID-6

I can see 2 problems here:

This is a bad card to be using. Have you created RAID6 across the 24 disks using the card's RAID? (https://www.truenas.com/community/t...s-and-why-cant-i-use-a-raid-controller.81931/)

Also, if you have correctly allowed FreeNAS to manage the RAID of the 24 disks (and presumably have a RAIDZ2 pool with a single VDEV), that's way wider than recommended for RAIDZ2 (usually 12 maximum).

 

[bones0008]

Start with iperf and test going both directions. Then do a dd test over the wire both directions. See what info that gives you.

sorry for the late reply, quite busy.

that is the number I tested on client, the iperf connect speed is 9.86Gb/s.

 

[bones0008]

I can see 2 problems here:

This is a bad card to be using. Have you created RAID6 across the 24 disks using the card's RAID? (https://www.truenas.com/community/t...s-and-why-cant-i-use-a-raid-controller.81931/)

Also, if you have correctly allowed FreeNAS to manage the RAID of the 24 disks (and presumably have a RAIDZ2 pool with a single VDEV), that's way wider than recommended for RAIDZ2 (usually 12 maximum).

Thanks for your reply,

Yes, I created RAID6 across 24 disks under the RAID card.

I will try to use JOB mode on that card and try again.

 

[SweetAndLow]

Thanks for your reply,

Yes, I created RAID6 across 24 disks under the RAID card.

I will try to use JOB mode on that card and try again.

Jbod isn't enough. You need hba