FC: write performance is fine, read stuck at 200MB/s @ 4GB FC

[fips]

Hi Guys,

I have a strange issue, which I can't figure out myself.
My FreeNAS System:
Xeon E3-1230v3
16GB DDR3 ECC
6x 1TB SAS (RAID 10 via onboard controller)
1x 256 SSD (for testing puposes)
Qlogic QLE2460 single port 4Gbit

My Client System:
HP DL360G5
QLogic QLE2460 single port 4Gbit
Windows 7

I installed the last stable FreeNAS 9.3.1, configured FC according to the howto from aran kaspar (https://forums.freenas.org/index.php?threads/freenas-9-3-fc-fibre-channel-target-mode-das-san.27653/).

On my client system I was able to see disk via FC.
But Benchmark always showed the same result: Write performance is fine, read performance is max. 200MB/s measured via ATTO.

I even installed the latest nightlines via Gui -> same result.

Afterward I tried the trial version of OpenE DSS 7 and the result was as expected, maximum write and read performance on the same infrastructure.

I really stuck with this topic, I dont know where should I check FreeNAS or what could I try else to get it perform as it should.

Pleas help me.

Thanks
Steve

 

[Letni]

Hi Guys,

I have a strange issue, which I can't figure out myself.
My FreeNAS System:
Xeon E3-1230v3
16GB DDR3 ECC
6x 1TB SAS (RAID 10 via onboard controller)
1x 256 SSD (for testing puposes)
Qlogic QLE2460 single port 4Gbit

My Client System:
HP DL360G5
QLogic QLE2460 single port 4Gbit
Windows 7

I installed the last stable FreeNAS 9.3.1, configured FC according to the howto from aran kaspar (https://forums.freenas.org/index.php?threads/freenas-9-3-fc-fibre-channel-target-mode-das-san.27653/).

On my client system I was able to see disk via FC.
But Benchmark always showed the same result: Write performance is fine, read performance is max. 200MB/s measured via ATTO.

I even installed the latest nightlines via Gui -> same result.

Afterward I tried the trial version of OpenE DSS 7 and the result was as expected, maximum write and read performance on the same infrastructure.

I really stuck with this topic, I dont know where should I check FreeNAS or what could I try else to get it perform as it should.

Pleas help me.

Thanks
Steve

Steve,

What HBA (Qlogic) firmware are you using? (Both in Freenas and in your Windows server)

What type of disks are in the pool where the LUNs are created that are presented out via FC?

Are you using ZVols or file extents as the basis of your LUNs?

are you usung hardware RAID and not RAIDZ?

 

[Letni]

Another thought, 200 MB/sec seems like a link limit.. I wonder if on the GBICs, the outbound laser negotiated at 4 Gbit and the inbound laser at 2GBit (if that's even possible).. Can you check lights on both HBAs and maybe even swap ends (reverse) on the FC cable?

 

[fips]

Firmware is 5.07 (newest).
the extent is directly the raid and the ssd.
/dev/da0 and /dev/ada0

Raid is that onboard SAS Raid.
But it doesn't matter because I have that problems with that SSD (connected via SATA) too.

Link on both cards is set to 4Gb.
The problem is not the card or cables or firmware.

Via Open-E FC work on full speed, as you can see on my Screenshot OpenEssd.jpg above.

 

[mangelot]

I can confirm this same issue with latest build of 9.3

Using 2x Qlogic 2464 HBA's (1x freenas fc target and 1x client)

I also switched the client HBA with an Emulex LPe1150 with the same result ?
Changed fiber cable to OM1 and OM3 types with same results.
All cards/ports have been manually set to 4Gbps with nu result.
All HBA's using latest firmware.


writing to disk is 400MB and reading is 200MB.

Test script 40GB RRD time 13:10 until 13:20:

clear
echo "writing 40000x 1MB = 1GB"
dd of=testfile if=/dev/zero bs=1M count=40000
echo ""
echo "reading 40000x 1MB = 1GB"
dd of=/dev/zero if=testfile bs=1M count=40000
echo ""
echo "reading 40000x 1MB = 1GB (without cache)"
sudo /sbin/sysctl -w vm.drop_caches=3
echo ""
dd of=/dev/zero if=testfile bs=1M count=40000
rm -f testfile

Test script (multiple times) 10GB RRD time 13:40 until 13:55:

clear
echo "writing 10000x 1MB = 1GB"
dd of=testfile if=/dev/zero bs=1M count=10000
echo ""
echo "reading 10000x 1MB = 1GB"
dd of=/dev/zero if=testfile bs=1M count=10000
echo ""
echo "reading 10000x 1MB = 1GB (without cache)"
sudo /sbin/sysctl -w vm.drop_caches=3
echo ""
dd of=/dev/zero if=testfile bs=1M count=10000
rm -f testfile

It really seems to me there is an limit on 200MB/s
can seem to find out, what is causing this behaviour!

any help?

 

[mangelot]

I can also confirm this on version latest release 9.10

 

[iqarmy]

Hi Guys,

Not sure if the right place to ask this but I'm using freenas with Qlogic 2432 on a HP Gen8 microserver and a Windows 10 as a client. Link is solid orange on both machines yet I don't know how I can get the iSCSI target disk on the windows OS system. Can some one please help me out with this?