10GB line speeds - write maximum?

[philhu]

Hi all, I have a 9.10 Freenas, that was connected to my pc over a 1G NIC card, to a switch to the basement switch to the NAS

Receiving single large files was at 112 mB/s, sending was 104-112 mB/s. Solid
Adding a second transfer, it splits the net traffic between them, etc, so it is maxxed out, as 112 mB/s is the GIG totally saturated.

Switch to a 10GBe card, with direct connect to downstairs switch.

Receiving a single large file from NAS is at 413 mB/s. Sending to NAS sits at 100-176 mB/s. So I am assuming it is overhead of allocating blocks to store file etc.

Extra data to bolster this is that if I start a 2,3,4 large file transfer to the NAS, the first stays at about 120-140, but the others are still getting 35-70!

This leads me to believe that the bottleneck is not network, but time required to allocate disk space in ZFS. I am not complaining, the cat7 wire and 2 10GBE nics were pretty cheap, so I am pleased by the throughput increases

 

[SweetAndLow]

Something else is wrong, you should be able to max out your pool with 10gig. What is the write performance of your pool locally?

Sent from my Nexus 5X using Tapatalk

 

[depasseg]

In addition to what SnL mentioned, what NIC are you using? Have you done an iperf test to check that the network link is OK? And have you tested the read and write speed of the client that is sending/receiving data.

 

[philhu]

hi

Link is intel x540-t2 on both sides. I will need to read up on the iperf tool. The data is using Windows standard folder/file moves to the smb share on FreeNAS

How do I test 'write performance on my pool locally'? I would love to get to the bottom of this

Thanks

 

[Mlovelace]

The Intel 10Gbe cards definitely need tuning in freeNAS. Check out this thread https://forums.freenas.org/index.ph...rdware-smb-cifs-bottleneck.42641/#post-277350

 

[philhu]

Ok, I did tune the windows side, and my windows rcv speed went from 200 to 400, and send was around 100-120 up to almost 180 now, as shown in my OP.

So I will attempt to tune my freenas side tonight

 

[philhu]

ok, did some testing.....Not great, 3.2 GB/s using:
dd if=/dev/zero of=testfile bs=1048576


Here is a wierd one...I was copying a large file during the test, I wanted to see if the copy slowed down. It didn't, but here is the wierd part, as you see from the windows screen shot, when I killed the dd test, the file write went way up, over 400 MB/s, next file started, it went back down.

See attached