10 GbE not reaching full potential

[TurboStore]

I am having trouble reaching the full 10 Gigabit of transfer throughput.

The drives on the server would be easily able to do that.

About the Server:
OS: FreeNAS 11.3
Network: Intel x550-T2 card
Storage: 4x NVMe (PCIe 3.0) SSDs in RaidZ1

Pool write test:

Code:

root@********[~]# dd if=/dev/zero of=/mnt/****/************/test.file bs=4096k count=1000
1000+0 records in
1000+0 records out
4194304000 bytes transferred in 2.128823 secs (1970245185 bytes/sec)

Results in 1.9702 Gbytes per second!

Pool read test:

Code:

root@********[~]# dd if=/mnt/****/************/test.file of=/dev/zero bs=4096k count=1000
1000+0 records in
1000+0 records out
4194304000 bytes transferred in 0.605741 secs (6924250990 bytes/sec)

Results in 6.924 Gbytes per second!

That should proof that the server is more than capable of saturating a 10 Gbit connection.

My networking equipment:
The FreeNAS and my computer both have official Intel x550-T2 Ethernet cards and are directly connected via a Cat6a cable.

I enabled Jumbo Packets of 9014 Bytes both systems.

On my computer I disabled "Interrupt Moderation" which resulted in a big performance boost.

I tried the config from the 45 drives article:

How to Tune a NAS for Direct-from-Server Editing of 5K Video

Typically, if one sets up a 10GbE network, and connects a Storinator running NAS or Server OS to a fast workstation, one will see out-of-the-box transf

www.45drives.com

On my system I set the Receive Buffers to 4096 and Transmit Buffers to 16384

On FreeNAS

Code:

kern.ipc.maxsockbuf=16777216
net.inet.ip.intr_queue_maxlen=2048
net.inet.tcp.recvbuf_inc=524288
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.recvspace=4194304
net.inet.tcp.sendbuf_inc=32768
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.sendspace=2097152
net.route.netisr_maxqlen=2048

These tunables on the FreeNAS made no difference.


Current performance:
SMB ( CIFS )

• Read: 5.9 to 6 Gbps
• Write: 6.8 Gbps

iSCSI is similar.

FTP ( plain, unencrypted )

• Read: 7.1 Gpbs
• Write: 7 Gbps

Any help on how to configure FreeNAS to reach 10 gigabit is appreciated!