Truenas and Intel 10GB X540-AT2 - Speed only 3-5MB

[truenas_newbee]

Hi, I installed Truenas in a VM with Del Perc H310 in IT mode and passthrough. I have two ESXI hosts (2 x Dell R720XD). When i copy between the host (servers) the speed ok. I added the Truenas as an NFS datastore and when try to copy/from to the Truenas to the ESXi hosts I get speed of 3-5MB a second which is pethatic when considering this is 10GB NIC.

The NIC is also passthrough and the ESXi is connected directly through a secondary NIC so there is no switch or anything else to create speed issue. A direct link between the two.

I did troubleshooting within my capacity by copying things around and managed to narrow it down to the Truenas. This speed issue only occurs with Truenas.
I am not sure what to check next or what to do. Any pointers appriciated?

 

[jgreco]

Where did you get the card from, and is it showing link at 10Gbps?

There are a lot of fake Intel cards out there. Pull the card and validate the Yottamark. If it doesn't validate, or doesn't have a Yottamark, it's likely fake and isn't expected to work well.

Generally speaking, copper 10G is kinda tragic and there are more things to go wrong. If you can, get rid of the copper and go to fiber. There is a great 10 gig networking primer elsewhere on these forums in the resources that list a variety of things that are known to work swimmingly well with less drama.

 

[truenas_newbee]

Where did you get the card from, and is it showing link at 10Gbps?

There are a lot of fake Intel cards out there. Pull the card and validate the Yottamark. If it doesn't validate, or doesn't have a Yottamark, it's likely fake and isn't expected to work well.

Generally speaking, copper 10G is kinda tragic and there are more things to go wrong. If you can, get rid of the copper and go to fiber. There is a great 10 gig networking primer elsewhere on these forums in the resources that list a variety of things that are known to work swimmingly well with less drama.

Thank you for the reply.

Thats a good point, but the card was used in QNAP and the speed was ok. I also installed Freenas and tested it, the speed seems bit better. on copying from the Freenas to ESXI it hit 600MB.

I really dont want to use another raid software and would love to get a solution to this issue.

 

[jgreco]

See, but you're not giving me much to work with. If the hardware is actually okay, which you still need to validate, then the other usual problem would be that you're trying to do ESXi block storage on RAIDZ, probably on a datastore that is fragmented and/or relatively full.

More on 10G networks:

https://www.ixsystems.com/community/threads/10-gig-networking-primer.25749/

More on how to build block storage:

https://www.ixsystems.com/community/threads/the-path-to-success-for-block-storage.81165/

If you want more than general help, please follow the Forum Rules, conveniently linked at the top of every page in red, and provide significantly more detailed information about your hardware, configuration, etc., because the devil is in the details.

 

[truenas_newbee]

See, but you're not giving me much to work with. If the hardware is actually okay, which you still need to validate, then the other usual problem would be that you're trying to do ESXi block storage on RAIDZ, probably on a datastore that is fragmented and/or relatively full.

More on 10G networks:

https://www.ixsystems.com/community/threads/10-gig-networking-primer.25749/

More on how to build block storage:

https://www.ixsystems.com/community/threads/the-path-to-success-for-block-storage.81165/

If you want more than general help, please follow the Forum Rules, conveniently linked at the top of every page in red, and provide significantly more detailed information about your hardware, configuration, etc., because the devil is in the details.

I really appriciate your efferot int trying to help me and in no way expecting you to do the running for me. I just dont know what information you want/need?
I hope the following helps:
as per the image copying between the ESXi servers, is no where near 10GB speed but its bearable. However, when i use anything to do with Truenas/Freenas the speed drops to the speed unberable. I tried both truenas on baremetal and truenas as a VM did not make a difference.

This is a new install, the storage i used for testing is a single drive tried both SSD and HDD. wanted to test the speed and make sure its working ok before doing it properly. Therefore the SSD and/or the HDD is empty to start with. Also tried using raidz2 with the rest of the drives but the results were the same.

 

[truenas_newbee]

Update: Installed Unraid on bare metal and hit 2Gb a second on the same hardware and NIC. so its definitly truenas not happy about something. Can anyone give any suggestions to try before being forced to move to unraid?