Intel x540t2 wont run past 300mbps

[oumpa31]

I installed an Intel X540t2 into my build listed below. I copied some of my movies from my win 10 pro tower that has ssd's in it with reads of over 500 to freenas and the fastest speed i am seeing was 300. When i had the build below running windows 10 pro i was seeing transfer reads and writes of over 600 from the Now freeNAS machine to my main tower ssd. When going from and to a ram disk i was seeing 1 gig on average. I'm not sure if im missing something in the set up of the card. Everything i read said it will just work.

 

[dlavigne]

What's the output of ifconfig?

Also, which FreeNAS version (from System -> Information)?

 

[oumpa31]

Sorry forgot to put that. I'm running 11.1
this is all i was able to get from my ifconfig in shell. i'm not exzactly sure how to scroll up to get the whole thing.

 

[dlavigne]

We need the exact version (eg U4, U6, something else?) plus the full output (the important stuff was at the top).

 

[jgreco]

We need the exact version (eg U4, U6, something else?) plus the full output (the important stuff was at the top).

Tsk tsk. Not helpful!

Run:

ifconfig -a | more

or

ifconfig ix0

since it appears you have an "ix" type ethernet device.

 

[oumpa31]

Sorry for the long wait i had to get home from work. It is version U6 stable. I've just recently fully come over to the freeNAS side. I had played with it back in the day with an old p4 but until now i didn't have the means to switch fully over. So im still learning unfortunately. but i do read this forum a lot now. And before you ask this is in a striped array. i have a second NAS in a raid 5 as my back up that has spinning rust wd red drives.

 

[jgreco]

I'm not sure if im missing something in the set up of the card. Everything i read said it will just work.

Well, it obviously works, but it can probably be made to work better.

I think Dru wanted to confirm that your link was negotiated at 10Gbps and not something slower.

You can try running iperf on both ends to see if you're actually capable of hitting those speeds across the network. You should be, but when debugging it's sometimes best to confirm the basics.

https://doc.freenas.org/9.3/freenas_cli.html

There are a bunch of tunables that can be tweaked. A previous round of discussion resulted in a suggestion of

kern.ipc.maxsockbuf: 16777216
net.inet.ip.intr_queue_maxlen: 2048
net.inet.tcp.recvbuf_inc: 524288
net.inet.tcp.recvbuf_max: 16777216
net.inet.tcp.recvspace: 4192304
net.inet.tcp.sendbuf_inc: 32768
net.inet.tcp.sendbuf_max: 16777216
net.inet.tcp.sendspace: 2097152
net.route.netisr_maxqlen: 2048

but those might not be the most up-to-date. But good search-box fodder.

 

[oumpa31]

That is what my thought was.

I completely understand that. Plus more info is always better than less.

i'll have to read into how to run that on both ends.

I'll look into those tunables. i know i have them turned on but i have not touched them.

 

[oumpa31]

So these are the tunables and the current settings running the iperf for 100 seconds the max speed is 364MBps.

I'll try and change some of those and see where I'm at. Some of the ones you posted are not located on this.

 

[oumpa31]

I tested each one and i never got over 275. not really sure what the problem is. i tested with all the settings at the numbers that were suggested but nothing changed or it got worse. :/

 

[jgreco]

So anyways, no real help, but you need to make that fly, it seems that something networking-related is a problem. Do you have other machines you can test with?

 

[oumpa31]

My main work station is the only other one that has a 10 gig card. It has an Asus xg-100c. When i was just testing this server out i had one of those cards in it and win 10 pro on an ssd and i was getting gig transfers before. I decided to go to freeNAS because i really was just using it as a Media server and storage for the laptops and my work station. When i went to freenas i read that it was not supported so i got this intel card because everywhere i read it was plug and play with version 11.1. My speeds are better than gigabit but i know with 6 ssd in a stripe i should be able to transfer close to a gig if not a gig from a ram disk and 500 or so straight from ssd. Those were the speeds i was getting from the Asus xg-100c when it was in it.

 

[oumpa31]

The switch between the server and workstation is a Buffalo BS-XP2012 which has been awesome.

If i need to i can toss a win 10 ssd into an old tower along with the other Asus xg-100c card i have.

 

[chris crude]

I noticed you are focused on tuning FreeNAS with your issues, have you done any basic tuning of your client win10 box? There are some basics I had to do to unlock full speed between client-server.
https://www.intel.com/content/www/u...005811/network-and-i-o/ethernet-products.html

 

[oumpa31]

Chris

I have checked the HP specs and it says it has an 8x and 16x wired as such. I have not switched the card to the 16x port yet. That is next on my list of things to try. I did have the Asus card in this slot but that is a 4x card. I was getting full speed though that card. I will go though that site you sent me and also try changing slots tonight and see if either of that works.

 

[chris crude]

Now Im at my PC and not phone, here is another link i found helpful on the Windows side of things. Specifically the 10GbE tweaking details. I think i did everything except set jumbo frames.
http://www.cinevate.com/blog/confessions-of-a-10-gbe-network-newbie-part-5-smb3/

 

[acquacow]

I'm running x540-T2s, untuned on the FreeNAS side, and maxing out 10gige, but I was stuck at 150MB/sec until I found two articles that pointed at the fix.

They are linked in my post here:
https://forums.freenas.org/index.ph...-reads-iperf-is-fine.69795/page-2#post-483471

-- Dave

 

[oumpa31]

Chris

I applied those tweaks to windows it helped a little on my read from nas side im getting right around 400 for my reads (ram disk and ssd) but my writes are still in the 250 range.

Dave

I looked at those articles and my setting was already at 3. so the article really didn't change anything.

 

[oumpa31]

Just sucks that i know this server was receiving 10 gig speeds before i put freenas on it. When i was testeing it out and had tossed win 10 on it.

 

[c32767a]

Sorry for the long wait i had to get home from work. It is version U6 stable. I've just recently fully come over to the freeNAS side. I had played with it back in the day with an old p4 but until now i didn't have the means to switch fully over. So im still learning unfortunately. but i do read this forum a lot now. And before you ask this is in a striped array. i have a second NAS in a raid 5 as my back up that has spinning rust wd red drives.

So in your ifconfig output, for ix0, the interface is showing in promiscuous mode. Are you running something that's doing packet capture? Or do you have VMs configured? At least for testing purposes you might want to disable any of that stuff and get the card out of promiscuous mode. Notably, tcp segmentation offload and some other performance enhancements appear to be disabled on the interface when it's in promiscuous mode. That could be the source of some of your issues.

Also, this may be a dumb question, but you did reboot after you set all the kernel tunables, but before you tested, correct?

A few comments on the tunables:

net.inet.tcp.recvspace (and sendspace) seem small. we typically use 4194304.

Most of the zfs tunables in your list are better left at their defaults. The zfs.arc parameters are largely ignored by the most recent versions of ZFS and most of that code is self tuning and better left at the defaults. In any event, based on your hardware config, none of those ZFS tunables will make a measurable difference on your read and write speeds.

There are also Linux and BSD live CDs around.. you could always try booting one on your windows workstation to rule out(or in) Windows as the problem.