HORRIBLY slow 10gbe... Gigabit is fine

Status
Not open for further replies.

rplucker

Dabbler
Joined
Jul 10, 2016
Messages
30
Hi everyone,

I have a problem I've been beating myself over the head over for the past week or so.

My current setup is a Supermicro X9SRL-F, 10x 4tb HGST 7200 drives in a raidz2, 4x8g of Hynix HMT31GR7CFR4C-PB, Intel E5 2670@2.6ghz, and a Intel X540-T2. Freenas v9.10.1.
I have a cat 6a cable running from my freenas box to my Win10 workstation, which has a x540-t1 in it. I'm using a point to point connection over 10gbe to connect my workstation and nas.
Nas is connected to the rest of my network via gigabit ethernet.
Point to point network is 192.168.1.x/24
Everything else is 10.0.0.x/8

ifconfig:
Code:
[root@echo419] ~# ifconfig
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS												 UM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
		ether a0:36:9f:1f:90:8c
		inet 192.168.1.10 netmask 0xffffff00 broadcast 192.168.1.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
		status: active
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS												 UM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
		ether a0:36:9f:1f:90:8e
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: no carrier
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 150												 0
		options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
		ether 0c:c4:7a:cc:28:26
		inet 10.0.0.5 netmask 0xff000000 broadcast 10.255.255.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL												 _MAGIC,VLAN_HWTSO>
		ether 0c:c4:7a:cc:28:27
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
		inet6 ::1 prefixlen 128
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
		inet 127.0.0.1 netmask 0xff000000
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		ether 02:65:d9:61:3c:00
		nd6 options=9<PERFORMNUD,IFDISABLED>
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
		member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 8 priority 128 path cost 2000
		member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 7 priority 128 path cost 2000
		member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 3 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu												  1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:07:0a
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu												  1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:08:0a
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active



My problem is that recently, whenever I connect to my cifs share via 10gbe, everything is PAINFULLY slow... It takes 20 seconds or so to navigate from one folder to another. Most of the time, windows explorer will crash/freeze after a few seconds and I won't be able to do anything. When I first set it up, I was getting 800MB/s+ read/write without any issues. Nothing was changed to my knowledge that would cause this. I'm now running stock settings on my nic on my windows box, but I was running large send/receive buffers, jumbo frames, etc to get max speeds.
My reasoning for thinking this is a networking issue is that everything works as expected over gigabit on every device that I've tested including my Windows workstation. 110MB/s read/write consistently.

Am I missing anything obvious?
Thank you in advance
 
Last edited:

indy

Patron
Joined
Dec 28, 2013
Messages
287
Anything showing up in the FreeNAS log?

What speeds are you getting with iperf?
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
What does iperf give you?

Sent from my Nexus 5X using Tapatalk
 

rplucker

Dabbler
Joined
Jul 10, 2016
Messages
30
@indy @SweetAndLow Thanks for the replies. This seems to be an intermittent problem and everything's working as expected at the moment. I'll post iperf results when it starts acting up again...
 
Status
Not open for further replies.
Top