High packet loss after switch failure.

Status
Not open for further replies.
James Bean

James Bean

Cadet
Joined
Dec 7, 2016
Messages
5
I apologise I have surfed google and the forums but didn't find a solution to my issue.

After a storm my cisco switch started dropping packets to everything, long story short, Cisco switch faulty, booted to the curb and new one installed.
All the devices (printers/pc's/routers/TV's) on the new switch can ping and talk to each other without packet loss, no issues, but I am getting 10% packet loss to my FreeNAS box.
Packet loss hits 30-40% if I start using the web interface (which starts dropping out), or transfer files off the samba shares.

Motherboard has a pair of Intel i210 nics, that were in load balance mode, I deleted this and configured the nics individually, same issue.
I had a spare Intel PRO/1000 PT Dual Port Server Adapter which I plugged into a PC and tested the ports were working first (which they were no issues), so I installed the card and set it up as individual nics and tested, still getting the same issues with packet loss as above. Pulled the card and installed in into my other workstation and it worked fine.
I used my Mikrotik Router to ping the FreeNAS box constantly and get the same results, although it was reporting "packet rejected" for most of the lost packets.

Twist is if I get on the console of the FreeNAS box and ping any device on the network, 0% packet loss, even while using the web interface from my PC (1000 ping tests at a time). This is while I get packet loss for the incoming packets from the router and my windows PC at the same time.

I have also tried resetting all the nic's, deleting all the lagg nic's and configuring each nic individually one at a time and testing, as well as brand new network cables.

So it doesn't sound like the NIC's are faulty, but I am not enough of a *nix expert to track the issue in FreeBSD, any help would be appreciated.

Output of ifconfig:-
Code:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
		ether 00:1f:29:57:0d:ae
		inet 192.168.69.10 netmask 0xffffff00 broadcast 192.168.69.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
		ether 00:1f:29:57:0d:af
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: no carrier
igb0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
		ether d0:50:99:6c:ab:6a
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: no carrier
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
		ether d0:50:99:6c:ab:6b
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
		inet6 ::1 prefixlen 128
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
		inet 127.0.0.1 netmask 0xff000000
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		ether 02:de:f2:6b:95:00
		nd6 options=1<PERFORMNUD>
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
		member: epair6a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 13 priority 128 path cost 2000
		member: epair5a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 12 priority 128 path cost 2000
		member: epair4a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 11 priority 128 path cost 2000
		member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 10 priority 128 path cost 2000
		member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 9 priority 128 path cost 2000
		member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 8 priority 128 path cost 2000
		member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 7 priority 128 path cost 2000
		member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 1 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:07:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:08:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:09:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair3a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:0a:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair4a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:0b:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair5a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:0c:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
epair6a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:0d:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active


Output of pciconf -lv
Code:
hostb0@pci0:0:0:0:  class=0x060000 card=0x0c081849 chip=0x0c088086 rev=0x06 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = 'Xeon E3-1200 v3 Processor DRAM Controller'
  class  = bridge
  subclass  = HOST-PCI
pcib1@pci0:0:1:0:  class=0x060400 card=0x0c011849 chip=0x0c018086 rev=0x06 hdr=0x01
  vendor  = 'Intel Corporation'
  device  = 'Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller'
  class  = bridge
  subclass  = PCI-PCI
xhci0@pci0:0:20:0:  class=0x0c0330 card=0x8c311849 chip=0x8c318086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family USB xHCI'
  class  = serial bus
  subclass  = USB
none0@pci0:0:22:0:  class=0x078000 card=0x8c3a1849 chip=0x8c3a8086 rev=0x04 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family MEI Controller'
  class  = simple comms
none1@pci0:0:22:1:  class=0x078000 card=0x8c3b1849 chip=0x8c3b8086 rev=0x04 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family MEI Controller'
  class  = simple comms
ehci0@pci0:0:26:0:  class=0x0c0320 card=0x8c2d1849 chip=0x8c2d8086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family USB EHCI'
  class  = serial bus
  subclass  = USB
pcib2@pci0:0:28:0:  class=0x060400 card=0x8c101849 chip=0x8c108086 rev=0xd5 hdr=0x01
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family PCI Express Root Port'
  class  = bridge
  subclass  = PCI-PCI
pcib3@pci0:0:28:2:  class=0x060400 card=0x8c141849 chip=0x8c148086 rev=0xd5 hdr=0x01
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family PCI Express Root Port'
  class  = bridge
  subclass  = PCI-PCI
pcib4@pci0:0:28:3:  class=0x060400 card=0x8c161849 chip=0x8c168086 rev=0xd5 hdr=0x01
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family PCI Express Root Port'
  class  = bridge
  subclass  = PCI-PCI
pcib5@pci0:0:28:7:  class=0x060400 card=0x8c1e1849 chip=0x8c1e8086 rev=0xd5 hdr=0x01
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family PCI Express Root Port'
  class  = bridge
  subclass  = PCI-PCI
ehci1@pci0:0:29:0:  class=0x0c0320 card=0x8c261849 chip=0x8c268086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family USB EHCI'
  class  = serial bus
  subclass  = USB
isab0@pci0:0:31:0:  class=0x060100 card=0x8c561849 chip=0x8c568086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = 'C226 Series Chipset Family Server Advanced SKU LPC Controller'
  class  = bridge
  subclass  = PCI-ISA
ahci0@pci0:0:31:2:  class=0x010601 card=0x8c021849 chip=0x8c028086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode]'
  class  = mass storage
  subclass  = SATA
none2@pci0:0:31:3:  class=0x0c0500 card=0x8c221849 chip=0x8c228086 rev=0x05 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '8 Series/C220 Series Chipset Family SMBus Controller'
  class  = serial bus
  subclass  = SMBus
em0@pci0:1:0:0: class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '82571EB Gigabit Ethernet Controller'
  class  = network
  subclass  = ethernet
em1@pci0:1:0:1: class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = '82571EB Gigabit Ethernet Controller'
  class  = network
  subclass  = ethernet
igb0@pci0:3:0:0:  class=0x020000 card=0x15331849 chip=0x15338086 rev=0x03 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = 'I210 Gigabit Network Connection'
  class  = network
  subclass  = ethernet
igb1@pci0:4:0:0:  class=0x020000 card=0x15331849 chip=0x15338086 rev=0x03 hdr=0x00
  vendor  = 'Intel Corporation'
  device  = 'I210 Gigabit Network Connection'
  class  = network
  subclass  = ethernet
pcib6@pci0:5:0:0:  class=0x060400 card=0x11501849 chip=0x11501a03 rev=0x02 hdr=0x01
  vendor  = 'ASPEED Technology, Inc.'
  device  = 'AST1150 PCI-to-PCI Bridge'
  class  = bridge
  subclass  = PCI-PCI
vgapci0@pci0:6:0:0:  class=0x030000 card=0x20001849 chip=0x20001a03 rev=0x21 hdr=0x00
  vendor  = 'ASPEED Technology, Inc.'
  device  = 'ASPEED Graphics Family'
  class  = display
  subclass  = VGA


Specs:

FreeNAS 9.10.1-U4
Asrock E3C226D21 Intel Xeon i3-1220 v3 and 2 x 8Gb ECC DDR3 1600MHz
4 x WD40EFRX WD Red 4Tb in RAIDZ2 and 1 16Gb SanDisk USB (boot)
 
Last edited:
J

jlpellet

Patron
Joined
Mar 21, 2012
Messages
287
Probably already tried but have you rebooted the switch & tried alternate ports on the switch as well as an intermediate switch between the Cisco & the FrreNAS box?
 
James Bean

James Bean

Cadet
Joined
Dec 7, 2016
Messages
5
jlpellet said:
Probably already tried but have you rebooted the switch & tried alternate ports on the switch as well as an intermediate switch between the Cisco & the FrreNAS box?
Click to expand...
Old switch was thrown out and yes I tested ports with my PC and the router and then swapped in the nic's from the FreeNAS box so the ports are working fine. I am starting to think it is a corruption in the OS and my best option might be a backup and re-install of FreeNAS.
 
James Bean

James Bean

Cadet
Joined
Dec 7, 2016
Messages
5
I did an in place upgrade on 9.10.1-U4 but this didn't fix the issue, currently running memtest to make sure no other components became faulty in the storm.
 
James Bean

James Bean

Cadet
Joined
Dec 7, 2016
Messages
5
God I am a dumbass and ignored the 1 principal of good IT diagnostics, unplug everything else but the machines you are testing.

Occam's razor, my ip phone that was turned off but plugged in took out a port in the main router during the storm, when unplugged the link light and traffic was still occurring on the interface. Disabled the port and the lost packets TO the FreeNAS box stopped.

Sorry for wasting everyones time and thank you to those that responded to my cry for help, looks like i sent a perfectly good Cisco smart switch to the recycling center for nothing and I will be sending an IP phone next week.
 
Status
Not open for further replies.

Similar threads

S
  • Locked
SOLVED Network performance
Replies
3
Views
2K
Stefan Konstantinovitsch
S
lexxai
  • Locked
wrong detetection VLAN_HWTAGGING for chipset of fxp intrerface ?
Replies
2
Views
1K
dlavigne
D
nikkon
  • Locked
SOLVED FreeNAS 11 incorrect driver
Replies
18
Views
3K
nikkon
nikkon
M
QNAP QXG-2G1T-I225 compatability
Replies
1
Views
2K
Samuel Tai
Samuel Tai
M
  • Locked
SOLVED 10GbE direct connection FreeNAS<-->Workstation stopped working
Replies
4
Views
4K
TheCowGod
T
Top