Intel 82599ES 10Gbit Ethernet ports seems not UP!!

zombiezafer

Cadet
Joined
Nov 29, 2021
Messages
4
I have a SUN FIRE X4470 SERVER with intel 82599ES chipset dual port 10Gbit Ethernet. Windows was installed on it and the cards were working. After I installed Truenas Core 12.0 U6.1 I could not figure out to make 10Gbit ports UP. I can see that cards are there but ports cannot get Up. I've changed the ports on the switch, SFPs and FC cable. But still the ports are not up. I tried to make a lagg port but it didn't help.
Here is pciconf and dmesg and ifconfig outputs related with 10GB card ports.

root@truenas[/tmp/driver]# pciconf -lv | grep -B4 network
ix0@pci0:193:0:0: class=0x020000 card=0x7b11108e chip=0x10fb8086 rev=0x01hdr=0x00
vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class = network
subclass = ethernet
ix1@pci0:193:0:1: class=0x020000 card=0x7b11108e chip=0x10fb8086 rev=0x01hdr=0x00
vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class = network



root@truenas[/tmp/driver]# dmesg | grep ix0
ix0: <Intel(R) X520 82599ES (SFI/SFP+)> mem 0xfbc00000-0xfbc7ffff,0xfbbfc000-0xfbbfffff irq 54 at device 0.0 on pci13
ix0: Using 2048 TX descriptors and 2048 RX descriptors
ix0: Using 16 RX queues 16 TX queues
ix0: Using MSI-X interrupts with 17 vectors
ix0: allocated for 16 queues
ix0: allocated for 16 rx queues
ix0: Ethernet address: 00:1b:21:8e:3b:1c
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: eTrack 0x000118c9 PHY FW V65535
root@truenas[/tmp/driver]# dmesg | grep ix1
ix1: <Intel(R) X520 82599ES (SFI/SFP+)> mem 0xfbd80000-0xfbdfffff,0xfbd7c000-0xfbd7ffff irq 61 at device 0.1 on pci13
ix1: Using 2048 TX descriptors and 2048 RX descriptors
ix1: Using 16 RX queues 16 TX queues
ix1: Using MSI-X interrupts with 17 vectors
ix1: allocated for 16 queues
ix1: allocated for 16 rx queues
ix1: Ethernet address: 00:1b:21:8e:3b:1d
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: eTrack 0x000118c9 PHY FW V65535






root@truenas[/tmp/driver]# ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb0
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:f8
inet 192.168.59.106 netmask 0xffffff00 broadcast 192.168.59.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:f9
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb2
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:fa
inet 192.168.59.107 netmask 0xffffff00 broadcast 192.168.59.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
igb3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:fb
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:1b:21:8e:3b:1c
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:1b:21:8e:3b:1d
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
 

cstamato

Cadet
Joined
Oct 22, 2017
Messages
8
I have a SUN FIRE X4470 SERVER with intel 82599ES chipset dual port 10Gbit Ethernet. Windows was installed on it and the cards were working. After I installed Truenas Core 12.0 U6.1 I could not figure out to make 10Gbit ports UP. I can see that cards are there but ports cannot get Up. I've changed the ports on the switch, SFPs and FC cable. But still the ports are not up. I tried to make a lagg port but it didn't help.
Here is pciconf and dmesg and ifconfig outputs related with 10GB card ports.

root@truenas[/tmp/driver]# pciconf -lv | grep -B4 network
ix0@pci0:193:0:0: class=0x020000 card=0x7b11108e chip=0x10fb8086 rev=0x01hdr=0x00
vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class = network
subclass = ethernet
ix1@pci0:193:0:1: class=0x020000 card=0x7b11108e chip=0x10fb8086 rev=0x01hdr=0x00
vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class = network



root@truenas[/tmp/driver]# dmesg | grep ix0
ix0: <Intel(R) X520 82599ES (SFI/SFP+)> mem 0xfbc00000-0xfbc7ffff,0xfbbfc000-0xfbbfffff irq 54 at device 0.0 on pci13
ix0: Using 2048 TX descriptors and 2048 RX descriptors
ix0: Using 16 RX queues 16 TX queues
ix0: Using MSI-X interrupts with 17 vectors
ix0: allocated for 16 queues
ix0: allocated for 16 rx queues
ix0: Ethernet address: 00:1b:21:8e:3b:1c
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: eTrack 0x000118c9 PHY FW V65535
root@truenas[/tmp/driver]# dmesg | grep ix1
ix1: <Intel(R) X520 82599ES (SFI/SFP+)> mem 0xfbd80000-0xfbdfffff,0xfbd7c000-0xfbd7ffff irq 61 at device 0.1 on pci13
ix1: Using 2048 TX descriptors and 2048 RX descriptors
ix1: Using 16 RX queues 16 TX queues
ix1: Using MSI-X interrupts with 17 vectors
ix1: allocated for 16 queues
ix1: allocated for 16 rx queues
ix1: Ethernet address: 00:1b:21:8e:3b:1d
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: eTrack 0x000118c9 PHY FW V65535






root@truenas[/tmp/driver]# ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb0
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:f8
inet 192.168.59.106 netmask 0xffffff00 broadcast 192.168.59.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:f9
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb2
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:fa
inet 192.168.59.107 netmask 0xffffff00 broadcast 192.168.59.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
igb3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:21:28:a5:6f:fb
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:1b:21:8e:3b:1c
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 00:1b:21:8e:3b:1d
media: Ethernet autoselect
status: no carrier
nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
Hello,

I realize the thread is a little bit old but I thought of posting what I did since I faced the same issue.
I am with TrueNAS 12 U7 and my 82599 interfaces wouldn't come up either, even though everything seemed to be ok.

All you have to do is issue an ifconfig up command. For some reason, TrueNAS won't get them up automatically.
I recall a post mentioning it is a FreeBSD issue.

Just create an Init/Shutdown Script command task, using the command ifconfig (name_of_the_interface) up, as POSTINIT.
Run the command in the shell and the interface will come up.
With the task, it will automatically come up after reboot.

Hope this helps.
 

zombiezafer

Cadet
Joined
Nov 29, 2021
Messages
4
I've ended up changing the SFPs with a DAC (Direct access cable). Sor some reason (possibly this is a driver issue) 10GB SFPs with FC LC-LC cables does not let the cards to be UP. But they are working on same server if I install linux or Windows on same server. So I decided to buy 3 meter DAC cable (these cables has sfps built-in on both ends of the cable). I've connected them to my ethernet ports... voila!! it works.
 
Top