Networking SNMP

[Robert gornall]

I'm running the following system:

FreeNAS-9.3-STABLE
Boston Igloo 4U 36-Bay Storage System
Supermicro Intel E5-2600v2 Series DP Motherboard
2x Intel E5-2609 2.5GHz Quad-Core Xeon
32GB (8x4GB) Supermicro DDR3 ECC-Registered 1333MHz
Mellanox Dual-Port 10GbE SFP+ Network Adapter

I'm seeing something a little odd, and wondering if I have some misconfiguration:
When I do ifconfig i'm seeing 10gbe but snmpwalk is showing me 1.4gbit.

This looks like it could be a casting/int max issue...

Code:

snmpwalk -v2c -c public -OQUs udp:goblet:161 | grep ifSpeed
ifSpeed.1 = 1410065408
ifSpeed.2 = 0
ifSpeed.3 = 1000000000
ifSpeed.4 = 1000000000
ifSpeed.5 = 1000000000
ifSpeed.6 = 1000000000
ifSpeed.7 = 0
ifSpeed.8 = 10000000
ifSpeed.9 = 0
ifSpeed.10 = 1410065408

Code:

[root@goblet] ~# ifconfig
mxge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: connected to EX-X650-FB12-E (2:4)
        options=6c03bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:60:dd:43:8a:76
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet 10Gbase-SR <full-duplex>
        status: active
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: connected to HP-2900-WMS-B36-E (38)
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:60:dd:43:8a:76
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 0c:c4:7a:32:1d:f1
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: no carrier
igb2: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 0c:c4:7a:32:1d:f2
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: no carrier
igb3: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 0c:c4:7a:32:1d:f3
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 00:60:dd:43:8a:76
        inet 10.16.1.148 netmask 0xfff00000 broadcast 10.31.255.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
        laggproto failover lagghash l2,l3,l4
        laggport: igb0 flags=0<>
        laggport: mxge0 flags=5<MASTER,ACTIVE>

 

[dlavigne]

Were you able to sort this out?

 

[Robert gornall]

Nope, no progress on this yet.

 

[tvsjr]

ifSpeed is a 32-bit integer... 10Gbps overflows it. 10,000,000,000 - 1,410,065,408 = 8,589,934,592 / 2 = 4,294,967,296 - the maximum value for a 32-bit unsigned integer. ifHighSpeed is a 64-bit value and should work better.

If you're planning to use all of this for statistics gathering (Cacti, etc.), you should also use ifHCInOctets and ifHCOutOctets (HC = high capacity) as these are also 64-bit counters. For something like Cacti, polling at 5 minute intervals, it's entirely possible to overrun a 32-bit counter in a single interval (125MB/sec = 37.5GB in 5 mins... the counter would overflow in a hair over 34 seconds assuming link saturation).

 

[Robert gornall]

Code:

snmpwalk -v2c -c public -OQUs udp:goblet:161 | grep ifHighSpeed
ifHighSpeed.1 = 1410
ifHighSpeed.2 = 0
ifHighSpeed.3 = 1000
ifHighSpeed.4 = 1000
ifHighSpeed.5 = 1000
ifHighSpeed.6 = 1000
ifHighSpeed.7 = 0
ifHighSpeed.8 = 10
ifHighSpeed.9 = 0
ifHighSpeed.10 = 1410

It's the same *wrong* number, just a different base.

Probably related to this:
http://sourceforge.net/p/net-snmp/mailman/message/20564960/

 

[tvsjr]

Yep, probably so. The other option you have is to graph from the switch perspective rather than the host... usually the data from the switch is somewhat better, and there will be an additional "virtual" interface added for any trunk groups on most decent switches. Or, just use a hard-coded value for the max speed.