SOLVED Chelsio S310E-CR, Active link but no ping

[AtomicTom]

Hi,

Long time reader/stalker but first time poster

I managed to grab myself some Chelsio S310E-CR cards off ebay with the inention of creating a direct 10Gb/s link between my main desktop and my FreeNAS server. Upon installtion in the FreeNAS box i'm able to add the interface in the GUI and assign it manually as 192.168.3.2, 255.255.255.255. In my main machine(windows 7) i installed it and downloaded the latest driver it showed up and configured it as 192.168.3.1, 255.255.255.255. (connected via a 3m twinax cable)

On Windows it shows it's connected to unidentified network and in FreeNAS it shows the link as active(as shown below in ifconfig) but i cannot ping either card or mount my CIFS share.
*EDIT* When i ping from the windows machine i just get 4x "Request timed out."

After encountering this problem i moved the card from the freeNAS box into a box with Windows Server 2008 and after adding some firewall exceptions to both machines could ping from win7 to server2008 freely and was able to transfer files as fast as my SDD could supply.

When i reinstalled the card into the FreeNAS box i chose the other avaliable pcie slot(if that was a point of failure) however i still have exactly the same issue. LEDs on both cards are green with a orange flash every 3 seconds.

This Server has been running fine for the past 9 months no problems of any kind with 2x6TB drives in Mirror. The only recent change is that i flashed the firmware of the LSI controller to p16 so i felt comfortable attaching 6x3TB harddrives in RAIDz2.

Forgive me if this exact problem has been posted previously. I have done a rather thorough seach of the forum and other sites but none of the problems have been quite the same as mine and solutions tried didn't work :/

Code:

[root@freenas ~]# ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
ether 0c:c4:7a:34:19:d6
inet 192.168.1.105 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
ether 0c:c4:7a:34:19:d7
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
cxgb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TX
CSUM_IPV6>
ether 00:07:43:06:aa:1f
inet 192.168.3.2 netmask 0xffffffff broadcast 192.168.3.2
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet 10Gbase-Twinax <full-duplex>
status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Hardware:
NIC: Chelsio S310E-CR
Build FreeNAS-9.3-STABLE-201505130355
Platform Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
Memory 8125MB (ECC)
Motherboard SuperMicro X10SL7-F
HDD - 2x6TB WD Red (mirrored)
- 6x3TB WD Red (RAIDz2)

Thankyou in advance for any help or suggestions.

 

[Ericloewe]

255.255.255.255

There's your problem, you're not getting anywhere with a 255 at the end of that subnet mask.

You probably want 255.255.255.0

 

[AtomicTom]

There's your problem, you're not getting anywhere with a 255 at the end of that subnet mask.

You probably want 255.255.255.0

The rest of my network is currently on 255.255.255.0
I did also try 255.255.255.128 and that did not work.

If that's not viable then could you suggest another that is? And can you explain why 255.255.255.255 isn't possible to use? (using all this a learning experience)

 

[AtomicTom]

Well i just tried 255.255.0.0 and that seemed to work! gah i feel silly for it just being that.

Would you still beable to explain why i was wrong?

 

[Mlovelace]

Well i just tried 255.255.0.0 and that seemed to work! gah i feel silly for it just being that.

Would you still beable to explain why i was wrong?

http://www.techrepublic.com/blog/data-center/ip-subnetting-made-easy-125343/

Enjoy ;)

 

[Ericloewe]

The subnet mask defines the subnet by masking out bits.

Basically, by setting 255.255.255.255, you're saying the whole address is a network prefix, with zero bits left over to identify network users. It's pure nonsense.

255.255.255.0 defines a subnet as being something like 192.168.1.XXX

But I was ninja'd with what looks like a decent explanation of the subject.

 

[AtomicTom]

That makes perfect sense! (i've seen it explain before but it's kinda not entirely clicked).

I tend todo this as a hobby so though i can generaly get things to work there are a few obvious gaps in my knowledge. Thankyou very much for both of you for your prompt help!

 

[Pheran]

Well i just tried 255.255.0.0 and that seemed to work! gah i feel silly for it just being that.

Would you still beable to explain why i was wrong?

Beware, I would not leave it at this setting, otherwise you risk your 10G link interfering with your regular network - because this netmask is telling your system that everything starting with 192.168 is the same subnet. What you really want is 255.255.255.0.

 

[depasseg]

The rest of my network is currently on 255.255.255.0

The rest of your network is actually on a combination of 192.168.1.xxx and 255.255.255.0. 255.255.255.0 by itself is meaningless.

Think of it this way, the subnet mask just indicates how large of a pool of ip addresses is considered to be in the same broadcast domain network. And items in the same broadcast domain can communicate to one another. 255.255.255.0 means that there are 255 hosts (based on the last .0) that are considered in the same network. So 192.168.1.10/255.255.255.0 is completely separate from 192.168.2.11/255.255.255.0. In order for those two machines to communicate, you would need to put a router in between them.