Can't get 10GBe to talk to each other

[Eli Singer]

I have FreeNAS running on a Supermicro X11SSH-CTF motherboard which has 2 10Gb ports.
1 port connected to a 1Gb router, the other port is connected directly to a 10Gb card on my main machine (which also have a 1Gb port connected to the router).

How do i make the FreeNAS and my main machine talk only using the 10Gb connection while both of them are also connected to the router?
Currently it seem to still go through the router which limits the speed to 1Gb...

The 10Gbe cards did work for a while but for some unknown reason all communication now goes only through the 1Gb connection going through the router.

This is my configuration:

FreeNAS IP
LAN 1 (ix0 Interface) Connected to router
IP: 192.168.10.21
Subnet: 255.255.255.0

LAN 2 (ix1 Interface) Connected to 10Gb card on main computer
IP: 192.168.9.20
Subnet: 255.255.255.0

Gateway (router)
192.168.10.1

On my Windows machine i have the 10Gb NIC configured for:
IP: 192.168.9.25
Subnet: 255.255.255.0

Any ideas?

Eli

 

[kdragon75]

How are you connecting? Do you map a drive to the IP: 192.168.9.20? Or are you "browsing" the network for your FreeNAS host name?

Do you have a binding address set in FreeNAS? If you bind SMB to your 192.168.9.20 address, it will only allow connections from that interface. This will also lock all other guests from connecting to the share as you are using a direct connection.
FreeNAS Docs - SMB

 

[Eli Singer]

I"m "browsing". Going to \\FREENAS\WinShare to access it.
I need it to be accessible through both the direct link to my main machine, and through the router so other computers can get to it, if binding it to the port connected to the main computer will not allow other computers to access the SMB using the router, it is a problem.
If i only needed connection to the main machine i wouldn't have connected it at all to the router... :)

 

[kdragon75]

You need to map a drive or connect by using \\192.168.9.20\
By using the host name, it will be resolved to the 192.168.10.21 address an not use your 10gb interface.

 

[Eli Singer]

You need to map a drive or connect by using \\192.168.9.20\
By using the host name, it will be resolved to the 192.168.10.21 address an not use your 10gb interface.

How do i do that?

 

[kdragon75]

In windows 7, open Computer and just under the address bar, click "Map network drive". Select a drive letter and enter your share path in the "Folder: " section.
NOTE: you can't map "\\192.168.9.20\" that's just the server not a share. You would use something like \\192.168.9.20\Movies"

 

[dir_d]

To add to kdragon75, check and make sure your share is binded to your 10g interface.

 

[Mirfster]

If you want to isolate 10GB Traffic, I would suggest you segregate it on its own SubNet. Use something other than 255.255.255.0 (/24) which you have the other interfaces using; might want to try something with less hosts like 255.255.255.240 (/28) which only has 16 addresses / 14 hosts.

As always, check CIDR to make sure you are assigning IPs in accordance.

 

[kdragon75]

If you want to isolate 10GB Traffic, I would suggest you segregate it on its own SubNet. Use something other than 255.255.255.0 (/24) which you have the other interfaces using; might want to try something with less hosts like 255.255.255.240 (/28) which only has 16 addresses / 14 hosts.

As always, check CIDR to make sure you are assigning IPs in accordance.

He has a separate subnets setup. They are 192.168.9.0/24 & 192.168.10.0/24. The /24 is only the mask length.

To add to kdragon75, check and make sure your share is binded to your 10g interface.

As noted:

I need it to be accessible through both the direct link to my main machine, and through the router so other computers can get to it

If he binds the service to a single interface, it will not serve to both subnets as he stated is required..

 

[kdragon75]

As always, check CIDR to make sure you are assigning IPs in accordance.

Classless Interdomain Routing? I don't think that applies as we are not doing any routing. You may be thinking of VLSM (Variable Length Subnet Masks) but there's no reason to use that. this is a direct connection on a small and otherwise single subnet-network.

 

[dir_d]

He should be able to bind it to multiple interfaces. Just don't want him to miss the 10g one.

 

[kdragon75]

And if he does that and still uses the host name, it will still use the 1gb interface over his main subnet. he needs to use the IP of the 10gb interface. If he wanted he could use a host file entry but that's clumsy and makes troubleshooting a pain in the backside as everyone forgets the host file.

 

[dir_d]

I agree make sure the 10g is bound, ping via 10g IP make sure there is a traffic, then use the IP to get to the share via 10g.

 

[kdragon75]

The big question to ask yourself when thinking about this is "How will windows know what port to connect to?" If you don't have a clear answer to how windows will differentiate what IP to use when connecting, it probably won't work. Keep in mind his windows box is using 192.168.10.1 as the default gateway and MAYBE for DNS. NetBIOS & Avahi are another can of worms.

 

[dir_d]

Using just an IP address is fine if hes just wanting to connect SMB, other services can get complicated quickly.

 

[Mirfster]

So explain to me what is going to ensure traffic from 192.168.9.25 (the 10GB NIC on Windows machine) is going to always be directed to 192.168.9.20 (I am assuming this is the 10GB on the FreeNAS Server) when there is a subnet of 255.255.255.0 being assigned by both the router to the DHCP enabled NIC(s) and the static settings on the 10GB NICs?

Make life easier and simply assign the 10GB NICs a different subnet mask and IP range...

 

[dir_d]

Ok, let's clear some things up. He has a 1g connection to the router on his windows machine, is that connection on his 192.168.10.x subnet? He has 2 subnets so it should be. If it is and he has the 10g on the 192.168.9.x subnet directly connected to each other there are no problems just using the IP.

If I read it wrong and he only has the 10g connection then the simplest thing to do would be to add the 1g connection to his windows machine and plug the 10g directly into the freenas machine. If not he will need to add static routes into his router to take the 10g connection and he will have to add hostnames like freenas10g to his DNS if he wants the hostnames to work.

 

[kdragon75]

So explain to me what is going to ensure traffic from 192.168.9.25 (the 10GB NIC on Windows machine) is going to always be directed to 192.168.9.20 (I am assuming this is the 10GB on the FreeNAS Server) when there is a subnet of 255.255.255.0 being assigned by both the router to the DHCP enabled NIC(s) and the static settings on the 10GB NICs?

Make life easier and simply assign the 10GB NICs a different subnet mask and IP range...

both 10gb interfaces are in the 192.168.9.0/24 subnet. That's 24 bits or three octets that tell the computer and routers what network we are talking about. So if we use the same netmask of 255.255.255.0 (aka /24) and have one interface as 192.168.10.xx and another as 192.168.9.xx, the third octet is seperating the network. basic IP addressing and subnetting.

On the windows machine there is an interface with the IP of 192.168.9.25. this means that windows has a DIRECT connection to that network and it does not need to look further into its routing table. EVEN if there was another route to the 192.168.9.0/24 network it would STILL use the 10gb card with the 192.168.9.25 as the exit interface as that would have a lower administrative distance.

simply assign the 10GB NICs a different subnet mask and IP range

That exactly what he did with the exception of the netmask. Again the netmask by its self does not define the network. The netmask is only use to identify what bits are part of the network address and what bits are the host address.
192.168.10.0/24 and 192.168.9.0/24 are NOT the same same subnet or even in this case the same (layer 2) network

 

[Eli Singer]

both 10gb interfaces are in the 192.168.9.0/24 subnet. That's 24 bits or three octets that tell the computer and routers what network we are talking about. So if we use the same netmask of 255.255.255.0 (aka /24) and have one interface as 192.168.10.xx and another as 192.168.9.xx, the third octet is seperating the network. basic IP addressing and subnetting.

On the windows machine there is an interface with the IP of 192.168.9.25. this means that windows has a DIRECT connection to that network and it does not need to look further into its routing table. EVEN if there was another route to the 192.168.9.0/24 network it would STILL use the 10gb card with the 192.168.9.25 as the exit interface as that would have a lower administrative distance.


That exactly what he did with the exception of the netmask. Again the netmask by its self does not define the network. The netmask is only use to identify what bits are part of the network address and what bits are the host address.
192.168.10.0/24 and 192.168.9.0/24 are NOT the same same subnet or even in this case the same (layer 2) network

That is correct.
So does that mean someone got a solution?

 

[kdragon75]

Basicly, just use the IP of the FreeNAS 10gb interface. Open explorer (not IExplorer) and in the address bar enter "\\192.168.9.20\" without the quotes. You should now see your shares. feel free to make a shortcut.