Nics on the same network

[firepowr]

I'm not sure if this is a bug, but in version 8.3 and prior I'm able to have each of my nics have an IP on the same network.

For example:

bce1: 1.1.1.93 /24
bce2: 1.1.1.94 /24

But on version 8.3.1p2 I'm unable to add more than one IP from the same network. I get would get an error station that the network is already in use.

 

[jgreco]

Right, because it's broken networking and it shouldn't have let you do it. There are several detailed discussions of this in the forums, I'm not finding the one I want right now so this one doesn't quite fit but should get you straightened out.

FreeBSD tends to let you to do things that might be broken because they might also let you do something clever, and users typically have a deeper understanding of UNIX. The same isn't true of FreeNAS, for the range of things that you might want to do is severely limited, and the reasons are also constrained, and the users are generally not UNIX admins, so as a result the safer choice in FreeNAS is to put the seat belts on and make sure things are likely to work as expected. The FreeNAS devs are busily finding all the appropriate seat belts to add.

 

[shaithis]

This has been annoying me slightly.....

Supposedly the reason is that FreeNAS takes 1 NIC and deices it has the default gateway and then sends all data via that connection...........which is quite annoying as the way I had it setup (and the way I want it setup) the multiple interfaces on the same subnet are in a different subnet to the gateway.....so it's impossible for it to do what they say is the issue!!!!!!

 

[jgreco]

This has been discussed at length in the forums. Regardless of how it might appear, FreeBSD does not directly bind IP addresses to physical interfaces, and there is a complicated layer for both incoming and outgoing traffic that allows all the various sorts of networking that FreeBSD is capable of to work correctly.

For outbound traffic, the primary control is the routing table, which picks an interface based on the destination address. What people most frequently seem to want is a way to pick on both the source AND destination address, which would result in significant additional per-packet lookups, and result in a substantially slower network stack. FreeBSD doesn't do this because it is generally considered to be broken networking, or at the very least a terrible networking strategy.

If you want multiple interfaces on the same network, the IEEE standard way to do this is link aggregation, which FreeNAS supports. No need to invent some nonstandard way to do it.

If you want guaranteed faster performance onto a single network, you also have the option to use 10GbE, which FreeNAS also supports. Again, no need to invent some nonstandard way to do it.

If you want multiple addresses on a single network, you can use interface aliases, which FreeNAS also supports. Once again, no need to invent some nonstandard way to do it.

Just because Microsoft lets you do stupid doesn't make it a good idea.

 

[shaithis]

Linux lets you do it
Windows let you do it
VMWare let you do it

and it works

And FreeNAS used to do it....and it worked.

But now it's deemed broken somehow....

LACP is not very useful for iSCSI traffic
10GBe is prohibitive expensive....we are using FreeNAS, not a NetApp Filer.
Aliases will cause the problem we were told was the reason this change was made!

1 IP on a virtual adapter, the virtual adapter bound to a physical nic. In effect that is an IP bound to a physical NIC.....I really do not understand what your getting at.

 

[jgreco]

Pretty clear that Linux does NOT let you do this.

Note that I define "do this" as meaning something that actually works in the way you imagine it should, as opposed to just kinda sorta working because of mitigating factors. Linux chooses interfaces based on subnet membership, not source IP.

Windows does let you do it. Think I already said that.

VMware does kind-of let you do it, but then again, there's a lot of performance potential lost in the process, and does anyone actually do that, given VMware has such nice support for LACP?

FreeNAS never did it. It allowed you to configure it, but underneath, FreeNAS is FreeBSD, and FreeBSD has based outbound interface selection based on routing table since long before FreeNAS existed, so it never worked the way you imagine, even if it let you configure it.

So, before you respond with another vapid "but it works" response, why don't you actually check.

You'll find that in both FreeBSD and Linux, the packets will prefer one interface over the other. Not sure how to get Linux to reveal which one; I don't really care enough to dig it up. For FreeBSD, you can see which interface will be picked with "route -n get <ip-address>"

You can convince yourself of the reality by actually setting up and configuring the scenario, then watching traffic rates. For Linux, use "ifconfig" to monitor traffic. For FreeBSD, "netstat <interface> 1".

More reference: "In the common case, route selection is based completely on the destination address."

I have no idea what you were told about aliases or what you're discussing in your last paragraph. I'm not against fixing your misconceptions but I have to understand them first ;-)