Multiple nics same subnet

ZombieHorde

Dabbler
Joined
Apr 19, 2016
Messages
13
Why do you not allow two nics on the same machine to be in the same subnet? I have one nic and two in a lagg that need to be in the same subnet because I only have one subnet. If I set the lagg to DHCP it ends up in the same subnet and everything works. But, I can't set the network address and set it to the same network manually because the GUI prevents it. Why?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Why do you not allow two nics on the same machine to be in the same subnet? I have one nic and two in a lagg that need to be in the same subnet because I only have one subnet. If I set the lagg to DHCP it ends up in the same subnet and everything works. But, I can't set the network address and set it to the same network manually because the GUI prevents it. Why?

Because if a second interface is configured on the same network, you end up with a broken network where things don't work the way a user might kinda hope that they would.

"because I only have one subnet" sounds like a good problem analysis and suggests the obvious fix: create a second subnet.
 

ZombieHorde

Dabbler
Joined
Apr 19, 2016
Messages
13
You know, it's a pretty typical setup for a small home network. One subnet, cause that's all I need, I have 5 ethernet ports on the machine, the first one is attached to the network for file serving duties and for administration. The next two are in a link aggregation lagg that is only there to serve my jail that's attached to it and the final two are unattached and undefined. I original had the first three in a link aggregation lagg, but the response times from the jail and the main file server was atrocious. When I split the two, the response times became good again, but I was originally able to set them up in the same subnet, along with the jail in the same subnet and it worked a treat. Experimented with round robin lagg to see if that was better, but it certainly was not. Then I changed it back to link aggregation and that's when the subnet error bit me and I had to set the lag to DHCP, which puts it in the same network segment, and it works a treat now. Now, I understand what the article is saying and even though it shouldn't work, it does. And telling me to setup another subnet with all of the associated problems with firewalls and everything else just to put a lagg on the network is not on my todo list when having them in the same network does me well. Just telling me that that's the solution when it worked before and works now is not a real explanation as to why I "shouldn't" be able to do it.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
You know, it's a pretty typical setup for a small home network. One subnet, cause that's all I need, I have 5 ethernet ports on the machine, the first one is attached to the network for file serving duties and for administration. The next two are in a link aggregation lagg that is only there to serve my jail that's attached to it and the final two are unattached and undefined. I original had the first three in a link aggregation lagg, but the response times from the jail and the main file server was atrocious. When I split the two, the response times became good again, but I was originally able to set them up in the same subnet, along with the jail in the same subnet and it worked a treat. Experimented with round robin lagg to see if that was better, but it certainly was not. Then I changed it back to link aggregation and that's when the subnet error bit me and I had to set the lag to DHCP, which puts it in the same network segment, and it works a treat now. Now, I understand what the article is saying and even though it shouldn't work, it does. And telling me to setup another subnet with all of the associated problems with firewalls and everything else just to put a lagg on the network is not on my todo list when having them in the same network does me well. Just telling me that that's the solution when it worked before and works now is not a real explanation as to why I "shouldn't" be able to do it.
Hmmm does your switch support lagg? Anything other than failover requires support from your switch.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Now, I understand what the article is saying and even though it shouldn't work, it does.

No, it doesn't. Unless your definition of "work" is that nothing is obviously broken and you're not bothering to look at what's actually going on. In which case, yes, it works swimmingly well.

The route to a connected network will always pick a single interface, even if there are multiple interfaces on that network. Having traffic from both 10.1.2.3 and 10.1.2.4 interfaces on your NAS egress through the physical interface that hosts 10.1.2.3 is NOT what most people consider "working".

ARP is also likely to not work as expected, and you may end up with traffic ingress on a different interface than expected. The behaviour here changes depending on which particular operating system you're doing this on, and I don't bother to keep track.

Now if you go to the linked article that was posted in #2, there's more detail. You don't have to like it. You don't have to believe me. I'm just telling you the way things work. I didn't implement it (though I agree with the strategy), so I'm not really all that interested in debate regarding it.

Multiple interfaces on a single subnet do not work as most people would expect. It's a broken configuration to put multiple interfaces on a single subnet. The GUI tries to enforce that.
 

lotustechie

Dabbler
Joined
Jun 3, 2020
Messages
27
I would like to have 1G NIC set to external and my 10G NIC set to internal. Is there a way to do this? Or is there no need to do that?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
It's not a good idea to do that. FreeNAS is not hardened against external attackers.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,947
Is there anything wrong with the following scenario.
Configure second NIC, do not assign IP and no DHCP. FreeNAS does not see a network on this card. Plugged into same network as main NIC
Assign Jail to that NIC on different IP (assuming that works)

My main NIC is a 10Gb card which for whatever reason stops working properly if a jail is assigned to it. I would like to use a second NIC, only seen by the jail for that jail. But given that the jail is a part of FreeNAS and not a separate OS I thought I would ask first before screwing things up
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Bearing in mind that I don't use FreeNAS jails, but do have intimate familiarity with jails on FreeBSD -- you should be able to configure the second interface as "up" and use it with a jail's vnet, as long as the FreeNAS UI allows such a configuration. It is totally fine from a networking perspective, because vnet is an independent network stack and this definitely works fine in FreeBSD.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
you should be able to configure the second interface as "up" and use it with a jail's vnet, as long as the FreeNAS UI allows such a configuration. It is totally fine from a networking perspective, because vnet is an independent network stack and this definitely works fine in FreeBSD.

I confirm that it works : i did that for my Plex server. FreeNAS itself has a first NIC plugged in my Data DMZ while that second one is plugged in the Application DMZ, two different zones managed by my internal pfSense firewall.

As you said, thanks to the fact that it is a complete independent IP stack, each one has its own IP, its own subnet, its own default gw, ...
 
Top