Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

SOLVED How to set Separate VLAN for Jail ?

Peter Brille

Member
Joined
Mar 6, 2015
Messages
204
Please don't confuse the users Kennyvb8. Iocage is going to replace warden. bhyve is not a replacment for anything. It's an all new hypervisor.
Also: Iocage is no UI. It's a command line jail manager.
 

Kennyvb8

Member
Joined
Mar 18, 2017
Messages
110
Please don't confuse the users Kennyvb8. Iocage is going to replace warden. bhyve is not a replacment for anything. It's an all new hypervisor.
Also: Iocage is no UI. It's a command line jail manager.
Ahh my bad. Warden is the old


Sent from my iPhone using Tapatalk
 

sretalla

Dedicated Sage
Joined
Jan 1, 2016
Messages
3,032
Peter Brille said:
Also: Iocage is no UI. It's a command line jail manager.
The new (still called Beta) UI in FreeNAS 11.1 has a jails section which serves as the UI for iocage.

The legacy (traditional) FreeNAS UI still plays with warden jails.

Both warden and iocage are command line jail managers which have both had GUI goodness delivered as part of FreeNAS.

An important note for FreeNAS users old and new... You can't manage iocage jails from the legacy UI, so expect to not see your warden jails when you upgrade to 11.1 and start using the new UI (you can still switch back to the legacy UI to see them if necessary, but it seems there will come a time when migration will be required... I was reading that 11.2 may offer something to assist with that).
 

ezra

Member
Joined
Jan 15, 2015
Messages
124

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
I am running 11.2U2 and still see the same issue. I moved from multiple 1GB NIC'S to 10Gb trunks as all data centers have. FreeNAS creates the single bridge0 for the native vlan only. I have created vlan interfaces with hopes of creating a nic in whether in a jail or vm so I can use the vlan interface as the parent but no bridges for those vlans, only bridge0 for native vlan 1. With a VM I can get a nic assigned to that parent interface but the tagging flawed and I cannot communicate though oddly my router responds to arp requests from that nic.....but no other traffic passes... so on to using jails for performance and thinking I could have multiples nics in the jail, again single bridge no way to tie it in... I see from this post destroying the bridge and adding new, I need the existing but need to add others and add the nic to that but it would be a manual process every time freenas reboot....which is sadly too often when update go sideways like U2 and U2.1 and Samba. I simply need to bridge services from freenas and a jail with contained services to 3 vlans, and cannot without major manual intervention upon each reboot, or go back to using a dedicated nic for each vlan. Even that way, with 11.2U2 we have to manually create the nic:bridge combo as UI no longer does it, and again, only a single bridge is created by default...
 

ezra

Member
Joined
Jan 15, 2015
Messages
124
I am running 11.2U2 and still see the same issue. I moved from multiple 1GB NIC'S to 10Gb trunks as all data centers have. FreeNAS creates the single bridge0 for the native vlan only. I have created vlan interfaces with hopes of creating a nic in whether in a jail or vm so I can use the vlan interface as the parent but no bridges for those vlans, only bridge0 for native vlan 1. With a VM I can get a nic assigned to that parent interface but the tagging flawed and I cannot communicate though oddly my router responds to arp requests from that nic.....but no other traffic passes... so on to using jails for performance and thinking I could have multiples nics in the jail, again single bridge no way to tie it in... I see from this post destroying the bridge and adding new, I need the existing but need to add others and add the nic to that but it would be a manual process every time freenas reboot....which is sadly too often when update go sideways like U2 and U2.1 and Samba. I simply need to bridge services from freenas and a jail with contained services to 3 vlans, and cannot without major manual intervention upon each reboot, or go back to using a dedicated nic for each vlan. Even that way, with 11.2U2 we have to manually create the nic:bridge combo as UI no longer does it, and again, only a single bridge is created by default...
You can add the creation of bridges and addition of IF's to it via system tunabels. If you need a screen i can provide one .
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
please, I will test it right away...
 

ezra

Member
Joined
Jan 15, 2015
Messages
124

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
https://github.com/iocage/iocage/issues/873

Check my last post there, if you need, hit me up on discord Ezra#8999
not sure what discord is, sent you a message here and replied to email, config all there, all members in correct bridges, still no connectivity outside freenas for those vlans, I think me using an untagged native vlan is why, it doesn't appear to handle both tagged and untagged well.
 

garm

Neophyte Sage
Joined
Aug 19, 2017
Messages
1,465
No, don’t mix untagged and tagged traffic on the same interface. It’s a common source of issues
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
Ok thanks, that's what I feared based on the config ezra shared with me, and correlates exactly with behavior I am seeing in FreeNAS networking when manually adding the bridges with proper member vlans and interfaces. Ok well If I have to burn a 2nd 10Gb port just for tagged vs untagged traffic then I hope I can figure how to get multiple interfaces to boot in my jail, any attempt to have multiple nic:bridge combo's crashes the freebsd jail. I will hook up another 10Gb and moved tagged traffic there, if networking properly works using a VM and tap interfaces to both tagged and untagged interfaces, then I'll move on and worry about the jail built to replace the vm due to latency concerns.

thanks for the response !
d-
 

garm

Neophyte Sage
Joined
Aug 19, 2017
Messages
1,465
It won’t solve everything, I put the bridge assaignment in post init.
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
I moved the tagged vlans from cxl0 to cxl1, using ifconfig added members to respective bridges and vm is working on ALL interface without rebooting. The vm's has multiple interfaces, 1 using untagged cxl0 and 2 using the tagged vlans moved to cxl1 for this test. Working as required... NOW was abandoning vm due to latency and resources used, need to figure out how to get those 2 tagged interfaces into the jail so I can shutdown the vm. Jail only allows a single network interface via the gui, adding multiple nic:bridge pairs in any method whether gui or config.json the jail won't boot. I got this far, anyone know how to add additional interface nic:bridge pairs to a jail ??

thanks for the help guys, I hate using 2 interfaces for this one solution but at least I had the interface and port :) . Hopefully in time it will get rectified, otherwise I can just move vlan 1 to tagged on that port during some downtime.
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
How do you create the multiple NIC configurations inside the jail ? I have added the 2nd and 3rd NIC configuration via:
vnet0:bridge0,vnet1:bridge1,vnet2:bridge2
vnet0 IP address/ipv6 info filled out, then nics 1,2,3 all have 2 mac addresses configured, but the jail boots up and still only has a single epair0b interface internally, 2nd and 3rd nics not created, also in freenas only see vnet0 interface. I assume once the interfaces were created and appeared, I could set the ip address for each interface normally, using its name in rc.conf to set ip, subnet, etc. The NIC's aren't being created in the jail ? Now I have bridges and tagging working, the jail allows up to 4 interfaces using vnet, mine only has 1. What did I miss ? I even increased the fibs etc in freenas to be sure.
 

ezra

Member
Joined
Jan 15, 2015
Messages
124
Hey Dave,

Just replying here for others to be able to chime in.

Well first off, your regular LAN is untagged right? I had that and it wouldnt work... I've switches my main LAN to all tagged VLAN 1.
Now also create a bridge for the VLAN1 (cant recall if you did for your LAN) mine is the default bridge0, just follow my screenshot.

Then set the jails to: ip4_addr:vnet0, interfaces:vnet0:bridgeX (for each jail that you want in another vlan, a new bridge) and vnet_default_interface:vlanXX and exec_fib:1 (number of the fib you set below)

After that set the net fibs (added a few days ago in the picture) https://user-images.githubusercontent.com/13510720/54566479-3d7b6400-49d1-11e9-97c3-fc9217d7ff1a.png
Then for all your vlans set the route inside your freenas cli like this:

Code:
route add -net 192.168.6.0/24 -iface vlan6 -fib 1

route add default 192.168.6.1 -fib 1


change the subnet, vlan and fib (1 to 10, so 10 vlans possible that i know off)

That should do it, got it working perfectly right now.

Edit: got your switch port configured in trunk (believe you do but just to be sure)?
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
Agreed, and yes I moved my tagged vlans to cxl1, so default bridge0 has cxl0 as member, 2 added vlans are on cxl1 physical now. doing that I could set tap1 to bridge1 vlan2 and tap2 bridge2 vlan5 and see it work in vm. However in the jail i need same 3 interfaces, the interfaces themselves will not create without iocage errors for middleware. vnet0:bridge0,vnet1:bridge1,vnet2:bridge2 and 2nd and 3rd will never create. A lso added IP and mac lines, have to remove to get jail to boot and able to ping vnet0 again.....can make it work with a vm cause it uses tap interfaces I think, used tuneables to add tap to proper bridge as well following your example.. Now stuck at jail wont create other vnet nics.....unless its cause vnet0 goes to bridge0 which is untagged, but wouldn't think so.. vm worked fine after I added another interface for tagged vlans.
 

ezra

Member
Joined
Jan 15, 2015
Messages
124
The VM worked for me to without any tinkering. I just added the tunables for a good overview and control.

Try not to use vnet1 and 2, i tried this without any success. They get seperated properly if you just specify vnet0.

Did you setup the fibs?
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
I did setup the fibs, are you saying not to increments the interface #'s in jail, just use vnet0:bridge0,vnet0:bridge1,vnet0:bridge2 for my 3 interfaces then ?
 

ezra

Member
Joined
Jan 15, 2015
Messages
124
Yes correct!
 

Dave Genton

Member
Joined
Feb 27, 2014
Messages
133
I tried naming each vnet0 as they appear in freenas ifconfig output, but that won't allow the jail to start. If I change them back to "vnet0:bridge0,vnet1:bridge1,vnet2:bridge2" it boots up without error, but no nic's are created for 2nd and 3rd networks.....I don't know what else to try, doesn't seem like there's much to it unless there's a setting in freenas that says whether or not a jail can have multiple NIC interfaces ?? Jail documentation and gui both states up to 4 interfaces, has mac addresses for 3, config for 3, but gui only has spot for 1 interface, others must be listed in network options
 
Top