Correctly using Iocage with vnet and bridge

[STREBLO]

I've been having some difficulties with iocage when using VNET. I have experience using iocage on FreeBSD with a shared IP, but never using VNET since I didn't need the functionality at that point.

After having some trouble figuring out how to configure a bridge correctly, I have finally gotten something working, but I keep getting this message in the log spewed out constantly:

Code:

Dec 27 14:03:47 lilan kernel: arp: 02:ff:60:13:e8:fc is using my IP address 172.20.40.21 on vnet1!

When I just attempted to use the bridge that already existed for my second nic by specifying interfaces="vnet1:bridge1" in iocage, for some reason FreeNAS creates a new bridge instead of using bridge0 and bridge1. In order to get around this problem I use the following tunables at boot:

Code:

Variable: ifconfig_bridge1
Value: addm igb1 up
Type: rc.conf

I created my jail with the following:

This is my exact setup:

Code:

iocage create --release 11.1-RELEASE --name emby \
		  boot="on" vnet=on \
		  ip4_addr="vnet1|172.20.40.21/24" \
		  interfaces="vnet1:bridge1" \
		  defaultrouter="172.20.40.1"

bridge1 one already existed on my system, I would assume for the old style jails.

Here's my iocage config

Show

Code:

CONFIG_VERSION:9
allow_chflags:0
allow_mount:0
allow_mount_devfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:1
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
available:readonly
basejail:no
boot:on
bpf:no
children_max:0
cloned_release:11.1-RELEASE
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:172.20.40.1
defaultrouter6:none
depends:none
devfs_ruleset:4
dhcp:off
enforce_statfs:2
exec_clean:1
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:unifi.ramsden.network
host_hostuuid:unifi
host_time:yes
hostid:5c8a3f6f-989f-11e6-8d50-0cc47a696994
interfaces:vnet1:bridge1
ip4:new
ip4_addr:vnet1|172.20.40.20/24
ip4_saddrsel:1
ip6:new
ip6_addr:none
ip6_saddrsel:1
jail_zfs:off
jail_zfs_dataset:data/unifi/data
jail_zfs_mountpoint:
last_started:2017-12-27 02:36:01
login_flags:-f root
mac_prefix:02ff60
maxproc:off
memorylocked:off
memoryuse:off
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:1
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nmsgq:off
notes:none
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
priority:99
pseudoterminals:off
quota:none
release:11.1-RELEASE-p6
reservation:none
resolver:search ramsden.network;nameserver 172.20.40.1;nameserver 8.8.8.8
rlimits:off
securelevel:2
shmsize:off
stacksize:off
state:up
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:no
type:jail
used:readonly
vmemoryuse:off
vnet:on
vnet0_mac:02ff608700b4,02ff608700b5
vnet1_mac:02ff60680091,02ff60680092
vnet2_mac:none
vnet3_mac:none
wallclock:off

[SPOILER/]

 

[STREBLO]

I figured out the problem. I was using the same IP on multiple jails by accident.

 

[BillCardiff]

I realize this is an older post, but in case anyone like me comes looking later, I also wanted a separate VNET for just my "media acquisition" software. Under System, Tuneables,

Code:

Variable: cloned_interfaces
Value: bridge0 bridge1
Type: rc.conf

and

Code:

Variable: ifconfig_bridge0
Value: addm igb2 up
Type: rc.conf

as well as..

Code:

Variable: ifconfig_bridge1
Value: addm igb5 up
Type: rc.conf

That sets up for the two bridges.
Then while creating the iocage instance that I wanted in the second bridge

Code:

iocage create --release 11.1-RELEASE --name deluge \
		 boot="on" vnet=on \
		 ip4_addr="vnet1|192.168.1.166/28" \
		 interfaces="vnet1:bridge1" \
		 defaultrouter="192.168.1.161"

and of course, under Network, Interfaces, I have IGB2 assigned as 192.168.1.5/25 while IDB5 is 192.168.1.165/28

The networking is odd because I mostly like it that way, and it works.

 

[aedan929]

I realize this is an older post, but in case anyone like me comes looking later, I also wanted a separate VNET for just my "media acquisition" software. Under System, Tuneables,

OMG!! Thanks you so much. I swear after every FreeNAS update plex gets more and more complicated! This ended my hours on confusion and counting! Thanks!!!

 

[BillCardiff]

NOW though, if you upgrade to the latest, it can be done in gui alone, but I would assume AFTER you create the 2 bridges.

 

[strelok]

Hi,

I'm trying make the same configuration , where uses separate bridge for jails.

I have two ports in my server: bge0 and bge1
bge1 is primary port in net 192.168.88.0
bge0 is second port without ip address but connected to net 10.10.10.0

If I installing plugin it comes to default bridge0 with vnet0.1 interface, for example and use net 192.168.88.0
What I need is that plugin installation uses bge1 interface and net 10.10.10.0

I creating separate bridge - > Network -> Interfaces -> Add new bridge with name bridge1
I added bge0 to bridge1. Below output of ifconfig after this.

But I can't create any plugin installation using bridge1. Every time, when I try install plugins I got error: "Jail: test has a missing configuration, please check that the dataset is mounted." But it's not about mounting dataset.

So, how I should correctly create plugin that installation comes to separate bridge1 ?



bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bge0-dmz
options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether a0:1d:48:c7:b9:b8
hwaddr a0:1d:48:c7:b9:b8
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bge1-lan
options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether a0:1d:48:c7:b9:b9
hwaddr a0:1d:48:c7:b9:b9
inet 192.168.88.2 netmask 0xffffff00 broadcast 192.168.88.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bridge-dmz
ether 02:68:9e:2d:30:01
nd6 options=9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
root@freenas[~]#