Correctly using Iocage with vnet and bridge

STREBLO

Patron
Joined
Oct 23, 2015
Messages
245
I've been having some difficulties with iocage when using VNET. I have experience using iocage on FreeBSD with a shared IP, but never using VNET since I didn't need the functionality at that point.

After having some trouble figuring out how to configure a bridge correctly, I have finally gotten something working, but I keep getting this message in the log spewed out constantly:

Code:
Dec 27 14:03:47 lilan kernel: arp: 02:ff:60:13:e8:fc is using my IP address 172.20.40.21 on vnet1!


When I just attempted to use the bridge that already existed for my second nic by specifying interfaces="vnet1:bridge1" in iocage, for some reason FreeNAS creates a new bridge instead of using bridge0 and bridge1. In order to get around this problem I use the following tunables at boot:

Code:
Variable: ifconfig_bridge1
Value: addm igb1 up
Type: rc.conf


I created my jail with the following:

This is my exact setup:

Code:
iocage create --release 11.1-RELEASE --name emby \
		  boot="on" vnet=on \
		  ip4_addr="vnet1|172.20.40.21/24" \
		  interfaces="vnet1:bridge1" \
		  defaultrouter="172.20.40.1"


bridge1 one already existed on my system, I would assume for the old style jails.

Here's my iocage config
Code:
CONFIG_VERSION:9
allow_chflags:0
allow_mount:0
allow_mount_devfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:1
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
available:readonly
basejail:no
boot:on
bpf:no
children_max:0
cloned_release:11.1-RELEASE
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:172.20.40.1
defaultrouter6:none
depends:none
devfs_ruleset:4
dhcp:off
enforce_statfs:2
exec_clean:1
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:unifi.ramsden.network
host_hostuuid:unifi
host_time:yes
hostid:5c8a3f6f-989f-11e6-8d50-0cc47a696994
interfaces:vnet1:bridge1
ip4:new
ip4_addr:vnet1|172.20.40.20/24
ip4_saddrsel:1
ip6:new
ip6_addr:none
ip6_saddrsel:1
jail_zfs:off
jail_zfs_dataset:data/unifi/data
jail_zfs_mountpoint:
last_started:2017-12-27 02:36:01
login_flags:-f root
mac_prefix:02ff60
maxproc:off
memorylocked:off
memoryuse:off
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:1
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nmsgq:off
notes:none
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
priority:99
pseudoterminals:off
quota:none
release:11.1-RELEASE-p6
reservation:none
resolver:search ramsden.network;nameserver 172.20.40.1;nameserver 8.8.8.8
rlimits:off
securelevel:2
shmsize:off
stacksize:off
state:up
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:no
type:jail
used:readonly
vmemoryuse:off
vnet:on
vnet0_mac:02ff608700b4,02ff608700b5
vnet1_mac:02ff60680091,02ff60680092
vnet2_mac:none
vnet3_mac:none
wallclock:off
[SPOILER/]
 

STREBLO

Patron
Joined
Oct 23, 2015
Messages
245
I figured out the problem. I was using the same IP on multiple jails by accident. :oops:
 

BillCardiff

Explorer
Joined
May 13, 2014
Messages
59
I realize this is an older post, but in case anyone like me comes looking later, I also wanted a separate VNET for just my "media acquisition" software. Under System, Tuneables,

Code:
Variable: cloned_interfaces
Value: bridge0 bridge1
Type: rc.conf


and

Code:
Variable: ifconfig_bridge0
Value: addm igb2 up
Type: rc.conf


as well as..

Code:
Variable: ifconfig_bridge1
Value: addm igb5 up
Type: rc.conf


That sets up for the two bridges.
Then while creating the iocage instance that I wanted in the second bridge

Code:
iocage create --release 11.1-RELEASE --name deluge \
		 boot="on" vnet=on \
		 ip4_addr="vnet1|192.168.1.166/28" \
		 interfaces="vnet1:bridge1" \
		 defaultrouter="192.168.1.161"


and of course, under Network, Interfaces, I have IGB2 assigned as 192.168.1.5/25 while IDB5 is 192.168.1.165/28

The networking is odd because I mostly like it that way, and it works.
 

aedan929

Dabbler
Joined
Jul 5, 2016
Messages
27
I realize this is an older post, but in case anyone like me comes looking later, I also wanted a separate VNET for just my "media acquisition" software. Under System, Tuneables,
OMG!! Thanks you so much. I swear after every FreeNAS update plex gets more and more complicated! This ended my hours on confusion and counting! Thanks!!!
 

BillCardiff

Explorer
Joined
May 13, 2014
Messages
59
NOW though, if you upgrade to the latest, it can be done in gui alone, but I would assume AFTER you create the 2 bridges.
 

strelok

Dabbler
Joined
Jan 28, 2015
Messages
36
Hi,

I'm trying make the same configuration , where uses separate bridge for jails.

I have two ports in my server: bge0 and bge1
bge1 is primary port in net 192.168.88.0
bge0 is second port without ip address but connected to net 10.10.10.0

If I installing plugin it comes to default bridge0 with vnet0.1 interface, for example and use net 192.168.88.0
What I need is that plugin installation uses bge1 interface and net 10.10.10.0

I creating separate bridge - > Network -> Interfaces -> Add new bridge with name bridge1
I added bge0 to bridge1. Below output of ifconfig after this.

But I can't create any plugin installation using bridge1. Every time, when I try install plugins I got error: "Jail: test has a missing configuration, please check that the dataset is mounted." But it's not about mounting dataset.

So, how I should correctly create plugin that installation comes to separate bridge1 ?



bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bge0-dmz
options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether a0:1d:48:c7:b9:b8
hwaddr a0:1d:48:c7:b9:b8
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bge1-lan
options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether a0:1d:48:c7:b9:b9
hwaddr a0:1d:48:c7:b9:b9
inet 192.168.88.2 netmask 0xffffff00 broadcast 192.168.88.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: bridge-dmz
ether 02:68:9e:2d:30:01
nd6 options=9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
root@freenas[~]#
 
Top