ezra
Contributor
- Joined
- Jan 15, 2015
- Messages
- 124
Hey!
100$ BOUNTY FOR WHOEVER HELPS ME FIX THIS!!!
So i've been trying to get this right for about 5 days now. I had around 4 reinstalls because i locked myself out (flood/storms or something like that). Also a few CLI config restores happend.
I'll first try to explain what hardware i have then ill explain what i try to achieve. I hope anyone can help me.
I have a OPNsense router 4 ports
OS Version:
FreeNAS-11.2-RELEASE-U1
(Build Date: Dec 20, 2018 22:41)
Processor:
AMD Ryzen 7 1700X Eight-Core Processor (16 cores)
Memory:
32 GiB
Documentation used (tried every single method and a mix of all, also sub links posted in each threads...):
https://forums.freenas.org/index.php?threads/vlan-explain-please-on-how-to-do-it.65943/page-2
https://forums.freenas.org/index.php?threads/how-to-set-separate-vlan-for-jail.54019/
https://forums.freenas.org/index.php?threads/freenas-jails-in-different-multiple-subnets.41539/
https://gist.github.com/sdebnath/086874c5df8b68e0df69
100$ BOUNTY FOR WHOEVER HELPS ME FIX THIS!!!
So i've been trying to get this right for about 5 days now. I had around 4 reinstalls because i locked myself out (flood/storms or something like that). Also a few CLI config restores happend.
I'll first try to explain what hardware i have then ill explain what i try to achieve. I hope anyone can help me.
I have a OPNsense router 4 ports
- Gateway: 192.168.3.1 (LAN)
- DNS: 192.168.3.9
- Ifaces: WAN, LAN (192.168.3.0/24), HTPC (192.168.7.0/24 vlan tag 2), HASS (192.168.4.0/24 vlan tag 3), GUEST (192.168.5.0/24 vlan tag 4)
- Above VLAN's HASS and GUEST work great right now.
- Port 8 has vlan id 1 (regular untagged traffic)
- Port 8 has vlan id 2 (HTPC vlan in OPNsense and FreeNAS)
- Port 8 is connected to Freenas
- -----
- Port 7 has vlan id 1 (regular untagged traffic)
- Port 7 is connected to my OPNsense router at LAN
- -----
- Port 1 has vlan id 1 (regular untagged traffic)
- Port 1 has vlan id 3 (Guest AP)
- Port 1 has vlan id 4 (Home automation IP)
- Port 1 is connected to a TPlink AC wifi AP (Guest AP, Home automation AP)
- Other ports contain pc/laptop/rpi etc
OS Version:
FreeNAS-11.2-RELEASE-U1
(Build Date: Dec 20, 2018 22:41)
Processor:
AMD Ryzen 7 1700X Eight-Core Processor (16 cores)
Memory:
32 GiB
- IP DHCP from OPNsense (statically configured in router) 192.168.3.2
- GW: 192.168.3.1
- 1x VM on TAP0
- Several jails on VNET
- Bridge0 created in the past, don't know what for anymore, please advise. (please see config below)
- Separate jails from my main LAN, totally isolated, nothing to do with LAN just HTPC (vlan 2)
- Be able to add/change vlans and jails later on
- What do i set in the jail config page for:
- dhcp (i'd like to use it on vlan)
- bpf (security issue? I'm not that technical so the docs only confuse me more)
- vnet, i assume yes... (i need to access each jail via its ip from another subnet, configured with rules in router)
- ipv4_addr
- default gw
- exec_fib (tried the suggested fib 1 with the new routing rules... no avail)
- interfaces vnet0:bridge0 or vnet1:bridge1 or else? (because i dont want the HTPC jails in the regular LAN bridge)
- Raw sockets? (security concern? I'm not that technical so the docs only confuse me more)
Code:
root@freenas:~ # ifconfig igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6> ether 70:85:c2:62:06:62 hwaddr 70:85:c2:62:06:62 inet 192.168.3.2 netmask 0xffffff00 broadcast 192.168.3.255 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect (1000baseT <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:05:16:ba:17:00 nd6 options=9<PERFORMNUD,IFDISABLED> groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0:7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 12 priority 128 path cost 2000 member: vnet0:6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 11 priority 128 path cost 2000 member: vnet0:5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 10 priority 128 path cost 2000 member: vnet0:4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 5 priority 128 path cost 2000 member: vnet0:3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 9 priority 128 path cost 2000 member: vnet0:2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 8 priority 128 path cost 2000 member: vnet0:1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 7 priority 128 path cost 2000 member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 1 priority 128 path cost 20000 bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:05:16:ba:17:01 nd6 options=9<PERFORMNUD,IFDISABLED> groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> ether 00:bd:f1:10:f8:01 hwaddr 00:bd:f1:10:f8:01 nd6 options=1<PERFORMNUD> media: Ethernet autoselect status: active groups: tap Opened by PID 4302 vnet0:1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: plex options=8<VLAN_MTU> ether 02:ff:60:14:fa:09 hwaddr 02:dc:d0:00:07:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: backuppc options=8<VLAN_MTU> ether 02:ff:60:12:30:b6 hwaddr 02:dc:d0:00:08:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: bitwarden options=8<VLAN_MTU> ether 02:ff:60:ef:c8:55 hwaddr 02:dc:d0:00:09:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: guac options=8<VLAN_MTU> ether 02:ff:60:fa:f0:c0 hwaddr 02:dc:d0:00:05:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: nextcloud options=8<VLAN_MTU> ether 02:ff:60:ba:b5:81 hwaddr 02:dc:d0:00:0a:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: influxdb options=8<VLAN_MTU> ether 02:ff:60:92:dd:64 hwaddr 02:dc:d0:00:0b:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair vnet0:7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: organizr options=8<VLAN_MTU> ether 02:ff:60:e1:fe:c6 hwaddr 02:dc:d0:00:0c:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair
Code:
root@freenas:~ # iocage list +-----+-----------+-------+--------------+--------------+ | JID | NAME | STATE | RELEASE | IP4 | +=====+===========+=======+==============+==============+ | 2 | backuppc | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 3 | bitwarden | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 4 | guac | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 6 | influxdb | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | jackett | down | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | lidarr | down | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | manager | down | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 5 | nextcloud | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 7 | organizr | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | 1 | plex | up | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | radarr | down | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | sabnzbd | down | 11.2-RELEASE | DHCP | +-----+-----------+-------+--------------+--------------+ | - | sonarr | down | 11.2-RELEASE | 192.168.7.10 | +-----+-----------+-------+--------------+--------------+
Code:
root@freenas:~ # netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.3.1 UGS igb0 127.0.0.1 link#2 UH lo0 192.168.3.0/24 link#1 U igb0 192.168.3.2 link#1 UHS lo0 Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 ::1 link#2 UH lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 fe80::/10 ::1 UGRS lo0 fe80::%lo0/64 link#2 U lo0 fe80::1%lo0 link#2 UHS lo0 ff02::/16 ::1 UGRS lo0
Documentation used (tried every single method and a mix of all, also sub links posted in each threads...):
https://forums.freenas.org/index.php?threads/vlan-explain-please-on-how-to-do-it.65943/page-2
https://forums.freenas.org/index.php?threads/how-to-set-separate-vlan-for-jail.54019/
https://forums.freenas.org/index.php?threads/freenas-jails-in-different-multiple-subnets.41539/
https://gist.github.com/sdebnath/086874c5df8b68e0df69
Attachments
Last edited: