big12boy
Cadet
- Joined
- Oct 29, 2020
- Messages
- 4
Hello everyone,
3 days ago I updated in the hopes of finally having native support for my 10GBE NIC (AQtion AQC107, issue NAS-101990).
The update went fine and I added the tunable to enable to NIC afterwards (was done before at the file level).
After a reboot I now have access to the NAS via LAN and services like SMB are working.
I then noticed that neither VMs, nor jails have access to my LAN.
What is working:
What I suspect (but haven't found a way to try/fix it yet):
ifconfig on TrueNAS:
netstat on TrueNAS: (192.168.1.150 is my main PC, 192.168.1.254 is the gateway/router)
ifconfig in jail: (using fixed ip)
iocage settings for the jail:
ipconfig and ping test Windows VM:
Information to my NAS can be found in my signature. Debug file: forum would not let me attach it, therefore linked here
I hope someone has an answer for me, I already upgraded my ZFS storage pool, as everything seemed to be working at first and I'm now unable to go back, having to download ~15TB from my online backup.
Thank you!
Marcel
3 days ago I updated in the hopes of finally having native support for my 10GBE NIC (AQtion AQC107, issue NAS-101990).
The update went fine and I added the tunable to enable to NIC afterwards (was done before at the file level).
After a reboot I now have access to the NAS via LAN and services like SMB are working.
I then noticed that neither VMs, nor jails have access to my LAN.
What is working:
- Pinging TrueNAS from jail/VM
- Pinging jail from VM
- Pinging jails from TrueNAS and LAN
- DHCP, DNS requests (or anything else that has to go out thru aq0) in the jails/VMs
- Pinging VM from jail
- connection with and without vnet set in the jails => without vnet it is working (does not help me with DHCP or VMs though):
- fixed IP addresses for VMs/jails => still no connection to the network, used this setting for ping testing
- setting vnet_default_interface and vnet_interfaces and IPv4 Interface in the jails
- uncheck Berkeley Packet Filter and set fixed IP
- follow advice from Ubuntu VMs (not the same issue)
- similar issue after update to 11.2, set correct netmask, allow raw sockets, set interface, generate new MAC
- there are many issues floating around where there seems to be issues with just jails (configuration of the interface in the jail, routing and so on), but these issues seem not to be related with this problem. The ones that were more general I tried anyway.
- adding/removing vnet0, bridge0 and resetting aq0 interfaces using ifconfig
- adding tunables to make sure bridge0 is created and members (aq0 & vnet0) are added
- set properties for the bridge
- net.link.bridge.ipfw = 1
- net.link.bridge.pfil_bridge = 0
- net.link.bridge.pfil_member = 0
- net.link.bridge.pfil_local_phys = 0
- FreeBSD Forum: Routing between bridged interfaces
- tried a complete new install, issue still persistent (had to do kldload if_atlantic once on first install to get the GUI and add the tunable)
- other ways of loading the driver earlier (was seeing very late link up compared to the intel and loopback nics)
- Also changed the rc.conf (in both /etc/ and /conf/...) before first boot on a new install, to make sure it is available during initial setup
What I suspect (but haven't found a way to try/fix it yet):
- driver unable to make use of bridging
- routing tables gone wrong
ifconfig on TrueNAS:
Code:
root@freenas[~]# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
aq0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: 10g
options=8100b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER>
ether 70:85:c2:d0:e2:37
inet 192.168.1.200 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect <full-duplex,rxpause,txpause> (10Gbase-T <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:4d:29:b4:33:00
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000
member: vnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000000
member: aq0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 2000
groups: bridge
nd6 options=1<PERFORMNUD>
vnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether fe:a0:98:03:af:ac
hwaddr 58:9c:fc:10:ff:a3
groups: tap
media: Ethernet autoselect
status: active
nd6 options=1<PERFORMNUD>
Opened by PID 2442
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: test as nic: epair0b
options=8<VLAN_MTU>
ether 72:85:c2:ae:1b:75
hwaddr 02:91:e8:b8:59:0a
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=1<PERFORMNUD>netstat on TrueNAS: (192.168.1.150 is my main PC, 192.168.1.254 is the gateway/router)
Code:
root@freenas[~]# netstat -4 Active Internet connections Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 localhost.x11 localhost.15890 ESTABLISHED tcp4 0 0 localhost.15890 localhost.6000 ESTABLISHED tcp4 0 0 freenas.http 192.168.1.150.56788 ESTABLISHED tcp4 0 0 localhost.6205 localhost.41816 ESTABLISHED tcp4 0 0 localhost.41816 localhost.6205 ESTABLISHED tcp4 0 0 freenas.6105 192.168.1.150.56657 ESTABLISHED tcp4 0 0 freenas.ssh win10vm.49679 ESTABLISHED tcp4 0 0 localhost.6205 localhost.16278 ESTABLISHED tcp4 0 0 localhost.16278 localhost.6205 ESTABLISHED tcp4 0 0 freenas.6105 srv-01.64244 ESTABLISHED tcp4 0 0 freenas.ssh 192.168.1.150.55230 ESTABLISHED tcp4 0 52 freenas.ssh 192.168.1.150.55229 ESTABLISHED tcp4 0 0 freenas.ssh win10vm.49673 ESTABLISHED tcp4 0 0 freenas.microsoft-ds 192.168.1.150.55212 ESTABLISHED tcp4 0 0 localhost.x11 localhost.27900 ESTABLISHED tcp4 0 0 localhost.27900 localhost.6000 ESTABLISHED tcp4 0 0 freenas.http 192.168.1.150.55211 ESTABLISHED tcp4 0 0 freenas.nut openhab.33140 ESTABLISHED tcp4 0 0 freenas.nut srv-01.63985 ESTABLISHED tcp4 0 0 localhost.nut localhost.27895 ESTABLISHED tcp4 0 0 localhost.27895 localhost.3493 ESTABLISHED tcp4 0 0 localhost.nut localhost.27894 ESTABLISHED tcp4 0 0 localhost.27894 localhost.3493 ESTABLISHED udp4 0 0 localhost.1031 *.* udp4 0 0 freenas.ntp *.* udp4 0 0 localhost.ntp *.* udp4 0 0 239.255.255.250.3702 *.*
ifconfig in jail: (using fixed ip)
Code:
root@test:/ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 72:85:c2:ae:1b:76
hwaddr 02:63:cb:f0:cc:0b
inet 192.168.1.197 netmask 0xffffff00 broadcast 192.168.1.255
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=1<PERFORMNUD>iocage settings for the jail:
Code:
root@freenas[~]# iocage get -a test CONFIG_VERSION:28 allow_chflags:0 allow_mlock:0 allow_mount:0 allow_mount_devfs:0 allow_mount_fusefs:0 allow_mount_nullfs:0 allow_mount_procfs:0 allow_mount_tmpfs:0 allow_mount_zfs:0 allow_quotas:0 allow_raw_sockets:1 allow_set_hostname:1 allow_socket_af:0 allow_sysvipc:0 allow_tun:0 allow_vmm:0 assign_localhost:0 available:readonly basejail:0 boot:0 bpf:1 children_max:0 cloned_release:12.2-RELEASE comment:none compression:lz4 compressratio:readonly coredumpsize:off count:1 cpuset:off cputime:off datasize:off dedup:off defaultrouter:192.168.1.254 defaultrouter6:auto depends:none devfs_ruleset:4 dhcp:0 enforce_statfs:2 exec_clean:1 exec_created:/usr/bin/true exec_fib:0 exec_jail_user:root exec_poststart:/usr/bin/true exec_poststop:/usr/bin/true exec_prestart:/usr/bin/true exec_prestop:/usr/bin/true exec_start:/bin/sh /etc/rc exec_stop:/bin/sh /etc/rc.shutdown exec_system_jail_user:0 exec_system_user:root exec_timeout:60 host_domainname:none host_hostname:test host_hostuuid:test host_time:1 hostid:03DE0294-0480-050E-5106-CC0700080009 hostid_strict_check:0 interfaces:vnet0:bridge0 ip4:new ip4_addr:vnet0|192.168.1.197/24 ip4_saddrsel:1 ip6:new ip6_addr:none ip6_saddrsel:1 ip_hostname:0 jail_zfs:0 jail_zfs_dataset:iocage/jails/test/data jail_zfs_mountpoint:none last_started:2020-10-29 13:49:13 localhost_ip:none login_flags:-f root mac_prefix:7285c2 maxproc:off memorylocked:off memoryuse:off min_dyn_devfs_ruleset:1000 mount_devfs:1 mount_fdescfs:1 mount_linprocfs:0 mount_procfs:0 mountpoint:readonly msgqqueued:off msgqsize:off nat:0 nat_backend:ipfw nat_forwards:none nat_interface:none nat_prefix:172.16 nmsgq:off notes:none nsem:off nsemop:off nshm:off nthr:off openfiles:off origin:readonly owner:root pcpu:off plugin_name:none plugin_repository:none priority:99 pseudoterminals:off quota:none readbps:off readiops:off release:12.2-RELEASE reservation:none resolver:/etc/resolv.conf rlimits:off rtsold:0 securelevel:2 shmsize:off stacksize:off state:up stop_timeout:30 swapuse:off sync_state:none sync_target:none sync_tgt_zpool:none sysvmsg:new sysvsem:new sysvshm:new template:0 type:jail used:readonly vmemoryuse:off vnet:1 vnet0_mac:7285c2ae1b75 7285c2ae1b76 vnet0_mtu:auto vnet1_mac:none vnet1_mtu:auto vnet2_mac:none vnet2_mtu:auto vnet3_mac:none vnet3_mtu:auto vnet_default_interface:auto vnet_default_mtu:1500 vnet_interfaces:none wallclock:off writebps:off writeiops:off
ipconfig and ping test Windows VM:
Information to my NAS can be found in my signature. Debug file: forum would not let me attach it, therefore linked here
I hope someone has an answer for me, I already upgraded my ZFS storage pool, as everything seemed to be working at first and I'm now unable to go back, having to download ~15TB from my online backup.
Thank you!
Marcel
