I have a old FreeBSD server with bhyve hosts, and my virtual machines are on different networks via one network interface (RE0). In my switch I have two VLAN's configured. This setup works today, with no problem. The network port in switch where my server is connected allow vlan 1 and 2.
My new setup on other hardware with FreeNAS is almost the same, two or more virtual machines and one NIC in server. One of the virtual machine is on the other subnet, usning virtio NIC setup in FreeNAS, and this vm can not ping outside itself. I can't ping the vm from other hosts at all. If I create a alias IP on my EM0 interface on the same subnet... I can ping this IP from my vm host, but not other hosts on the same subnet.
I use the same network cable, I move it from old server to the new FreeNAS server. Everything works as expected on FreeNAS server and all vm's located on the same subnet as the FreeNAS server itself, the vm on the other subnet is not accessible at all.
Here is my network setup on FreeNAS server:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
ether 88:51:fb:5c:1f:e2
hwaddr 88:51:fb:5c:1f:e2
inet 192.168.1.230 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to Primus
options=80000<LINKSTATE>
ether 00:bd:a4:65:f8:00
hwaddr 00:bd:a4:65:f8:00
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 4048
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:cf:9e:36:18:00
nd6 options=1<PERFORMNUD>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000000
member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 2000000
tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to HomeAssistant
options=80000<LINKSTATE>
ether 00:bd:0e:72:f8:01
hwaddr 00:bd:0e:72:f8:01
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 4139
tap2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to Barcadi
options=80000<LINKSTATE>
ether 00:bd:78:22:77:02
hwaddr 00:bd:78:22:77:02
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 18018
And here my IP info VM machine (Barcadi) that is not working:
IP address: 192.168.2.10/24
Gateway: 192.168.2.1
I'm using the same Switch and Firewall in both cases (old and new server).
And pinging the host from FreeNAS server is not working:
root@freenas[~]# ping 192.168.2.10
PING 192.168.2.10 (192.168.2.10): 56 data bytes
92 bytes from 192.168.1.1: Destination Host Unreachable
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 85f8 0 0000 3f 01 7070 192.168.1.230 192.168.2.10
Please help me, maybe I forgot something very easy...otherwise I need to re-install my new FreeNAS server and only use FreeBSD as I did on my old server.
My new setup on other hardware with FreeNAS is almost the same, two or more virtual machines and one NIC in server. One of the virtual machine is on the other subnet, usning virtio NIC setup in FreeNAS, and this vm can not ping outside itself. I can't ping the vm from other hosts at all. If I create a alias IP on my EM0 interface on the same subnet... I can ping this IP from my vm host, but not other hosts on the same subnet.
I use the same network cable, I move it from old server to the new FreeNAS server. Everything works as expected on FreeNAS server and all vm's located on the same subnet as the FreeNAS server itself, the vm on the other subnet is not accessible at all.
Here is my network setup on FreeNAS server:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
ether 88:51:fb:5c:1f:e2
hwaddr 88:51:fb:5c:1f:e2
inet 192.168.1.230 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to Primus
options=80000<LINKSTATE>
ether 00:bd:a4:65:f8:00
hwaddr 00:bd:a4:65:f8:00
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 4048
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:cf:9e:36:18:00
nd6 options=1<PERFORMNUD>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000000
member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 2000000
tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to HomeAssistant
options=80000<LINKSTATE>
ether 00:bd:0e:72:f8:01
hwaddr 00:bd:0e:72:f8:01
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 4139
tap2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to Barcadi
options=80000<LINKSTATE>
ether 00:bd:78:22:77:02
hwaddr 00:bd:78:22:77:02
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 18018
And here my IP info VM machine (Barcadi) that is not working:
IP address: 192.168.2.10/24
Gateway: 192.168.2.1
I'm using the same Switch and Firewall in both cases (old and new server).
And pinging the host from FreeNAS server is not working:
root@freenas[~]# ping 192.168.2.10
PING 192.168.2.10 (192.168.2.10): 56 data bytes
92 bytes from 192.168.1.1: Destination Host Unreachable
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 85f8 0 0000 3f 01 7070 192.168.1.230 192.168.2.10
Please help me, maybe I forgot something very easy...otherwise I need to re-install my new FreeNAS server and only use FreeBSD as I did on my old server.
