One-way connectivity from network to jail?

[funkyg]

New FreeNAS user here! Loving it so far.

I'm setting up Plex and can't get access to its web interface from outside freenas. I can ping the plex jail from inside freenas and, strangely, I can ping FROM the jail to a PC on the network but the ICMP responses never make it back to the jail.

I verified the MAC addresses are what they are supposed to be in a wireshark capture on the PC which tells me it's not being NATed. It does appear to be going through a layer 2 bridge inside freenas as expected.

Is there some firewall I don't know about that could be causing this? This is the latests 9.3 stable release.

PC FreeNA PLEX
10.0.0.1 --- 10.0.1.50 -- 10.0.1.51

.1 ping .50 works
.1 ping .51 FAILS
.50 ping .51 works
.51 ping .50 works
.51 ping .1 FAILS but I can see that .1 receives it and responds. Also .51 DOES have the correct MAC addresses in the ARP table.

Any ideas? I figure it must be something simple but in all the searching I've done I haven't found a solution yet. I've tried setting the jail with DHCP as well but it never gets an address from the network.

Thanks!

 

[anodos]

New FreeNAS user here! Loving it so far.

I'm setting up Plex and can't get access to its web interface from outside FreeNAS. I can ping the plex jail from inside FreeNAS and, strangely, I can ping FROM the jail to a PC on the network but the ICMP responses never make it back to the jail.

I verified the MAC addresses are what they are supposed to be in a wireshark capture on the PC which tells me it's not being NATed. It does appear to be going through a layer 2 bridge inside FreeNAS as expected.

Is there some firewall I don't know about that could be causing this? This is the latests 9.3 stable release.

PC FreeNA PLEX
10.0.0.1 --- 10.0.1.50 -- 10.0.1.51

.1 ping .50 works
.1 ping .51 FAILS
.50 ping .51 works
.51 ping .50 works
.51 ping .1 FAILS but I can see that .1 receives it and responds. Also .51 DOES have the correct MAC addresses in the ARP table.

Any ideas? I figure it must be something simple but in all the searching I've done I haven't found a solution yet. I've tried setting the jail with DHCP as well but it never gets an address from the network.

Thanks!

Is the jail using vimage?

 

[funkyg]

Is the jail using vimage?

Yes. And according to what I've read that means it should get a DHCP address since the VM shares the hardware interface with FreeNAS at layer 2. There is no NAT.

 

[pirateghost]

Are you running a /23 or something bigger? If not, I see an issue with your subnets.

 

[funkyg]

Are you running a /23 or something bigger? If not, I see an issue with your subnets.

It's 10.0.1.0/24

I made a mistake with my first IP. They're all in the 10.0.1 subnet

 

[pirateghost]

You say it's 9.3 release. Is there some reason you're on such an old version?

Did you set up a lagg on your NICs?

 

[funkyg]

You say it's 9.3 release. Is there some reason you're on such an old version?

Did you set up a lagg on your NICs?

Sorry... I'm on FreeNAS-9.10.1-U4 (ec9a7d3). Not 9.3.

No LAGG on any of the NICs either.

I am running FreeNAS on top of ESXi 5.5 but this shouldn't (I think) have anything to do with it. At least I can't think of any reason why this would be a problem. There are several virtual machines running on ESXi and they all share the same physical NIC but, like the jail, they all share it at layer 2. Each VM has its own MAC address.

Another difference is that I have FreeNAS connected to two networks. One that is externally facing that shares the physical NIC with the other VMs and another which is virtual and internal to ESXi. It's on a different subnet but the Jail doesn't touch it.

Here's the ifconfig of FreeNAS. The IPs are different from my example above. One interesting thing I just noticed is that the ARP tables of FreeNAS and the Jail have entries for the IP of each but the Jail thinks the IP of FreeNAS lives on the MAC address of the bridge. Is this right?

Code:

em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
		ether 00:0c:47:d3:3c:99
		inet6 fe80::20c:29ff:fed3:3c99%em0 prefixlen 64 scopeid 0x1
		inet 10.0.1.62 netmask 0xffffff00 broadcast 10.0.1.255
		inet6 2605:7000:1520:9040:20c:29ff:fed3:3c99 prefixlen 64 autoconf
		nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
		ether 00:0c:29:d3:3c:a3
		inet 192.168.50.3 netmask 0xffffff00 broadcast 192.168.50.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
		inet6 ::1 prefixlen 128
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
		inet 127.0.0.1 netmask 0xff000000
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		ether 02:26:1c:8a:5e:00
		nd6 options=9<PERFORMNUD,IFDISABLED>
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
		member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 5 priority 128 path cost 2000
		member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 1 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ff:20:00:05:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active

Here's part of the ARP table on FreeNAS.

Code:

arp -a
? (10.0.1.71) at 02:ff:24:00:06:0b on epair0a expires in 400 seconds [ethernet]

And here is ifconfig for the Jail:

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384			
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>				
		inet6 ::1 prefixlen 128												
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1							
		inet 127.0.0.1 netmask 0xff000000									
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>							
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>													
		ether 02:ff:24:00:06:0b												
		inet 10.0.1.71 netmask 0xffffff00 broadcast 10.0.1.255				
		inet6 fe80::ff:70ff:fe00:60b%epair0b prefixlen 64 scopeid 0x2		
		inet6 2605:7000:1520:8029:ff:70ff:fe00:60b prefixlen 64 autoconf		
		nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>				
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)					
		status: active

Here's the ARP table on the Jail:

Code:

arp -a											
? (10.0.1.40) at ac:24:a3:0d:0f:53 on epair0b expires in 1147 seconds [ethernet]
? (10.0.1.33) at 00:24:5b:ee:63:86 on epair0b expires in 1170 seconds [ethernet]
? (10.0.1.1) at 90:24:40:08:f5:1d on epair0b permanent [ethernet]			
? (10.0.1.71) at 02:ff:70:00:06:0b on epair0b permanent [ethernet]			
? (10.0.1.62) at 02:ff:20:00:05:0a on epair0b expires in 240 seconds [ethernet]

^^^ Why is this the MAC address of .62 from the perspective of the Jail? .62 lives on 00:0c:47:d3:3c:99 interface em0. When the Jail ARPs for .62 why is the bridge responding? Is it supposed to rewrite the MAC address and resend the packet like a router? That doesn't make sense. Also note that the other IPs listed here are outside of FreeNAS on the network. All these MAC addresses are correct so the bridge is layer 2 transparent.

Just for kicks here's part of the ARP table for 10.0.1.71 on a PC running out on the network. The MAC address is the same as the interface of the Jail.

Code:

10.0.1.71			 02-ff-70-00-06-0b

As a reminder 10.0.1.71 is set manually. DHCP does not work but IPv6 is able to obtain an address (not by DHCP but through the route advertisements or whatever they're called). Also, I cannot ping the IPv6 address of the Jail from the network nor can I ping an IPv6 address on the network from the Jail.

I should mention I think pings might be blocked on the Jail by default but I can't reach the web interface of Plex on port 32400 either. And yes, it is listening.

Code:

root@plexmediaserver_1:/ # netstat -an | grep -i listen						
tcp4	   0	  0 127.0.0.1.14779		*.*					LISTEN		
tcp4	   0	  0 127.0.0.1.14764		*.*					LISTEN		
tcp4	   0	  0 127.0.0.1.14710		*.*					LISTEN		
tcp4	   0	  0 127.0.0.1.14683		*.*					LISTEN		
tcp4	   0	  0 127.0.0.1.14677		*.*					LISTEN		
tcp4	   0	  0 *.32469				*.*					LISTEN		
tcp4	   0	  0 *.1843				 *.*					LISTEN		
tcp4	   0	  0 127.0.0.1.14583		*.*					LISTEN		
tcp4	   0	  0 127.0.0.1.32401		*.*					LISTEN		
tcp4	   0	  0 *.32400				*.*					LISTEN		
tcp4	   0	  0 10.0.1.71.12346		*.*					LISTEN

 

[pirateghost]

Well, running it in esx is a major part of this.

Enable promiscuous mode on the vnic

 

[funkyg]

Well, running it in esx is a major part of this.

Enable promiscuous mode on the vnic

That's it!

Thank you VERY much for your help. I did a search and I do see promiscuous mode mentioned in relation to ESXi and Plex on a couple blogs. Sorry I missed that. Would have saved me a few hours. ;)

Thanks again

 

[pirateghost]

It's important to provide as much information up front as possible. The rules state you should provide all details of your setup when asking a question. That one detail that it's running on esx would have saved you more than a day of wasted time. Yours and mine. ;)