Accessing FreeNAS host storage from bhyve guest

[John Rushford]

Greetings,

I'm a new FreeNAS user and have just finished building my first NAS with FreeNAS 9.10. I'm using the NAS as a CIFS file share on my local network for Apple Mac OS macbook laptops. Also I'm using the NAS as a plex media server for music, TV shows and movies. So I build a jail on the NAS running plex media server and spent a lot of time getting videos and music in the library I set aside on the FreeNAS ZFS volume. So far everything is working fantastically, hats off to the FreeNAS developers.

After reading Jordan Hubbard's announcement on FreeNAS 9.10, I started playing with iohyve I now have a FreeBSD 10.3 guest running. I've added the rc.conf tunable's for iohyve and set the boot flag on the guest zfs file system so that /usr/local/etc/rc.d/iohyve starts this guest automatically. I now plan to replace the jail running plex media server with this bhyve vm but am wondering how to import my library to the vm from the FreeNAS host. I think that at this point, I'm going to have to use NFS to do that but before I do that, I was wondering if anyone might know of a better way for the vm guest to access my storage on the ZFS volume on the FreeNAS host? Is there another way to make my media zfs filesystem available to the vm besides NFS?

 

[fracai]

For now I've been using NFS. It looks like v10 will bring 9pfs.

 

[John Rushford]

For now I've been using NFS. It looks like v10 will bring 9pfs.

Okay, good to know. Looking forward to V10.

 

[Chris Moore]

For now I've been using NFS. It looks like v10 will bring 9pfs.

I did search first, that is how I found this, but I didn't find the answer I was looking for. I hate to resurrect such an old question but it appears applicable and unanswered.
I have a VM that I created under FreeNAS 11.1 and it can access the internet, but not access my SMB share on the FreeNAS that it is running on.
Any guidance?

 

[KrisBee]

@Chris Moore What OS is running in your VM? Any firewalling that might be blocking SMB?

 

[Chris Moore]

@Chris Moore What OS is running in your VM? Any firewalling that might be blocking SMB?

I wanted to test the functionality of creating a VM before I did anything more serious, so I just loaded the Mint Linux 18.3 and have not made any changes to it. It boots and I can VNC into it and use the Linux install in any way I might want, including to brows the internet and watch videos although the video isn't smooth. It works great with the exception that I can't access any systems on my local network. I even created a share on my windows computer that I can access from other computers on the network but the Linux VM can't. The error appears to be a timeout on the windows system but it appears as a permission denied when trying to access the NAS.
I have not tried to setup any firewall. I was thinking that there might be some network setting that needs to be adjusted in the virtual network that the VM is attached to.

 

[colmconn]

Since it's a linux box, have you tried NFS? I've got a freebsd VM running that accesses my data through NFS export from the host. It's not the fastest but it works.

 

[KrisBee]

I wanted to test the functionality of creating a VM before I did anything more serious, so I just loaded the Mint Linux 18.3 and have not made any changes to it. It boots and I can VNC into it and use the Linux install in any way I might want, including to brows the internet and watch videos although the video isn't smooth. It works great with the exception that I can't access any systems on my local network. I even created a share on my windows computer that I can access from other computers on the network but the Linux VM can't. The error appears to be a timeout on the windows system but it appears as a permission denied when trying to access the NAS.
I have not tried to setup any firewall. I was thinking that there might be some network setting that needs to be adjusted in the virtual network that the VM is attached to.

Does Mint Linux 18.3 have ufw installed and is it disabled? What does smbclient show in Mint? Were you trying to mount cifs share via CLI in Mint? How complex is your FreeNAS ifconfig?

 

[Chris Moore]

Since it's a linux box, have you tried NFS? I've got a freebsd VM running that accesses my data through NFS export from the host. It's not the fastest but it works.

That might be a way to solve this, but I would ultimately like to be able to install a Windows VM (or several) and it probably won't work for that.

 

[KrisBee]

A linux VM is perfectly capable of connecting to SMB share on the FreeNAS host if it is configured correctly.

 

[Chris Moore]

Does Mint Linux 18.3 have ufw installed and is it disabled?

Unless that comes with Mint, which I doubt, but I certainly didn't disable it. I have a physical Linux installation that I have been using for years and it never had a problem accessing the share but that doesn't prove anything.

What does smbclient show in Mint?

I will have to run that once I get home.

Were you trying to mount cifs share via CLI in Mint?

No, I am able to browse the network from the virtual Linux Mint (LM) system in the same way that I can with the physical LM, but with the physical system, when I click on the FreeNAS system I am shown a list of shares and on the virtual system it won't even show me the list of shares available. Sometimes the virtual LM system doesn't even see that there are other systems on the network, but when it does, it won't even look to see what they are sharing.

How complex is your FreeNAS ifconfig?

I have one interface on the FreeNAS connected to my LAN, and I have made no setting changes. I set a static IP address on the system and I have not adjusted it other than the automatic changes that the system made when I created warden jails and the settings made by the system when I created this VM which I am guessing is a bhyve guest. I can't recall when they introduced this relatively new VM structure, but I played with it once when it first came out and couldn't get a VM to even finish installing, windows or Linux, so I gave it some time to mature before giving it another go.

 

[Chris Moore]

A linux VM is perfectly capable of connecting to SMB share on the FreeNAS host if it is configured correctly.

I suppose there could be some residual settings hanging around from things I have done in the past. I have been running this system for around 5 years and just updating to the new FreeNAS / adding or removing features.
Would it help if I posted / uploaded the text that it dumps to email after a reboot?

 

[KrisBee]

If you created the VM via the FN11.1 webui it will be a bhyve guest. ( Just tail /var/log/middlewared.log to see the bhyve command that's executed on starting the VM. )

Did you select virtio for the VM NIC device? If not, I'd would switch to that and reconfigure the VM's network as necessary, perhaps choosing a static IP address for the VM. There should be nothing to do in your FreeNAS network. although it might be worth adding the VM to the FreeNAS host list in the FreeNAS network global config to eliminate a possible reverse DNS problem.

I would double check the presence of a possible firewall on Mint and, assuming smbclient returns something useful, see if you can mount the SMB share via the CLI , bypassing the Mint file manager.

It is possible that editing /etc/samba/smb.conf in your Mint VM will change/help the way samba client resolves host names.

 

[KrisBee]

@Chris Moore Did you make any progress with this? I don't run graphical linux VMs, but after trying out a couple of linux live isos the results are mixed. Kubuntu, with the dolphin file manager, has no problem finding windows shares, but bunsenlab ( debian, openbox + thunar file mangaer) does have a problem. Using gigolo in bunsenlabs works, as does connecting via a CLI mount, and this can been seen using smbstatus in FreeNAS.

Running bunsenlabs on a different host to my FreeANS box has no problem seeing and browsing windows shares using the thunar file manager, so that's a puzzle. Nor does Ubunut18 desktop have any problem connecting to smb shares via the nautilus file manager (can't run that as a VM on FN11 as bhyve doesn't support linux kernel >=4.15 until FN11.2).

Which DE are you using in Linux Mint: Cinnamon, Mate or Xfce4?

 

[KrisBee]

After a bit of poking around this seems to boil down to name resolution and the default config of a given linux distro for remote file access via its file manager. Assuming a simple network with no SAMBA AD-DC and no local DNS server, the Ubuntu family seems to be set up with the avahi pakages installed, the avahi-daemon running and a "hosts" line in /etc/nsswitch.conf which uses mDNS, e.g:

Code:

hosts:		  files mdns4_minimal [NOTFOUND=return] dns myhostname

In contrast, the debian based distro bunsenlabs has samba winbind daemon running and needs "wins" added the "host" line of /etc/nsswitch.conf in order to use "smb://<freenas ip>" as a location in its file manager, thunar, when this distro is used as a VM.

So what is the setup in Mint Linux?

 

[Chris Moore]

Did you make any progress with this?

I didn't get a lot of time to work on it last night, but I did change to the virtio NIC like you suggested and restarted the VM. It didn't appear to make a change to the behavior. The things that work are still working and the thing that doesn't work...

Which DE are you using in Linux Mint: Cinnamon, Mate or Xfce4?

I went with the Xfce interface because I was thinking it would be more light-weight. The physical Linux system I have has been running for so many years that I don't even remember what it started out with but I recall there were some problems with it to begin with that I had to tinker with.
I will do some more work on this tonight and get back to you. I really appreciate your help.

 

[KrisBee]

I was curious, so I've downloaded a linux Mint 18.3 xfce4 iso and created a VM. In live mode, browsing the network shows the correct workgoup folders but you can only see the share folders by typing

Code:

smb://<freenas ip>

in the location bar of the thunar file manger.

Once installed, and fully updated except for the newer 4.15 kernel, the situation is the same as in live mode. Not surprisingly, Ubuntu based Mint Linux follows the avahi-daemon + mDNS config default config. The "Failed to retrieve share list from server" message appears to have various diagnosis on Ask Ubuntu. Still looking at this ...

 

[Chris Moore]

I was curious, so I've downloaded a linux Mint 18.3 xfce4 iso and created a VM. In live mode, browsing the network shows the correct workgoup folders but you can only see the share folders by typing

Code:

smb://<freenas IP>

in the location bar of the thunar file manger.

Once installed, and fully update except for the newer 4.15 kernel, the situation is the same as in live mode. Not surprisingly, Ubuntu based Mint Linux follows the avahi-daemon + mDNS config default config. The "Failed to retrieve share list from server" message appears to various diagnosis on Ask Ubuntu. Still looking at this ...

I sure appreciate the effort.


Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[KrisBee]

@Chris Moore I made time to return to this question. The failure to browse FreeNAS host samba shares from within a Mint VM appears to be due to the "name request" never reaching or being replied to by the FreeNAS SAMBA server.

Using smbtree in debug mode in the VM shows the "broadcast look up" fails - see pic below.

A simple packet sniff on the FreeNAS network shows the smbtree command generates NBT UDP packets on all the relevant interfaces but a response packet never appears, nor does the nmbd log show any incoming name request from the IP associated with the Mint VM.

Code:

root@freenasB:~ # tcpdump -i tap0 ether broadcast	
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:48:34.099872 IP mint-vm.47719 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:48:35.101421 IP mint-vm.57268 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:48:36.103245 IP mint-vm.53257 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:48:46.000146 IP sweep.3483 > 255.255.255.255.3483: UDP, length 26
^C
4 packets captured
13 packets received by filter
0 packets dropped by kernel
root@freenasB:~ # tcpdump -i bridge0 ether broadcast	
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bridge0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:49:03.495009 IP mint-vm.59623 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:49:04.498494 IP mint-vm.45294 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:49:05.501098 IP mint-vm.37849 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
^C
3 packets captured
19 packets received by filter
0 packets dropped by kernel
root@freenasB:~ # tcpdump -i em0 ether broadcast	
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:49:23.378951 IP mint-vm.56691 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:49:24.381062 IP mint-vm.38718 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
09:49:25.382633 IP mint-vm.55875 > 192.168.0.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST

When you use the thunar file manger to attempt to browse the smb shares it generates the same NBT UDP packets on the FreeNAS host. But as there is no server response, the error message "Failed to retrieve share list ... " appears - see pic 2 below.

Turning "zeroconf share discovery" on the FreeNAS SMB server does not help as both the relevant avahi-resolve and dig commands fail to resolve. Again the correct multicast packets seem to appear of the FreeNAS side, but there's no response.

Code:

root@freenasB:~ # tcpdump -i em0 ether multicast and host 192.168.0.90
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:59:03.667804 IP mint-vm.mdns > 224.0.0.251.mdns: 0 [2q] A (QM)? freenasB.local. AAAA (QM)? freenasB.local. (38)
09:59:04.675123 IP mint-vm.mdns > 224.0.0.251.mdns: 0 [2q] A (QM)? freenasB.local. AAAA (QM)? freenasB.local. (38)
09:59:06.680139 IP mint-vm.mdns > 224.0.0.251.mdns: 0 [2q] A (QM)? freenasB.local. AAAA (QM)? freenasB.local. (38)
^C
3 packets captured
7235 packets received by filter
0 packets dropped by kernel
root@freenasB:~ # tcpdump -i em0 ether multicast and host 192.168.0.90
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:59:36.788403 IP mint-vm.59356 > 224.0.0.251.mdns: 27831+ [1au] A (QM)? freenasB.local. (43)
09:59:41.794803 IP mint-vm.59356 > 224.0.0.251.mdns: 27831+ [1au] A (QM)? freenasB.local. (43)
10:00:30.158576 IP mint-vm > igmp.mcast.net: igmp v3 report, 1 group record(s)
10:02:35.696111 IP mint-vm > igmp.mcast.net: igmp v3 report, 1 group record(s)

I've included my FreeNAS ifconfig and sysctl net.link output below for completeness, but at this point need a more knowledgeable opinion.

Code:

root@freenasB:~ # ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
		ether 68:05:ca:16:f3:fe
		hwaddr 68:05:ca:16:f3:fe
		inet 192.168.0.99 netmask 0xffffff00 broadcast 192.168.0.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
		ether 9c:5c:8e:7c:79:26
		hwaddr 9c:5c:8e:7c:79:26
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (none)
		status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
		inet6 ::1 prefixlen 128
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
		inet 127.0.0.1 netmask 0xff000000
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
		groups: lo
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=80000<LINKSTATE>
		ether 00:bd:bc:fa:13:00
		hwaddr 00:bd:bc:fa:13:00
		nd6 options=1<PERFORMNUD>
		media: Ethernet autoselect
		status: active
		groups: tap
		Opened by PID 6572
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		ether 02:bf:32:fa:02:00
		nd6 options=9<PERFORMNUD,IFDISABLED>
		groups: bridge
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
		member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 1 priority 128 path cost 20000
		member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 4 priority 128 path cost 2000000
root@freenasB:~ # sysctl net.link
net.link.generic.system.ifcount: 5
net.link.ether.inet.allow_multicast: 0
net.link.ether.inet.log_arp_permanent_modify: 1
net.link.ether.inet.log_arp_movements: 1
net.link.ether.inet.log_arp_wrong_iface: 1
net.link.ether.inet.garp_rexmit_count: 0
net.link.ether.inet.max_log_per_second: 1
net.link.ether.inet.maxhold: 1
net.link.ether.inet.wait: 20
net.link.ether.inet.proxyall: 0
net.link.ether.inet.maxtries: 5
net.link.ether.inet.max_age: 1200
net.link.ether.ipfw: 0
net.link.vlan.mtag_pcp: 0
net.link.vlan.soft_pad: 0
net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 0
net.link.bridge.inherit_mac: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 1
net.link.bridge.pfil_onlyip: 1
net.link.gif.parallel_tunnels: 0
net.link.gif.max_nesting: 1
net.link.tap.debug: 0
net.link.tap.devfs_cloning: 1
net.link.tap.up_on_open: 0
net.link.tap.user_open: 0
net.link.tun.devfs_cloning: 1
net.link.lagg.default_flowid_shift: 16
net.link.lagg.default_use_flowid: 1
net.link.lagg.failover_rx_all: 0
net.link.lagg.lacp.default_strict_mode: 0
net.link.lagg.lacp.debug: 0
net.link.epair.netisr_maxqlen: 2100
net.link.log_promisc_mode_change: 1
net.link.log_link_state_change: 1
net.link.ifqmaxlen: 50
root@freenasB:~ #