Jail's alias IP taking over netbios name

[philiplu]

I'm setting up my first FreeNAS box, with version 9.2.1.7. This is on a SM X10SL7-F, with only one of the two NICs plugged in (plus the IPMI NIC). I'm now playing around with jails, and ran into a problem. I've set up a default portjail, with VIMAGE unchecked so there's a shared network stack. The NAS is on 192.168.1.10, and the jail is assigned 192.168.1.200. DHCP is via an Asus RT-AC66U router, with the DHCP range set to 192.168.1.20 to 192.168.1.199, so there's no conflict there.

The problem is that I'm no longer able to reliably browse or SSH to the NAS by name. If I "ping marvin" (the NAS) from a Win 7 cmdline, I sometimes see 192.168.1.200, and sometimes 192.168.1.10. If the jail is running, then browsing shares still works by name even if the netbios name MARVIN is set to .200 instead of .10. But if I stop the jail, browsing to \\marvin (or attempting to open a file via \\marvin\share\somefile) hangs and fails while MARVIN is still cached on the Windows machine and apparently on the NAS as well. If I clear the cache under windows with "nbtstat -R" and recycle the CIFS service via the FreeNAS GUI, then a ping to marvin finally gets the .10 address I'm expecting.

Also, if the jail is up and MARVIN is at .200, then a SSH to marvin (via putty on Win 7) ends up in the jail and not the main OS. That's what actually first alerted me that something was wrong - putty was complaining about the fingerprint not matching the cached value on startup, which made me think I'd broken the sshd_config somehow. But no, turning on putty logging showed that I was connecting to the .200 address, not the expected .10 (at least, after the jail had sshd running).

I'm running a simple workgroup network here - no AD, no WINS, no local DNS, so name resolution is all by netbios broadcast queries. Here are some dumps to show my configuration. First, smb4.conf's [global] section - I've added "preferred master = yes" and "os level = 255" in the CIFS Auxiliary parameters through the GUI, just to make sure FreeNAS is the local browser master:

Code:

[global]
    server max protocol = SMB2
    encrypt passwords = yes
    dns proxy = no
    strict locking = no
    oplocks = yes
    deadtime = 15
    max log size = 51200
    max open files = 11070
    syslog only = yes
    syslog = 1
    load printers = no
    printing = bsd
    printcap name = /dev/null
    disable spoolss = yes
    getwd cache = yes
    guest account = nobody
    map to guest = Bad User
    obey pam restrictions = Yes
    directory name cache size = 0
    kernel change notify = no
    panic action = /usr/local/libexec/samba/samba-backtrace
    server string = Main FreeNAS Server
    ea support = yes
    store dos attributes = yes
    time server = yes
    acl allow execute always = false
    local master = yes
    idmap config *:backend = tdb
    idmap config *:range = 90000000-100000000
    server role = standalone
    netbios name = MARVIN
    workgroup = LUCIDOHOME
    security = user
    pid directory = /var/run/samba
    smb passwd file = /var/etc/private/smbpasswd
    private dir = /var/etc/private
    create mask = 0644
    directory mask = 0755
    client ntlmv2 auth = yes
    dos charset = CP437
    unix charset = UTF-8
    log level = 2
    preferred master = yes
    os level = 255

Here's "netstat -rn":

Code:

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.1.1        UGS         0      783   igb0
127.0.0.1          link#6             UH          0  6640162    lo0
192.168.1.0/24     link#2             U           0 20878440   igb0
192.168.1.10       link#2             UHS         0       96    lo0
192.168.1.200      link#2             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#6                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#6                        U           lo0
fe80::1%lo0                       link#6                        UHS         lo0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%lo0/32                     ::1                           U           lo0

Here's ifconfig:

Code:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 0c:c4:7a:30:06:82
        inet 192.168.1.10 netmask 0xffffff00 broadcast 192.168.1.255
        inet 192.168.1.200 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
        ether 0c:c4:7a:30:06:83
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Finally, here's nmblookup run on marvin, showing the netbios name resolution returning the jail's IP:

Code:

Marvin# nmblookup -d=3 marvin
lp_load_ex: refreshing parameters
Initialising global parameters
max_open_files: increasing sysctl_max (11095) to minimum Windows limit (16384)
rlimit_max: increasing rlimit_max (11095) to minimum Windows limit (16384)
params.c:pm_process() - Processing configuration file "/usr/local/etc/smb4.conf"
Processing section "[global]"
added interface igb0 ip=192.168.1.10 bcast=192.168.1.255 netmask=255.255.255.0
added interface igb0 ip=192.168.1.200 bcast=192.168.1.255 netmask=255.255.255.0
Socket opened.
name_resolve_bcast: Attempting broadcast lookup for name marvin<0x0>
Got a positive name query response from 192.168.1.10 ( 192.168.1.200 )
192.168.1.200 marvin<00>

There are probably a bunch of ways to work around this: /etc/hosts, /etc/local/lmhosts, set up a WINS server, set up a local DNS server on my router, using a jail with VIMAGE so it's got an independent network stack, .... But I'm trying to understand why this default configuration is hitting this. Seems unexpected.

I noticed that the man pages for ifconfig (here) mention under the "alias" parameter than if an alias address is on the same subnet as the 1st address, a non-conflicting netmask must be given. That's not what I've got set here, but when I configure the jail with a /32 netmask instead of /24, I still see the same unexpected behavior.

 

[cyberjock]

So here's the problem...

When you choose to not use VIMAGE the machine now uses 2 IPs. I created a jail and here's what you see outside the jail....

Code:

ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
  ether d0:50:99:1b:64:4b
  inet 192.168.2.109 netmask 0xffffff00 broadcast 192.168.2.255
  inet 192.168.2.131 netmask 0xffffff00 broadcast 192.168.2.255
  nd6 options=9<PERFORMNUD,IFDISABLED>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active

Inside the jail you see:

Code:

ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
  ether d0:50:99:1b:64:4b
  inet 192.168.2.131 netmask 0xffffff00 broadcast 192.168.2.255
  nd6 options=9<PERFORMNUD,IFDISABLED>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active

Now the network stacks are meshed together and use a single MAC address. Your router must be picking up the second device and choosing to use both addresses for "marvin".

Unfortunately I think the problem is going to be with your router picking up stuff it probably shouldn't. To be honest VIMAGE works just fine for me and I use it for every jail because it avoids certain problems (like this one). In my case when I do VIMAGE in the same jail my MAC changes to 02:69:47:00:08:0b.

Also if you look outside the jail you'll see bridge0 and you'll see this:

Code:

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  ether 02:fe:4a:c8:9c:00
  nd6 options=1<PERFORMNUD>
  id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
  maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
  root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
  member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
  ifmaxaddr 0 port 7 priority 128 path cost 2000
  member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
  ifmaxaddr 0 port 1 priority 128 path cost 20000

I'm not 100% certain on this, but I believe the problem is that both your FreeNAS machine and the jail are announcing themselves from the same MAC and so your router is just assuming that both must be the same machine. That's not the case but the router doesn't seem to know that. Unless you have a pressing reason to not use VIMAGE I'd recommend you just enable it and be happy. It "should" solve this problem. ;)

Edit: Another possible fix is to do DHCP on your FreeNAS machine and setup your router to use a static IP. Usually a static IP won't have multiple IPs attached to it from the software-side of things so you won't have this screwed up conflict. But as I refuse to buy store-bought stuff like what you have (I use pfsense) I can't vouch that this *will* work on your router. It's just a guess.

 

[philiplu]

Thanks, I figured VIMAGE was going to be the answer. FWIW, I was following your guide on setting up Minecraft in a jail, with VIMAGE unchecked as instructed, as a trial run on jail setup when I ran into the problems trying to ssh to FreeNAS (not the jail). Might be a good idea to say something about enabling VIMAGE in that guide. And thanks for that and all your guides - they've been helpful.

This actually has nothing much to do with the router. The router doesn't enter into the name resolution in my case. I took a look with Wireshark to make sure (see attached), where I cleared the Windows netbios cache with "nbtstat -R" then ran a ping back to FreeNAS. Only thing showing up was my Win 7 box first using Link-Local Multicast Name Resolution looking for "marvin", not getting a response, then switching to a NetBIOS Name Server broadcast, which was answered by 192.168.1.10 (the primary FreeNAS IP address) replying that Marvin was at 192.168.1.200 (the jail's IP address). That was followed by Windows issuing an ARP request for the MAC to match 192.168.120, then the actual ping.

Just to make sure the Wireshark trace running on Win 7 wasn't missing some router communications, I disconnected the router and did it all again. The trace looked the same, except there were a lot of extra ARP requests from various network members wondering what had happened to 192.168.1.1.

Anyway, I'll just turn on VIMAGE for pretty much any jail I create. I was trying without VIMAGE both because I was following the Minecraft guide, and I'd seen warnings about VIMAGE being experimental. Seems like that's out of date, though.

 

[anodos]

For sshd you can configure the 'listen address' variable in your sshd_config on both FreeNAS and jail. (Probably through modifying auxiliary parameters in your ssh config)
Likewise you will have to set the 'interfaces' parameter in your smb4.conf through the auxiliary parameters field. (If you choose to go without vimage).

 

[philiplu]

Thanks, anodos, but that's not actually the problem. If you SSH to the IP address, you'll end up in the correct place, FreeNAS or jail. The problem is that trying to SSH to the FreeNAS netbios name can dump you at either IP address, and it's not clear which happens when. It looks like having a single netbios name map to multiple IP addresses, as happens with a non-VIMAGE jail, doesn't have any good way to force the netbios name resolution to choose the FreeNAS IP instead of the jail's.

As cyberjock said, the answer is just to avoid the whole situation by only using VIMAGE jails.

I've been tracing through the nmbd source code and looking at debug dumps from nmbd and Wireshark traces, mostly out of intellectual curiosity, and I haven't found any reliable way yet. /etc/lmhosts doesn't work, nor does setting FreeNAS as a WINS server. Interestingly, the WINS server attempt seems to push the problem back over to Windows - with FreeNAS as a WINS server, name resolution queries from my Win 7 box get a response that lists both IP addresses, the FreeNAS and the jail. But Windows isn't consistent on which one it chooses - repeatedly issuing the command 'nbtstat -R & ping marvin' hits both addresses with no discernible pattern and roughly equal frequency.

Anyway, I've taken this about as far as I can. Time to enable VIMAGE.

 

[anodos]

Did you try setting the interfaces parameter as I mentioned above? For reference see here http://www.samba.org/~tpot/articles/multiple-interfaces.html

Note that name resolution happens in the following order: Hosts file, DNS, NetBIOS. If you configure a hosts file on your workstation it *should* take care of the problem. I emphasized 'should' because we're still dealing with samba/NetBIOS. Unexpected behavior should be expected. :)

 

[philiplu]

Sorry, I did not try the interfaces param. I have now, and it does indeed work to keep nmbd from giving out the jail IP as the address for the FreeNAS netbios name. Thanks for making me take a second look.