SOLVED CIFS/Samba looping generate_smb4_conf.py?

[vilane]

Currently running FreeNAS-9.3-STABLE-201602031011, had updated a couple days ago.

I was forced to also update my Sonicwall TZ100 yesterday as I had discovered they deprecated their self-signed SSL certs with old protocols to the point that Chrome would not let me connect to administrate it. But i could bypass that with IE.

So the only changes that have happened in the past 2 weeks to the environment has been the FreeNAS update to the latest stable, and the firmware upgrade on the firewall.

Now, the issue I am seeing is that when connecting to my home network via VPN (SonicWall NetXtender), I get an active connection everything looks fine just like it was prior to updates. I can browse through all of the folder locations and files and everything is snappy and responsive. The moment I attempt to copy and paste a file using CIFS, the system starts calculating transfer times and everything and will then prompt me with a message that it can't connect and to try connecting to the network again. I am prompted with the Try Again/Skip/Cancel options that Windows Explorer loves to show off, but the data will not transfer.

My first reaction is that the VPN/Firewall is the problem, so I am looking there now, but I also am seeing this looping in my logs on the FreeNAS, which prompted my post:

Code:

09:30:54 nmbd[44841]:   STATUS=daemon 'nmbd' finished starting up and ready to serve connections*****
09:30:54 nmbd[44841]:  
09:30:54 nmbd[44841]:   Samba name server DENMOTHER is now a local master browser for workgroup WORKGROUP on subnet 192.168.168.170
09:30:54 nmbd[44841]:  
09:30:54 nmbd[44841]:   *****
09:34:42 generate_smb4_conf.py: [common.pipesubr:71] Popen()ing: /usr/local/bin/net -d 0 getlocalsid
09:34:42 generate_smb4_conf.py: [common.pipesubr:71] Popen()ing: /sbin/sysctl -n 'kern.maxfilesperproc'
09:34:42 generate_smb4_conf.py: [common.pipesubr:71] Popen()ing: mount
09:34:42 generate_smb4_conf.py: [common.pipesubr:71] Popen()ing: /usr/local/bin/net -d 0 getlocalsid
09:34:42 generate_smb4_conf.py: [common.pipesubr:71] Popen()ing: /usr/bin/getent passwd 'media'
09:34:42 notifier: Stopping winbindd.
09:34:42 winbindd[44849]:   STATUS=daemon 'winbindd' finished starting up and ready to serve connectionsGot sig[15] terminate (is_parent=1)
09:34:42 winbindd[44850]:   STATUS=daemon 'winbindd' finished starting up and ready to serve connectionsGot sig[15] terminate (is_parent=0)
09:34:42 notifier: Waiting for PIDS: 44849.
09:34:42 notifier: Stopping smbd.
09:34:43 notifier: Waiting for PIDS: 44845, 44845.
09:34:43 notifier: Stopping nmbd.
09:34:43 nmbd[44841]: [2016/02/09 09:34:43.517660,  0] ../source3/nmbd/nmbd.c:57(terminate)
09:34:43 nmbd[44841]:   Got SIGTERM: going down...
09:34:43 notifier: Waiting for PIDS: 44841.
09:34:43 notifier: Performing sanity check on Samba configuration: OK
09:34:43 notifier: winbindd not running? (check /var/run/samba/winbindd.pid).
09:34:43 notifier: smbd not running? (check /var/run/samba/smbd.pid).
09:34:43 notifier: nmbd not running? (check /var/run/samba/nmbd.pid).
09:34:43 notifier: Performing sanity check on Samba configuration: OK
09:34:43 notifier: Starting nmbd.
09:34:43 notifier: Starting smbd.
09:34:43 nmbd[45656]: [2016/02/09 09:34:43.676443,  0] ../lib/util/become_daemon.c:136(daemon_ready)
09:34:43 notifier: Starting winbindd.
09:34:43 smbd[45660]: [2016/02/09 09:34:43.709755,  0] ../lib/util/become_daemon.c:136(daemon_ready)
09:34:43 winbindd[45664]: [2016/02/09 09:34:43.727487,  0] ../source3/winbindd/winbindd_cache.c:3196(initialize_winbindd_cache)
09:34:43 winbindd[45664]:   initialize_winbindd_cache: clearing cache and re-creating with version number 2
09:34:43 winbindd[45664]: [2016/02/09 09:34:43.729660,  0] ../lib/util/become_daemon.c:136(daemon_ready)
09:34:43 notifier: Stopping mdnsd.
09:34:43 smbd[45660]: dnssd_clientstub DNSServiceProcessResult called with DNSServiceRef with no ProcessReply function
09:34:43 notifier: Waiting for PIDS: 44893.
09:34:43 notifier: Starting mdnsd.
09:35:06 nmbd[45656]:   STATUS=daemon 'nmbd' finished starting up and ready to serve connections*****
09:35:06 nmbd[45656]:  
09:35:06 nmbd[45656]:   Samba name server DENMOTHER is now a local master browser for workgroup WORKGROUP on subnet 192.168.168.170
09:35:06 nmbd[45656]:  
09:35:06 nmbd[45656]:   *****

It looks to me as if the Samba services keep crashing on themselves and restarting? does that seem accurate from reading this? I am unsure if I need to make any configuration changes to CIFS or any of the underlying samba config files, but this seems a bit suspect to me after recent patches on the FreeNAS stables.

Let me know what you guys think. I haven't been able to test a local transfer without the VPN because I've been away the past 5 days, but I will try tonight when I am actually home to see if the local transfer is also affected without the VPN so I can rule the firewall out of the equation.

Any thoughts and assistance is much appreciated. Thanks!

 

[anodos]

Post output of "testparm". Review logs at /var/log/samba4/log.smbd

 

[vilane]

Information requested is below, however I am feeling like this is a firewall/VPN tunnel issue now. I tested the local transfer on my home network without the VPN involved and the same test files transferred successfully.

Testparm output

Code:

[root@denmother] ~# testparm
Load smb config files from /usr/local/etc/smb4.conf
Processing section "[media]"
Loaded services file OK.
Server role: ROLE_STANDALONE
Press enter to see a dump of your service definitions

[global]
        dos charset = CP437
        server string = FreeNAS Server
        interfaces = 127.0.0.1, 192.168.168.170
        bind interfaces only = Yes
        server role = standalone server
        map to guest = Bad User
        null passwords = Yes
        obey pam restrictions = Yes
        max log size = 51200
        time server = Yes
        deadtime = 15
        max open files = 469325
        load printers = No
        printcap name = /dev/null
        disable spoolss = Yes
        lm announce = Yes
        dns proxy = No
        pid directory = /var/run/samba
        panic action = /usr/local/libexec/samba/samba-backtrace
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        acl allow execute always = Yes
        create mask = 0666
        directory mask = 0777
        directory name cache size = 0
        kernel change notify = No
        map archive = No
        map readonly = no
        strict locking = No
        dos filemode = Yes

[media]
        path = /mnt/denpool/media
        read only = No
        guest ok = Yes
        veto files = /.snapshot/.windows/.mac/.zfs/
        vfs objects = zfs_space, zfsacl, aio_pthread, streams_xattr
        zfsacl:acesort = dontcare
        nfs4:chown = true
        nfs4:acedup = merge
        nfs4:mode = special

I dug through the logs and other than seeing Oplock timeouts, nothing really seems out of place from what I can see. Here is a snip from that showing connection through VPN/browsing the media share in Windows Explorer, attempting a copy paste, and getting the timeout/network connection error message in Windows, and then closing out.

Code:

[2016/02/09 11:14:33,  0] ../source3/smbd/server.c:1189(main)
  smbd version 4.1.21 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2013
[2016/02/09 11:14:33.279674,  1] ../source3/smbd/files.c:218(file_init_global)
  file_init_global: Information only: requested 469325 open files, 59392 are available.
[2016/02/09 11:14:33.282531,  0] ../lib/util/become_daemon.c:136(daemon_ready)
  STATUS=daemon 'smbd' finished starting up and ready to serve connectionsOplock break failed for file movies/I Am Chris Farley (2015)/folder.jpg -- reply$
  STATUS=daemon 'smbd' finished starting up and ready to serve connectionsl-jrr-acl (ipv4:192.168.168.180:52036) closed connection to service media
[2016/02/09 11:19:07.667675,  1] ../source3/smbd/service.c:1130(close_cnum)
  l-jrr-acl (ipv4:192.168.168.180:51948) closed connection to service media

 

[anodos]

Information requested is below, however I am feeling like this is a firewall/VPN tunnel issue now. I tested the local transfer on my home network without the VPN involved and the same test files transferred successfully.

Testparm output

Code:

[root@denmother] ~# testparm
Load smb config files from /usr/local/etc/smb4.conf
Processing section "[media]"
Loaded services file OK.
Server role: ROLE_STANDALONE
Press enter to see a dump of your service definitions

[global]
        dos charset = CP437
        server string = FreeNAS Server
        interfaces = 127.0.0.1, 192.168.168.170
        bind interfaces only = Yes
        server role = standalone server
        map to guest = Bad User
        null passwords = Yes
        obey pam restrictions = Yes
        max log size = 51200
        time server = Yes
        deadtime = 15
        max open files = 469325
        load printers = No
        printcap name = /dev/null
        disable spoolss = Yes
        lm announce = Yes
        dns proxy = No
        pid directory = /var/run/samba
        panic action = /usr/local/libexec/samba/samba-backtrace
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        acl allow execute always = Yes
        create mask = 0666
        directory mask = 0777
        directory name cache size = 0
        kernel change notify = No
        map archive = No
        map readonly = no
        strict locking = No
        dos filemode = Yes

[media]
        path = /mnt/denpool/media
        read only = No
        guest ok = Yes
        veto files = /.snapshot/.windows/.mac/.zfs/
        vfs objects = zfs_space, zfsacl, aio_pthread, streams_xattr
        zfsacl:acesort = dontcare
        nfs4:chown = true
        nfs4:acedup = merge
        nfs4:mode = special

I dug through the logs and other than seeing Oplock timeouts, nothing really seems out of place from what I can see. Here is a snip from that showing connection through VPN/browsing the media share in Windows Explorer, attempting a copy paste, and getting the timeout/network connection error message in Windows, and then closing out.

Code:

[2016/02/09 11:14:33,  0] ../source3/smbd/server.c:1189(main)
  smbd version 4.1.21 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2013
[2016/02/09 11:14:33.279674,  1] ../source3/smbd/files.c:218(file_init_global)
  file_init_global: Information only: requested 469325 open files, 59392 are available.
[2016/02/09 11:14:33.282531,  0] ../lib/util/become_daemon.c:136(daemon_ready)
  STATUS=daemon 'smbd' finished starting up and ready to serve connectionsOplock break failed for file movies/I Am Chris Farley (2015)/folder.jpg -- reply$
  STATUS=daemon 'smbd' finished starting up and ready to serve connectionsl-jrr-acl (ipv4:192.168.168.180:52036) closed connection to service media
[2016/02/09 11:19:07.667675,  1] ../source3/smbd/service.c:1130(close_cnum)
  l-jrr-acl (ipv4:192.168.168.180:51948) closed connection to service media

Sounds reasonable. You can increase logging verbosity in samba to see if it will give more information about what's going on. If needed, you can selectively turn off oplocks on a per-share basis. This will affect performance.
https://www.samba.org/samba/docs/man/manpages-3/smb.conf.5.html#OPLOCKS

Of course, the best course of action is to fix your VPN. It looks like your firewall will go EOL in 1 year. You can always replace with a pfsense appliance. :D

 

[vilane]

Indeed, thanks anodos. I will be looking at a new appliance soon enough. I got this one for free through my work, so I can't really scoff at free.

This is now resolved though, turns out the latest stable firmware from Dell/Sonicwall has some crippling VPN (SSL and IPSec) bugs. gasp! I rolled back the firmware to a confirmed stable version (newer than what I had) and transfers are working just fine again.

Thanks again for jumping in and giving some options anodos