CIFS and Network Dropping

[hunden5]

Hello!

I am trying to LOVE FreeNAS however I have the craziest problem. I am running the latest version from your homepage and have it installed on a PowerEdge 1100C with 72GB RAM and two quad core processors. I am trying to copy 12TB of data from another NAS to my FreeNAS. The PowerEdge is connected to Norco enclosure with 8 3TB disks. At first glance everything was ok, started copying files and was getting about 50MB/s transfer rate. BUT randomly the CIFS share becomes unavailable and the data copy stops. When this happens I am also unable to access the freenas gui. I don't have AIO enabled, I've adjusted settings on my Windows 7 machine. The most I've been able to sustain without crapping out is about 900GB. Now I can only get maybe 18GB before it craps out. Any thoughts?

 

[cyberjock]

Any messages locally on the server?

 

[hunden5]

None. And I have CIFS in Debug logging mode. It just died this very second after about 30GB.

 

[hunden5]

OK. I just got this in the log..

Aug 29 18:13:24 freenas last message repeated 2 times
Aug 29 18:15:52 freenas smbd[6829]: [2013/08/29 18:15:52.592489, 0] smbd/process.c:244(read_packet_remainder)
Aug 29 18:15:52 freenas smbd[6829]: read_fd_with_timeout failed for client 0.0.0.0 read error = NT_STATUS_IO_TIMEOUT.
Aug 29 18:19:23 freenas manage.py: [common.pipesubr:57] Popen()ing: /usr/local/bin/warden list -v

 

[cyberjock]

And I'm guessing nothing tantalizing in the logs?

 

[hunden5]

I can only see the log screen I've setup in the Web GUI since I'm not physically in front of the server....

 

[pirateghost]

I can only see the log screen I've setup in the Web GUI since I'm not physically in front of the server....

Enable SSH

Sent from my Galaxy Nexus

 

[hunden5]

done.

 

[cyberjock]

Hmm.. one thread via google said it was a workstation issue and not a server issue... But I don't know.. I'll have to read more on that error.

 

[hunden5]

I've already done this....

I disabled SMB signing on my win 7 and 8 clients. By setting the values of the registry keys: EnableSecuritySignature and RequireSecuritySignature to 0 (zero) in the the following registry entry: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters

Now I'm trying to do this:

After weeks I finally found it. There is a line in smb.conf that sets no delay on the connection. It is by default commented out. I merely uncomment the line and voila! This is the line:
socket options = TCP_NODELAY
it is by default commented out. If you are having an issue with this, uncomment that line in your smb.conf and restart samba.

But I have no idea how to change smb.conf file.

 

[cyberjock]

There's a section of the GUI for adding stuff like that... :)

 

[hunden5]

Found it and have added it, now restarting CIFS service. If this doesn't work I'm not sure what else to do!!!

 

[hunden5]

No dice. Died at 16GB

 

[cyberjock]

What happens if you copy data from a different machine? Does it still fail out?

 

[hunden5]

I'm trying that today..

 

[JoeV]

im interested in this too. I seem to be having the same/similar issue. I can run a few backups just fine and then it just dies. I checked the aio settings and my smb.conf has nothing about aio in it at all. I also found this link that might be related to this issue. I am willing to run some tests as well. At this point I have to power cycle my backup system after every few backups, as it crashes hard and becomes unresponsive remotely and also unresponsive at the console. I was running 8.3 and recently upgraded to 9.1.1. After the upgrade I started having these crashes. I then thought that something in my upgrade might have caused the problem so I did a fresh install of 9.1.1 and imported the zpool. The problem still is happening, sorry I didn't mean to jack your thread. I hope we can get you working and then maybe your fix will be mine too. Im getting tempted to downgrade.

https://bugzilla.samba.org/show_bug.cgi?id=8726

 

[JoeV]

So I did some more testing today and it looks like I have a little more info to share. I did a lot of file transfers to my main backup system from my mac laptop with smb share mounted up with no errors. I was able to transfer many large 40Gb + size files with out incident. I then mounted up the cifs share on another freenas box so I have freenas 9.1.1 release on both boxes and a cross over between them. I mount up the cifs share and so far have transferred over a Tb of lots of large files. I am thinking this has to be an issue with windows in the mix. Im going to keep my large file transfer going just in case it does break, going to move over 13Tb of data total between the two freenas boxes. I am going to let it run all day and all night and if it doesn't crash I will try to pull and push some files with my windows server to see if it causes issue, windows server is 2008 R2 64bit. I hope some of my testing is helping.

 

[RjBradlow]

Hello!

I am trying to LOVE FreeNAS however I have the craziest problem. I am running the latest version from your homepage and have it installed on a PowerEdge 1100C with 72GB RAM and two quad core processors. I am trying to copy 12TB of data from another NAS to my FreeNAS. The PowerEdge is connected to Norco enclosure with 8 3TB disks. At first glance everything was ok, started copying files and was getting about 50MB/s transfer rate. BUT randomly the CIFS share becomes unavailable and the data copy stops. When this happens I am also unable to access the freenas gui. I don't have AIO enabled, I've adjusted settings on my Windows 7 machine. The most I've been able to sustain without crapping out is about 900GB. Now I can only get maybe 18GB before it craps out. Any thoughts?

I'm glad you found and solved the problem but you may find better performance using NFS.

Windows systems can connect to NFS shares using Services for NFS. Connecting to NFS shares is often faster than connecting to CIFS shares due to the single-threaded limitation of Samba.
~TRUE.~ RjB

NOTE: Services for NFS is only available in the Ultimate or Enterprise editions of Windows. ~FALSE ! Windows XP had NFS support and Windows 7 comes with NFS support but deactivated. ~ RjB

Source: http://doc.freenas.org/index.php/Unix_(NFS)_Shares#From_Microsoft_Clients

Mount Linux NFS Share on Windows 7​

http://www.hackourlife.com/mount-linux-nfs-share-on-windows-7/

Additional info:
How to install Client for NFS on Windows for a UNIX-to-Windows migration
http://support.microsoft.com/kb/324055
NFS client for Windows 7 (Microsoft Community)
http://answers.microsoft.com/en-us/...indows-7/42aae25d-d077-4ff9-abdf-7314a589c46d
Utilities and SDK for Subsystem for UNIX-based Applications in Microsoft Windows 7 and Windows Server 2008 R2
http://www.microsoft.com/en-us/download/details.aspx?id=2391
Google: ' Client for NFS ' ... for more info.

 

[JoeV]

Thanks for all the info but I don't consider this a fix I consider it a work around. I ended up messing with a lot of windows 2008 r2 settings and finally just gave up on that and back revved to 8.3. With 8.3 back on the freenas system my stability came back and so far none of my cifs have given me any problems. The particular backup app that I use doesn't support an NFS target so I needed cifs to work for it to continue. At this point I am looking at other options instead of freenas. I got zfsonlinux working in the lab today and will be moving back to linux going forward.

 

[cyberjock]

I've played with ZFS on linux a bit. Just be warned it is, in my opinion, NOT production machine ready. It does work, but has some really weird performance and reliability issues.