Problem copying large file from OSX to CIFS share

[nogi]

I am on FreeNAS-8.3.0-RELEASE-x64 (r12701M) on a HP N36L with 4 x Hitachi 7k3000 configured as RAIDZ.

I seem to have problems everytime I copy a large file to FreeNAS from OSX. It bombs out with a can't read from source error. Doing the same copy from OSX to Ubuntu and then to FreeNAS works fine. This kind of leads me to maybe the way my CIFS share is setup but I have large file support enabled as well as maintain dos support disabled.

What else could I try? I am trying to backup a 80Gb VM file.

 

[William Grzybowski]

Does the bail out at ~4GB?

try changing the smb protocol using auxiliary parameters from SMB2 to something else.

 

[joeschmuck]

Isn't the default protocol NT1? Or does FreeNAS change the default to SMB2? I thought SMB2 was still considered experimental.

I'm sure I'm wrong since you suggested it but I haven't seen anything contrary. Of course I'm talking about FreeNAS 8.3.0-Release with Samba 3.6.7 (I believe).

 

[William Grzybowski]

I dunno, I run ALPHA release it is SMB2 there, I don't recall whether and if so when it was merged to 8.3.

 

[joeschmuck]

I just looked at the /usr/local/etc/smb.conf and the 'max protocol' parameter is not listed. I assume that is the correct spot to look? I will add 'max protocol = SMB2' to see if it works.

Is there any way to check (command line thing) to see which protocol is being used?

 

[William Grzybowski]

If it is not listed the it defaults for NT4, you were right.

I am not aware of any way to detect it (cli wise).

 

[joeschmuck]

Thanks.

To nogi: I would recommend you change the protocol to see if this works better for you.

Code:

max protocol = SMB2

Please post if this helps or not. BTW, I was having problems writing large backup files but things use to work fine. This thread caught my eye but I originally thought it could be one of my hard drives causing the issue. I hope this fixes it but who knows, in the meantime I'm ordering a new WD Red 2TB drive to replace my only Samsung (non-NAS) drive. I should have a spare anyway.

 

[joeschmuck]

My results with SMB2, crash! Didn't get much transferred before the FreeNAS died, no network connectivity. I'm removing that option as soon as I reboots.

 

[cyberjock]

The default for Samba is SMB2. What you want is to enable SMB1 by using "max protocol = NT1". Some of the older documentation still says SMB2 is experimental and not the default, but with the version FreeNAS 8.3.0-p1 it is unless you override it.

But the thing to keep in mind is that SMB2 is actually a simpler protocol and changing to NT1 isn't likely to fix your problem. :P

It would be real nice if you had posted your server hardware and FreeNAS version...

 

[joeschmuck]

So I went to the samba.org site and it states that samba version 3.x default is NT1. I'm not trying to pick a fight but do you have a reference that says SMB2 is the default for Samba 3.6.7 or do you know of a way to tell which protocol is running? You know, one place I haven't checked was the ports collection to see if there were any notes about the port itself.

Thanks

 

[cyberjock]

Well, for starters, if you were using SMB1 you would be thrilled to get 50MB/sec. SMB1 is extremely chatty and extremely small amounts of latency on a local network tank the performance of SMB1. I know on my Windows server I replaced with FreeNAS in December I couldn't even get 35MB/sec across my LAN if I forced SMB1. SMB2 added a small handful of new opcode but removed like 5x as many, so it is considered to be simpler. This is one of the ways that the chattiness was brought down. Attempting to saturate Gb was virtually impossible with SMB1. Of course, I am talking about Windows SMB1 and not the SMB1 that is Samba, but I think(or at least thought) that the issue was protocol based and not the implementation.

I even have a thread where I questioned SMB2 and if it was the default because of the speeds I was getting with what I thought was SMB1.

Now you're no doubt thinking someone should test this.. so I will be.. :/ Damn the stuff I do for the forum. LOL.

Here's the commands for SMB1:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP_ANDX, TRANS2_SESSION_SETUP, LOGOFF_ANDX, PROCESS_EXIT, TREE_CONNECT, TREE_CONNECT_ANDX, TREE_DISCONNECT)

File, directory and volume access (CHECK_DIRECTORY, CLOSE, CLOSE_PRINT_FILE, COPY, CREATE, CREATE_DIRECTORY, CREATE_NEW, CREATE_TEMPORARY, DELETE, DELETE_DIRECTORY, FIND_CLOSE, FIND_CLOSE2, FIND_UNIQUE, FLUSH, GET_PRINT_QUEUE, IOCTL, IOCTL_SECONDARY, LOCK_AND_READ, LOCK_BYTE_RANGE, LOCKING_ANDX, MOVE, NT_CANCEL, NT_CREATE_ANDX, NT_RENAME, NT_TRANSACT, NT_TRANSACT_CREATE, NT_TRANSACT_IOCTL, NT_TRANSACT_NOTIFY_CHANGE, NT_TRANSACT_QUERY_QUOTA, NT_TRANSACT_QUERY_SECURITY_DESC, NT_TRANSACT_RENAME, NT_TRANSACT_SECONDARY, NT_TRANSACT_SET_QUOTA, NT_TRANSACT_SET_SECURITY_DESC, OPEN, OPEN_ANDX, OPEN_PRINT_FILE, QUERY_INFORMATION, QUERY_INFORMATION_DISK, QUERY_INFORMATION2, READ, READ_ANDX, READ_BULK, READ_MPX, READ_RAW, RENAME, SEARCH, SEEK, SET_INFORMATION, SET_INFORMATION2, TRANS2_CREATE_DIRECTORY, TRANS2_FIND_FIRST2, TRANS2_FIND_NEXT2, TRANS2_FIND_NOTIFY_FIRST, TRANS2_FIND_NOTIFY_NEXT, TRANS2_FSCTL , TRANS2_GET_DFS_REFERRAL, TRANS2_IOCTL2, TRANS2_OPEN2, TRANS2_QUERY_FILE_INFORMATION, TRANS2_QUERY_FS_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_REPORT_DFS_INCONSISTENCY, TRANS2_SET_FILE_INFORMATION, TRANS2_SET_FS_INFORMATION, TRANS2_SET_PATH_INFORMATION, TRANSACTION, TRANSACTION_SECONDARY, TRANSACTION2, TRANSACTION2_SECONDARY, UNLOCK_BYTE_RANGE, WRITE, WRITE_AND_CLOSE, WRITE_AND_UNLOCK, WRITE_ANDX, WRITE_BULK, WRITE_BULK_DATA, WRITE_COMPLETE, WRITE_MPX, WRITE_MPX_SECONDARY, WRITE_PRINT_FILE, WRITE_RAW)

Other (ECHO, TRANS_CALL_NMPIPE, TRANS_MAILSLOT_WRITE, TRANS_PEEK_NMPIPE, TRANS_QUERY_NMPIPE_INFO, TRANS_QUERY_NMPIPE_STATE, TRANS_RAW_READ_NMPIPE, TRANS_RAW_WRITE_NMPIPE, TRANS_READ_NMPIPE, TRANS_SET_NMPIPE_STATE, TRANS_TRANSACT_NMPIPE, TRANS_WAIT_NMPIPE, TRANS_WRITE_NMPIPE)

Here's the opcode for SMB2:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP, LOGOFF, TREE_CONNECT, TREE_DISCONNECT)
File, directory and volume access (CANCEL, CHANGE_NOTIFY, CLOSE, CREATE, FLUSH, IOCTL, LOCK, QUERY_DIRECTORY, QUERY_INFO, READ, SET_INFO, WRITE)
Other (ECHO, OPLOCK_BREAK)

Notice the significantly smaller list :P

Here's a small presentation I found that shows a real world test of tranferring 10.7GB over 76ms of lag for a WAN connection. With SMB1 it took about 6 hours with a 0.25MB/sec speed. With SMB2 it took about 8 minutes with a speed of about 25MB/sec. :P

Anyway, I'll be back in a bit. Gonna disable SMB1 and try to copy stuff from my server...

 

[William Grzybowski]

When max protocol option is not used auto-negotiation takes care of choosing the appropriate protocol. At least thats what smb.conf manual says.

 

[cyberjock]

Ok, just finished my test. You can do the same in Vista/7/Server 2008+ with the following commands. You must be at a command line that has Administrator priviledges. You must reboot for the changes to take effect.

Force SMB1:

sc config lanmanworkstation depend= bowser/mrxsmb10/nsi
sc config mrxsmb20 start= disabled

Enable SMB2: (there is no force for SMB2)
sc config lanmanworkstation depend= bowser/mrxsmb10/mrxsmb20/nsi
sc config mrxsmb20 start= auto

Note: the space between the = and the parameter are on purpose.

My speeds copying a 6GB ISO.

SMB1 forced: 35.1MB/sec
SMB2 enabled: 77.1MB/sec

Note that I do not have the max protocol = SMB2 on my FreeNAS server because I had found someone that said it was enabled by default.

I then tried adding max protocol = NT1 and restarted the CIFS service on my server.

SMB1 on server: 37.1MB/sec

Personally, and I'll agree this is not hard scientific evidence(but it is pretty damning) but it seems like this is proof that SMB2 must be working on Samba. It may be that its not the default but Windows 7 forces it. I don't know. I feel pretty confident when I say that with Windows 7 clients SMB2 is absolutely working with no tweaks.

The whole performance improvements between SMB1 and SMB2 has been near and dear to my heart because in 2008 I was upgrading from a machine with Server 2003 to a machine with Server 2008 and I had to copy the entire 6TB of data to the new server. Since Server 2003 didn't support SMB2 I actually installed Vista just to copy the data because it saved us 2 days of copying time despite being a Gb connection between the 2 machines. At this time I refused to upgrade any of my machines to Vista(I hate Vista and refused to use it) and I was stuck with a very lowly 35MB/sec to/from my machines at home because I was still using XP. Of course, the 35MB/sec was a per connection, so if I had 2 local hard drives I could copy files to each one and have a combined network usage of about 70MB/sec. I ended up upgrading to 7 out of desperation to have SMB2 support and >4GB of RAM on my machines and I wasn't about to try to go to Linux at that point.

Edit: So I found out that even if after I restarted CIFS on the FreeNAS server and deleted the max protocol = NT1 line Windows 7 won't renegotiate SMB2. It appears that once you negotiate down to SMB1 there is no way to get back up to SMB2 without reboot.

Also, for fun I tried adding max protocol = smb2 and I got 77.5MB/sec. So it really appears that smb2 is being used between FreeNAS server and Windows 7 clients.

 

[joeschmuck]

Thanks for all that hard work, I do appreciate it. As you know I'm no stranger to testing and even though I had a crash when I changed the protocol earlier to NT1, later I tried it again but this time I turned off CIFS before making the change and then enabled it, yes, very slow in the 35-40MB/sec and changing to SMB2 returned me to to 80-90MB/sec range. I'm still having read issues and I still suspect my one non-NAS drive being the culprit. I still need to buy a WD Red 2TB as indicated above.

Again, thanks for the info. I'm not sure if the OP has done anything.

 

[nogi]

Thanks all for the replies and testing that's been done. I'm sorry that I wasn't able to respond yesterday but I can post again so I'll check out my box later this week and try some of the suggestions. :)

It would be real nice if you had posted your server hardware and FreeNAS version...

In my haste, I missed it. I've included it above. It would have been nice to get a warning first though before a ban, thought it was a tad harsh

Also, I notice that we've started to talk about performance. Just a friendly reminder that my issue isn't performance it's about being able to copy large file from OSX to FreeNAS via a CIFS share.

 

[cyberjock]

In my haste, I missed it. I've included it above. It would have been nice to get a warning first though before a ban, thought it was a tad harsh

We've had to crack down on it because it seems like every 5th thread isn't following the forum rules and its getting out of control to the extreme.

Also, I notice that we've started to talk about performance. Just a friendly reminder that my issue isn't performance it's about being able to copy large file from OSX to FreeNAS via a CIFS share.

Correct, I never considered the protocol to be the issue. But I felt it was necessary to clear the air on the whole protocol thing. :)

As for your issue, I'd try copying from another machine to FreeNAS and see if the problem persists. If it doesn't then it may be something to do with OSX.

 

[JaimieV]

If it doesn't then it may be something to do with OSX.

Though perhaps not a general OSX problem - having just this morning copied a 41gig archive file from OSX to a FreeNAS 8.3.0 without issues. Not as large as yours, Nogi, I'll build a larger one and try it.

Most VMs (or rather the virtual disk images) are split into segments; is your VM really a single 80gig file? Just wondering if there's something else inside the package that might be the issue, rather than the size itself. Right-click it and see if you get a "show package contents" menu entry.

/Edit - Bother, just did the copy over AFP without thinking. 96gig in half an hour, anyway. I'll do it over CIFS next...
/Edit edit: "About 4 hours"?!? Wow. That's a big difference.

 

[JaimieV]

A very big difference. Under SMB, "The Finder can’t complete the operation because some data in “steam.tar” can’t be read or written. (Error code -36)" within a few minutes/few gig. Is that the same error you see, Nogi?

 

[JaimieV]

cp fails too,
cp: .tar: Operation not supported

I get exactly 4294967296 bytes written (by cp or Finder) to the NAS before the error. Why yes, I do believe that size is rather significant! 2^32 on the nose.

I'll see if I can work out if it's an Apple SMB cockup, or elsewhere. Unfortunately there are a lot of variables here...

 

[joeschmuck]

That is a huge difference. I think I'd want to use AFP if it's available. Not only does it speed up the copy but it may allow nogi to finish without an error.