SOLVED Windows 10 cloud accounts access to CIFS Windows Shares? AND: extreme 1 file/s max slow file copying

[devnullius]

Ok, importing old disks (volumes) now does work. Still, it is strange that the GUI had so much problems before I had the IT firmware. Being able to recognize the volumes and folders on it, but not showing it still does sound like a bug to me. But that said, it does work as advertised now I'm on IT firmware (see above).

Still, there is a problem with Chrome and the GUI. For example, when I run the initial wizard, after clicking next (without entering information) at the Directory Services integration part, I just end up with an empty box. Previous & Next won't work. Other browsers now seem to work fine though, though some have problems when Ad-Muncher is enabled. Another thing that makes no sense, to me ;p

Last step: testing file sharing... :)

 

[devnullius]

:( All I get is access denied. First with a fresh 9.3.1 install and 1 folder 'R' on the volume, shared once assigned to user 'devvi_000' and later a second share 'R2' assigned to root.

None of them give access to my Windows 10 client. I tried user '\devvi_000', 'devvi_000', 'root', '\root': all access denied. On the console I see a lot of messages with freenas smbd[pid]: matchname failed on IP.
Enabling guest access allows entering (read) rights, but no creating rights (no credentials are asked either when Guest user is enabled).

I don't know where to look next. Using Internet Explorer. I guess I'll leave both machines running overnight?

PS: 2 folders are shared. I'll remove 1 once more to prevent any bugs as mentioned before (https://forums.freenas.org/index.php?threads/cannot-connect-to-1-out-of-2-cifs-shares.40121/)
PS: devvi_000 user & group details:
User: 1001 devvi_000 1001 /nonexistent /bin/csh devvi_000
Disable password login false, Lock user false, Permit Sudo false, Microsoft Account false
Group: 1001 devvi_000
Built-in group false, Permit sudo false

I'll play a bit more with these variables...



Sigh.

Devvie

 

[gpsguy]

1% left on my phone. Will reply back in ~30 minutes.


Sent from my iPhone using Tapatalk

 

[devnullius]

1% left on my phone. Will reply back in ~30 minutes.
Sent from my iPhone using Tapatalk

That's the spirit! :)

xx

Hostname BigCube.local
Build FreeNAS-9.3-STABLE-201602031011
Uptime 9:35PM up 1:24, 0 users
Load Average 0.09, 0.08, 0.02

 

[gpsguy]

I'm glad to hear you were able to import the volume. I misread your reply, when reading it on my phone (it did run out of gas).

I know it might have taken some effort to flash your controller, but doing some fixed that issue. Hopefully you took notes, since FreeNAS occassionally updates the driver in the OS and when that happens, you'll need to reflash your HBA. I've seen it updated 3-4 times in the past 4 years.

Ok, importing old disks (volumes) now does work. ... But that said, it does work as advertised now I'm on IT firmware.

While Chrome might be your favorite browser, I'd just put it aside for now, and use Firefox (or whatever works best) to manage FreeNAS. I'm an IT professional by trade and have to jump back and forth between Firefox (default) and IE all day long. With the Stable Channel updates for Chrome being released every 2 weeks or so, incompatibilities with other software is certain.

Another thing to remember is that the developers primary focus is on FreeNAS 10.x right now. They are still making some changes to 9.3 and backporting some of the newer drivers, but 10.x is still their primary focus. As an example Microsoft Vista's end of extended support is April 11, 2017. Compared with Windows 10, Vista will only receive minimal support for the next year or so.

Still, there is a problem with Chrome and the GUI.

 

[devnullius]

Yeah, Google's war against the internet has introduced many problems with internet & browser compat. Switching back and forth to many browsers here too :/

Good heads up for the firmware updating - as you noticed, I did make some notes ;-)

I don't understand your point with Vista? Probably not important either.

That all said, I still can't work with my shares. I even changed "R" to "r", just to be sure.

Thanks!!

Devvie

 

[gpsguy]

Correct - disregard it.

I'm working on a reply to your share problem.

I don't understand your point with Vista? Probably not important either.

 

[devnullius]

:) it did left me wondering how long FreeNAS 9.3 will be supported? I'd rather stay on that channel unless there will be no more security updates after release of stable 10.0?

I'll await your suggestions, curious what you'll come up with :)

Devvie

 

[gpsguy]

Devvie,

Please don't use the "\" for your usernames, etc.

@m0nkey_ has an excellent YouTube video showing how create Windows shares and permissions. See this thread -

[How-to] FreeNAS and Samba (CIFS) permissions (Video)

https://forums.freenas.org/index.php?threads/how-to-freenas-and-samba-cifs-permissions-video.41210/

First with a fresh 9.3.1 install and 1 folder 'R' on the volume, shared once assigned to user 'devvi_000' ...

 

[devnullius]

:( agonizing to watch, but I did, twice. I 'corrected' owner of data set to user nobody (kinda smart actually). I only used 1 group lan_users (renamed from group: devvi_000). I assigned user devvi_000 to that group. I assigned permissions to each as it should and... Nothing.

I'm gonna leave 1 PC & FreeNAS server on tonight, but I don't have high hopes. Time for bed now, thanks GPS Guy :)

Devvie

 

[gpsguy]

Devvie,

The dates for FreeNAS 10.x have slipped several times, that I can't remember how long 9.3 will be supported.

Given the significant change being made to FreeNAS 10.x, my guess, is that 9.3 will be supported for sometime afterwards. One of the reasons for the delay is that v10 will have a new webGUI and sometime late in the game, they decided to start over from scratch on that GUI.

:) it did left me wondering how long FreeNAS 9.3 will be supported? I'd rather stay on that channel unless there will be no more security updates after release of stable 10.0?

 

[devnullius]

:( I assigned permissions to each as it should and... Nothing.
Devvie

I went to user account devvi_000 once more, and (re)set the password for it. This time it all worked like a charm. Finally :) Sigh. Now time for the advanced NAS features ;p

Devvie

 

[devnullius]

Now I have it working, I'm left wondering... Creating and assigning user names should work transparant, no? Or is that simply not possible with Windows 10 'cloud' accounts? For in the old days, Windows would have tried my current credentials when accessing the share and be allowed access. Now, we have to enter the credentials because the automatic verification failed. Correct?

Good night :)

 

[gpsguy]

Hurray!

Now I have it working, ...

Yes.

Depending on the username, you might be able to save your FreeNAS credentials in Windows Credential Manager.

For in the old days, Windows would have tried my current credentials when accessing the share and be allowed access.

Sweet dreams!:)

 

[devnullius]

Yep, slept well, thank you once more. First massive (1.2TB; >350k files) file copy started yesterday. It was interrupted this morning by antivirus. Since then, I did not get the full speeds I had yesterday (about 80MB/s). I restarted the copy process first; restarted the windows 10 PC second; and the FreeNAS 3th.

It's now copying once more, with very slow speeds (20KB/s, fluctuating). I hope this is because some of the files are already there (I've chosen to overwrite the target files if they exist, after the nas reboot). Not sure though. I'll see tomorrow morning if all files are transferred.

Also I noticed huge memory usage (as predicted), about 20GB was being used. What surprised me though is that when IO activity by long was stopped, FreeNAS kept using 20GB of memory... Shouldn't FN release the RAM at some point? EDIT: read this: https://forums.freenas.org/index.php?threads/memory-usage-by-freenas.39331/ :)

 

[gpsguy]

I presume many of the 350k files are small, like JPG's, etc. if so, with the overhead associated with each file it's normal for it to be slower than copying big multi gigabyte files.

Search the forum with keywords like thousands of small files slow copy ... There are threads out there, I just can't search them from my phone.

If you are copying the files from Windows, a tool like Robocopy, Richcopy, Synctoy, etc. are useful if you need to restart the copy process. For other OS's Rsync is another option.

I see you have posted a message in another thread regarding memory use. In general, don't worry about unless you see FreeNAS swapping to disk. FreeNAS isn't Windows.


Sent from my iPhone using Tapatalk

 

[devnullius]

No worries about RAM, though it's like looking to a piece of vandalized art :)

I understand some variation when small files are involved, but seen the stable 800Mb/s transfer rates I had yesterday, I'm really surprised about the slow speeds I'm experiencing now. No worries for today: save your battery, I'll keep it running through the night :)

Devvie

 

[devnullius]

Search the forum with keywords like thousands of small files slow copy ... There are threads out there, I just can't search them from my phone

From https://forums.freenas.org/index.php?search/2132352/&q=small+files+slow+copy&o=date
- disable DOS attributes: https://forums.freenas.org/index.php?threads/cifs-directory-browsing-slow-try-this.27751/
- lack of cores: https://forums.freenas.org/index.ph...t-40-45mb-s-per-host-copying-to-freenas.28296
- https://forums.freenas.org/index.php?threads/slow-file-transfer-to-nas.1155/#post-6537

None of it seems to explain my extreme slow speeds fully. I'll look at the DOS attribs tomorrow.

Good night

Post-edit: still slow as sh*t :( More in +12 hours...
Post-edit: +12 hours later; still slow though if my flacs come by, speed goes 'up' to about 5 or 7 MB/s... Not good. I just did the DOS attributes thing, not noticing any advantages either. Sigh. This ain't fun, and I'm open to suggestions once more :)

 

[devnullius]

It's more like 1 file/sec than 10 or 30... In response to: https://forums.freenas.org/index.ph...peeds-with-many-small-files.11360/#post-52124 Currently I'm looking at 2 to 4 KB/s... That's 1992 levels! :(

Quote: "There's 2 reason why speed suffers... here's why: (I'll assume copying a file to the server for my examples, but going the other way has the same limitations)

When copying a file the sharing protocol (as far as I know all of them work this way) requires extra communication for each file. Here's basically what the protocol looks like. Each step won't occur until the previous step is complete.

1. Workstation -> Server : "I want a new file called document.doc"
2. Server -> Workstation : "Your file is created.. send me any contents you want" (The server verifies appropriate permissions and creates the file)
3. Workstation -> Server : File contents sent. (The server does send acknowledgement packets as the file transfers too)
4. Workstation -> Server : "File contents completely sent. Please close the file"
5. Server -> Workstation : "File is closed and saved to the hard drive" (The server MUST complete all applicable writes from the write cache before sending this command. This is often called a sync write.)*
6. Workstation will then delete the file locally or begin with #1 if only copying.

Overall, at each step you are waiting for the other machine to complete their step.

Now, also consider the fact that as each read or write is performed, if you are using magnetic media you will also be adding in seek times for the hard disk heads. Typically, if you have a seek time of 5ms you can expect an absolute maximum of 200 writes per second without any consideration of any other latency. Now consider that you are also executing 3 independent writes for each file... the file system creating the file, the file contents being written, and the file system closing the file. So now, you are really talking about a maximum of, at best, about 65 files per second. Add in the latency of the protocol plus the fact that the workstation will also have it's own 5ms of seek times and you can easily find yourself with only 10-30 files per second. If each of those files is only a few kb, then you can expect only 10-30 times the file size for total transfer rate.

There's no easy way to beat this. Even when one side has an SSD (thereby removing the 5ms penalty for seeks) the other side will still be quite busy trying to complete those seeks.

* - If the amount of data that needs to be written is less than 64 kilobytes and you have a ZIL for the zpool (preferably on an SSD) then the sync write will happen to the ZIL instead of your pool and you will save a small amount of time. However, your ZIL will still have to be process at some point in the future (typically within 2-15 seconds) so your zpool will still end up busy at some point in the very near future. This could potentially slow down future writes too. There is a way to disable ZIL sync writes without a ZIL, but that is extremely dangerous for your data and can result in corruption of the zpool and loss of data. I don't remember the exact tweak to disable sync writes, but if I did know it I don't know if I'd share it because it is almost irresponsible to use it in most situations. For small file transfers it won't significantly help anyway.

Hopefully this all makes sense."

Somebody else describing the same slow speed: https://forums.freenas.org/index.ph...l-files-copy-speed-through-cifs-sharing.9378/

Once I'm done copying the first disk, I'll try again with a 2nd disc, this time from linux. See where the bottleneck is :/

 

[devnullius]

I'm now on Linux (ZevenOS to be exact) and started 2 discs at once.

First disc started 5 minutes ahead, and in that timeframe it had HALF the speeds I had with Windows FOR MY BIG files (time will tell what happens when it comes at all my small files). Speed is about 52MB/s.

THEN I started the second disc (same kind of disc when it comes to big and small files). That one too only runs at about 53MB.

BUT they BOTH are running now at their respective max speeds, giving a total throughput of about 106MB according to FreeNAS (with 800Mb/s throughput on the LAN interface; with 106MB I expected it to be at the full 1Gb speed though... Not at 800).

All in all, I'll keep it running overnight. For comparison, 7zip did about 17 hours about copying it all into an archive. ZevenOS expects to be done in about 7 hours.

We'll see :)

Devvie

Post-edit: Linux was a bit better, not great.