Expected SMB performance difference between single big file vs mixed folder?

seanm

Guru
Joined
Jun 11, 2018
Messages
570
I'm setting up my first FreeNAS and doing performance tests of SMB shares.

If I read or write, from Mac or Windows, a single large (6 GB) file, I get between 80-110 MB/s, which seems to be what I'd expect.

But if, instead of a single big file, I test with a big folder with many files, big and small, then performance is significantly worse (20-50 MB/s). I would expect it to be somewhat slower, but the magnitude surprises me. Is this expected?

(My hardware is from iXsystems: Xeon E5-2630 v4 @ 2.2 GHz, 64 GiB RAM, 1 pool with: 2 vdevs of 2 mirrored HDDs and 200 GB SSD SLOG, 4x1GbE using LACP.)
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
How do you know the client isn't the slow part? Also that shot probably isn't doing anything.

Random smaller file operations are always going to be slower.
 

seanm

Guru
Joined
Jun 11, 2018
Messages
570
How do you know the client isn't the slow part?

I don't. Though I've tried with two clients, one Mac and one Windows, both have similar slowdowns.

Also that shot probably isn't doing anything.

shot=slog? It's maybe not helping here, but wasn't much more $ to have it, and from what I've read it may help with Time Machine over SMB.

Random smaller file operations are always going to be slower.

Yes, as I said I expected it to be somewhat slower, but am wondering if 2x-4x slower is what I should be expecting.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
I don't. Though I've tried with two clients, one Mac and one Windows, both have similar slowdowns.



shot=slog? It's maybe not helping here, but wasn't much more $ to have it, and from what I've read it may help with Time Machine over SMB.



Yes, as I said I expected it to be somewhat slower, but am wondering if 2x-4x slower is what I should be expecting.
Sorry yes I was talking about slog, auto correct on phone. Slog with have zero impact on anything over smb, in the default config.

What happens if you run something like crystal disk benchmark in your smb share?
 

seanm

Guru
Joined
Jun 11, 2018
Messages
570
Sorry yes I was talking about slog, auto correct on phone. Slog with have zero impact on anything over smb, in the default config.

That wasn't my impression from this comment:
https://www.ixsystems.com/community...ime-machine-over-smb.70604/page-2#post-513609

You imply there is a config to change?

What happens if you run something like crystal disk benchmark in your smb share?

Never heard of it, thanks for the suggestion! Will investigate it...
 
Joined
Jan 4, 2014
Messages
1,644

seanm

Guru
Joined
Jun 11, 2018
Messages
570
Great thread, thanks! I will digest it.

(I think I missed that thread in my searches because I was looking for "SMB" and that thread calls it "CIFS", I should remember to search both in the future.)
 

seanm

Guru
Joined
Jun 11, 2018
Messages
570
So APM was already off for me (the default I guess).

I also already had case insensitive datasets and "case sensitive = true" in the samba options (turning it off does worsen performance a little bit).

I also tried those various options:
Code:
ea support = no
store dos attributes = no
map archive = no
map hidden = no
map readonly = no
map system = no
aio write size = 0
aio read size = 0


None made any difference (neither faster nor slower).

My test case is a folder described by macOS Finder as "9 117 137 579 bytes (9.14 GB on disk) for 289 items". Copying it *to* my FreeNAS takes about 5m15s (about 29 MB/s). I've also tried copying it to another Mac I have, which has a single SSD, and that takes about 9m45s (15.6 MB/s). So I guess FreeNAS performance beats the competition at least. During the copy, FreeNAS CPU usage reported by top is 50-75% in the WCPU column for smbd. Copying a single large file WCPU is steady at about 18%. This is a 10 core Intel Xeon E5-2630 v4 @ 2.2 GHz. Reading my test folder *from* FreeNAS is faster (3m20s, 45.6 MB/s), and WCPU is lower, around 15-40% for smbd.

So I'm not sure if this is the best that can be achieved, or if this performance is lower than expected.
 
Top