Slow deletes from FreeNAS CIFS share.

[mangyDOG]

I have build a FreeNAS based storage server which is working very well providing file shares to my various windows based computers, except when it comes to deleting (or cutting & pasting) files from the NAS.

Here are a series of screenshots to illustrate the issue:

Small file copy from NAS to PC:

Small file CUT from NAS to PC:

File Delete from NAS:

The NAS is connected via 10Gb ethernet and when I use a windows computer with a NVMe drive I have transferred large files to and from it at around 800MB/s. The only issue I have is when it comes to removing files from the NAS share (either delete or cut) then it just tanks. If use the console and delete files directly it only takes a few seconds to complete, but that is not a solution I can use for normal operations.

The NAS is currently running 11.1-U2

Hardware consists of:
AMD Ryzen5 1600 six core CPU
32Gb Unbuffered ECC RAM
2*Sandisk USB thumb drives for OS
8*Seagate enterprise 3Tb 7200rpm SATA hard-drives (single ZFS-Z2 volume with LZ4 compression)
1 * Samsung 960EVO 250Gb NVMe SSD (ARC)
1 * Intel P3600 400Gb PCIe SSD (L2ARC)
1 * Intel X520-DA2 with intel fibre transceiver.


I look forward to any assistance
regards,
mangyDOG.

 

[DrKK]

Your system seems like the posterchild for a system that should not have an L2ARC. Are you sure you need one?

Also, is this purely a SMB/CIFS issue? If you go to the command line and delete the 20000 files with a simple "rm *" (or whatever), does it still operate this slowly?

 

[mangyDOG]

Your system seems like the posterchild for a system that should not have an L2ARC. Are you sure you need one?

Also, is this purely a SMB/CIFS issue? If you go to the command line and delete the 20000 files with a simple "rm *" (or whatever), does it still operate this slowly?

1) I haven't tried the system without the L2ARC cache drive, is there a process to remove the drive?

2) RM from the command line works as expected with all files deleted in a few seconds.

Thanks

 

[DrKK]

Deleting 20000 files from Windows SMB/CIFS, even on a local drive, takes quite a bit of time in many cases, as does transfering a whole boatload of small files. I don't know what to tell you. But if the problem disappears at the command line, and when using, say, FTP to transfer files, then the issue is the general craptasticness of SMB.

Unless someone else has a more constructive idea?

 

[Bidule0hm]

Are the 20k items in the same folder?

 

[Mirfster]

IMHO everything from the GUI in Windows is slow. Have you tried simply doing this stuff from within a CMD Window?

Like using RoboCopy or even RD (to delete entire directories, subfiles and subfolders)?

Also, FWIW (and if my old mind is recalling correctly); when you "Copy and Paste" the file's permissions are inherited from the destination. However, when you "Cut and Paste" Windows will try to maintain the file's perms on the destination. Not 100% if that is a possible issue, but figured I would toss that out there as well.

 

[mangyDOG]

Are the 20k items in the same folder?

One top level folder and multiple sub-folders.

The screen shots are just an example I used. The issue persists with all delete operations. If there are only ten files in a folder, it will still take over 10 seconds (< 1 file delete operation per second) to delete the folder.

I am aware that deletes across the network using SMB are slow, but on my much older NAS units (Synology RAID6 with EXT4 and QNAP RAID6 with EXT3) I get at least >50 delete operations per second and I was expecting that with the one based on FreeNAS.

 

[mangyDOG]

IMHO everything from the GUI in Windows is slow. Have you tried simply doing this stuff from within a CMD Window?

Like using RoboCopy or even RD (to delete entire directories, subfiles and subfolders)?.

Using RM from the FreeNAS console works as expected with large numbers of files deleted within a few seconds. I just tested using a Windows command prompt "del * /s /q" and it was running at about 1 delete per second.

 

[TwistMaitre]

Did you ever find a solution to this mangyDOG? I have a a pretty similar system and an identical problem.

 

[anodos]

From MacOS client (macbook pro with gigabit ethernet) on FreeNAS master, I can delete a directory containing 8200 objects in about 14 seconds, which seems acceptable. When I have time I'll take a look at deletions on 11.2.

 

[kurbis]

Hello. I have same problem. Did anybody find resolution?

 

[anodos]

I changed defaults for 11.3 to improve delete speeds in MacOS. The slowness is a direct result of the type of network traffic generated by the MacOS client. Try following: (1) create dataset that is case-insensitive (2) enable ixnas and remove zfsacl and zfs_space from VFS objects (3) set share auxiliary parameter "case sensitive = true". Restart samba and re-do your test.

 

[kurbis]

I changed defaults for 11.3 to improve delete speeds in MacOS. The slowness is a direct result of the type of network traffic generated by the MacOS client. Try following: (1) create dataset that is case-insensitive (2) enable ixnas and remove zfsacl and zfs_space from VFS objects (3) set share auxiliary parameter "case sensitive = true". Restart samba and re-do your test.

Thank you for the reply. 1. I have windows clients, not MacOS. This OS have same problems? How can i create new pool without removing old pool? 2. When i removed zfacl and zfs_space my cifs permissions stopped work :(

 

[kurbis]

I found resolution fo me! Just disable acl_tdb and acl_xattr . I hope it will not destroy my ntfs permissions on the folders :)

 

[anodos]

I found resolution fo me! Just disable acl_tdb and acl_xattr . I hope it will not destroy my ntfs permissions on the folders :)

acl_xattr does not work on FreeBSD (on ZFS). acl_tdb works except when it's crashing the box. Better not to use those ever and just stick to zfsacl or ixnas.

 

[kurbis]

Is modules streams_xattr and zfsacl enough for working ntfs permissions?

 

[anodos]

Is modules streams_xattr and zfsacl enough for working ntfs permissions?

Yes. There is nearly a 1-1 correspondence between the Windows DACL bits and the zfs ACL bits.

 

[kurbis]

Thank you for replyes!