SMB Directory List Times
2 minute read.
TrueNAS and ZFS by default can support large numbers of files. Per directory this can grow to 281+ trillion and per file system there is no upper limit. However, these are theoritical limits, and there are host and client side practical limits dealing with large quantity of files. Specifically when it comes to how fast you can list or “enumerate” them.
Starting in Dragonfish 24.04 and later, the meta-data performance of directory listings over SMB is greatly improved. Depending on your SMB client, it is possible to support a single directory with anywhere from 100,000 to 1,000,000 files on flash systems with decent load times.
iXsystems has done some very preliminary testing to provide guidance on the approximate time ranges that are possible. Users should understand that as directory file counts increase, listing times increase too.
CPU and Memory speed play a major factor in metadata listing times. For this testing, flash media was used. Spinning disks will generally not perform at the same speeds as flash.
These numbers are provided as rough guides as to what kind of performance might be expected. Many external factors can impact this and your mileage might vary.
Test System Specifications:
- CPU: Intel i5-12600
- Memory: 64GB
- Storage: SAS Flash - Two-Drive ZFS Mirror
Workload: 1K, 100K and 200K Mixed PNG Data
File Count | ||||
---|---|---|---|---|
Operating System | 100K | 250K | 500K | 1 Million |
Windows 11 | 1.5~ Seconds | 4-6~ Seconds | 7-10~ Seconds | 14-20~ Seconds |
OS X (Sonoma) | 15.5 Minutes | Did not run | Did not run | Did not run |
Ubuntu - File Manager (GUI) | 13-15~ Seconds | 1~ Minute | Did not run | Did not run |
Linux SMB (In-Kernel CIFS CLI) | 3~ Seconds | 8~ Seconds | 17~ Seconds | 30~ Seconds |
Linux smbclient (Userspace CLI) | 1.2~ Seconds | 4~ Seconds | 6~ Seconds | 12-14~ Seconds |