bartuc
Cadet
- Joined
- Nov 29, 2021
- Messages
- 7
Hello,
I'm using Truenas Core with SAS disks(1TB NVMe cache). I have 63TB total space on RAIDZ2 pool. lz4 compression is active.
I'm using this space for backing up extremely many files. I cannot see exact number, but I believe there are trillions of files in different directories.
The problem is, removing files/directories is extremely slow, it is almost impossible. I cannot remove dataset, I have to remove a few directories in a dataset. A directory with 1.5M files in it cannot be removed for over a week. I've tried rm -rf and rsync to an empty directory so far. I cannot see any big difference.
Even if I remove a dataset completely with trillions of files, deletion is instant but space is not being freed up instantly, even it is taking forever. No visible usage and free space is going up very slowly.
I can write to the pools over 450MiB/s which is as fast as an SSD. But removing files is a really big pain, almost impossible.
Can you share any suggestions?
P.S. This system have 2 x E5-2430v2 CPUs and 32GB of ram.
I'm using Truenas Core with SAS disks(1TB NVMe cache). I have 63TB total space on RAIDZ2 pool. lz4 compression is active.
I'm using this space for backing up extremely many files. I cannot see exact number, but I believe there are trillions of files in different directories.
The problem is, removing files/directories is extremely slow, it is almost impossible. I cannot remove dataset, I have to remove a few directories in a dataset. A directory with 1.5M files in it cannot be removed for over a week. I've tried rm -rf and rsync to an empty directory so far. I cannot see any big difference.
Even if I remove a dataset completely with trillions of files, deletion is instant but space is not being freed up instantly, even it is taking forever. No visible usage and free space is going up very slowly.
I can write to the pools over 450MiB/s which is as fast as an SSD. But removing files is a really big pain, almost impossible.
Can you share any suggestions?
P.S. This system have 2 x E5-2430v2 CPUs and 32GB of ram.
