Hi Gurus,
I have a Raidz-2 X 3 array built out of enterprise 8TB drives in a SAS enclosure connected to SAS 3008 adapter. Drives seem to be healthy according to smart
After extensive snapshot cleanup and removal of a child dataset and a reboot, It takes a very long time to import the pool. after the import there is a very small amount of Read activity on all the drives and the amount of free space does not increase. The pool reports it has to free 44.1 TB , but the progress is very slow. There is no/minimal processor usage on the server ( 24 cores, 2.4 GHZ , 48GB RAM ) .
I have experimented with vfs.zfs.vdev settings to see if I can improve concurrent IO but they have no effect and IO is limited to ~100 IOPS distributed across the drives 4-5 IOPS per each drive.
Is there a Tunable to improve disk utilization and speedup space reclamation ?
I have a Raidz-2 X 3 array built out of enterprise 8TB drives in a SAS enclosure connected to SAS 3008 adapter. Drives seem to be healthy according to smart
After extensive snapshot cleanup and removal of a child dataset and a reboot, It takes a very long time to import the pool. after the import there is a very small amount of Read activity on all the drives and the amount of free space does not increase. The pool reports it has to free 44.1 TB , but the progress is very slow. There is no/minimal processor usage on the server ( 24 cores, 2.4 GHZ , 48GB RAM ) .
I have experimented with vfs.zfs.vdev settings to see if I can improve concurrent IO but they have no effect and IO is limited to ~100 IOPS distributed across the drives 4-5 IOPS per each drive.
Is there a Tunable to improve disk utilization and speedup space reclamation ?
Code:
zpool status MIR pool: MIR state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Jan 27 19:57:06 2018 0 scanned at 0/s, 0 issued at 0/s, 157T total 0 resilvered, 0.00% done, no estimated completion time config: NAME STATE READ WRITE CKSUM MIR ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 gptid/66e02f58-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/674d5c05-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/67bfe02b-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/683dae34-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/68ac69a1-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/691c2fb8-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6990bb62-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6a02b3e3-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 gptid/6a7741b5-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6ae8e468-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6b5fea1d-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6bd92302-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6c50b6e6-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6cc71f8c-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6d3d3c2b-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 gptid/6db228f7-8b45-11e5-848b-90e2ba2bc0b8 ONLINE 0 0 0 raidz2-2 ONLINE 0 0 0 gptid/4a2bb2e3-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4aa8bb63-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4b2825b5-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4ba9b48e-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4c235dd0-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4ca161b5-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4d1e660f-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0 gptid/4d9b30f9-876f-11e6-b6b8-7845c4f283a4 ONLINE 0 0 0
Code:
# zfs list MIR NAME USED AVAIL REFER MOUNTPOINT MIR 131T 9.67T 1.54M /MIR
Code:
# zpool get all MIR NAME PROPERTY VALUE SOURCE MIR size 174T - MIR capacity 88% - MIR altroot - default MIR health ONLINE - MIR guid 12821844164206093981 default MIR version - default MIR bootfs - default MIR delegation on default MIR autoreplace off default MIR cachefile - default MIR failmode continue local MIR listsnapshots off default MIR autoexpand on local MIR dedupditto 0 default MIR dedupratio 1.25x - MIR free 19.7T - MIR allocated 154T - MIR readonly off - MIR comment - default MIR expandsize - - MIR freeing 44.1T default MIR fragmentation 0% - MIR leaked 0 default MIR feature@async_destroy active local MIR feature@empty_bpobj active local MIR feature@lz4_compress active local MIR feature@multi_vdev_crash_dump enabled local MIR feature@spacemap_histogram active local MIR feature@enabled_txg active local MIR feature@hole_birth active local MIR feature@extensible_dataset active local MIR feature@embedded_data active local MIR feature@bookmarks enabled local MIR feature@filesystem_limits enabled local MIR feature@large_blocks active local MIR feature@sha512 enabled local MIR feature@skein enabled local