Dismal resilver speeds

[Evan Richardson]

First...the bad news:

I'm seeing between 1MB and 10MB/sec resilver speed on an 18 drive raidz2 setup. Typically the speeds hover around 1-2MBps.

Code:

scan: resilver in progress since Mon Nov 30 14:17:43 2020
        70.4T scanned at 35.8G/s, 47.7T issued at 24.3G/s, 96.7T total
        13.8G resilvered, 49.38% done, 00:34:20 to go

13.8GB in 30 minutes

The pool was created with ashift=12.
Drives are all 8TB WD Red 5400 RPM drives
E5-2630L v3, 128GB of ram
Truenas release version.


tunables:
vfs.zfs.resilver_min_time: 6000
vfs.zfs.resilver_delay: 0
vfs.zfs.top_maxinflight: 512 (raised from 128)
vfs.zfs.scrub_delay: 2

There has to be something wrong here. at this speed, it's going to take like 2 weeks to replace this drive?

 

[Evan Richardson]

as soon as I posted this...

Code:

raidz2                                          31.8T  11.7T  2.92K    179   982M   164M

    gptid/d1c1390e-db25-11e7-a7b6-0025905decac        -      -    499      0   164M      0

    replacing                                         -      -    491    179   162M   164M

      gptid/d272a6bb-db25-11e7-a7b6-0025905decac      -      -    491      0   162M      0

      gptid/d97a286d-3359-11eb-b62b-90e2ba266fca      -      -      0    179      0   164M

it sped up. go figure