Resilver time

[MtK]

Hey,
Assuming 1-2 Tb 5400 RPM SATA disks.
What should be the expected resilver time after a disk replacement, on a 2-way mirror vs a 6-8 Raidz2?

 

[joeschmuck]

Unknown. There are other factors involved like your CPU horsepower, the work load on the system, and how much data is stored on the drives. So this makes the issue system specific. Is there an issue you are having?

 

[danb35]

I'd like to emphasize one of @joeschmuck's points, that being the amount of data on the pool. Traditional RAID rebuilds hit every block of the drive, but ZFS (since it's both the filesystem and the volume manager) knows which blocks are used and which aren't, and thus only rebuilds the used blocks. Thus, a 100 TB array with only 100 GB used will rebuild very quickly, while a 10 TB array with 6 TB used will take much longer, pretty much irrespective of the other factors involved.

 

[joeschmuck]

I was thinking the poster was resilvering and it was taking longer then they expected which is why I asked if there was an issue. It wouldn't be the first time that question was asked.

 

[MtK]

I'm not resilvering and not having any issues.
I'm planning and thinking for a future build or re-build.

For the sake of argument, let's assume the same amount of data...

 

[danb35]

Assuming everything is equivalent (same disk size, same amount of data, same motherboard/CPU/RAM), I'd expect the resilver to be faster on a 6-disk RAIDZ2 array than on a 2-disk mirror. Even though RAIDZ2 requires some involved computations to use two disks' worth of parity, each disk would only have 1/4 as much data. The resilver would involve reading the same amount of data, but from 5 disks rather than one, and writing 1/4 of the total amount of data, rather than the entire amount as needed in a mirror. How much faster? That's the wildcard. I'd guess 2-3x faster, but that's nothing more than a slightly-educated guess.

 

[MtK]

Assuming we have 6 disks it also means 3 striped mirror, so the read woukd still be from 5 disks.
Am I right?

 

[danb35]

The question you posted asked about a two-disk mirror vs. a six-disk RAIDZ2. If you change those parameters, of course you'll change the outcome. In the case of three striped two-disk mirrors, I would expect the resilvering to involve reading from a single disk (the pair of the one you're replacing), and writing its contents to the new disk. Assuming the pool was built with six disks in three striped two-disk mirrors from the beginning, each disk should have 1/3 of the pool's total data on it. Thus, resilvering would involve reading 1/3 of the pool's total data from one disk, and writing that same data out to its mirror (the new/replacement disk you're installing). Still more data to write than needed to resilver a six-disk RAIDZ2 pool, but much less to read, and much less math to do. I'd expect that the mirrors in this case would be faster, but I wouldn't care to speculate how much faster.

 

[MtK]

Thanks