SOLVED Vertical growth: offline, replace or detach?

[zamana]

Hi!

I'm planning to increase my data storage by replacing 2x4TB by 2x8TB (it's a mirrored vdev), like this:

I know that I'll need to replace the disk 3 first, let it resilver, than replace the disk 4 and let it resilver, and then my storage will automagically (I hope...) growth from 20 to 24TB (rounded numbers to easy the calculation).

My question is: what is the correct way to replace healthy disks in this case? Should I use offline, replace or detach over the disks 3 and 4?

Thanks.
Regards.

 

[sretalla]

If data security is a concern (although it shouldn't be much of one since the removed disk is a full and good copy at that point in time anyway), then you can attach a third disk to the mirror, wait for resilver then remove a smaller one and repeat.

Worth noting that this is only possible at CLI in 11.3, but can be done in the GUI in TrueNAS 12.0 Core

 

[zamana]

Hi!

Yes. That's a good strategy.

But let's suppose that I don't have a free slot available in my mobo. How should I proceed taking into account that I will need to take off a healthy disk before insert the new one?

Thanks.
Regards.

 

[HoneyBadger]

But let's suppose that I don't have a free slot available in my mobo. How should I proceed taking into account that I will need to take off a healthy disk before insert the new one?

Performing a scrub and verifying the SMART attributes of the two drives in the "mirror" you're about to break is a good start.

However, I would strongly suggest finding a way to connect the new drives so that you can perform some burn-in testing. The last thing you want to do is replace them all one-by-one and then find out that some of your drives were "factory failures." This doesn't necessarily have to be in the TrueNAS system itself, you could dedicate a standalone machine as a burn-in box to run badblocks for a pass.

 

[Scharbag]

I think the replace command requires the drive to be present so if you cannot add a drive to your system, I think you will need to use the detach command first.

As others said, make sure you burn in your drives before attaching them to your pool.

Cheers,

 

[zamana]

I think the replace command requires the drive to be present so if you cannot add a drive to your system, I think you will need to use the detach command first.

As others said, make sure you burn in your drives before attaching them to your pool.

Cheers,

Ok!

Thank you for the reply and the advice.

Regards.

 

[Sawtaytoes]

I came to this thread wanting to know the answer to this question:
> Should I use offline, replace or detach over the disks 3 and 4?

I'm assuming Replace allows you to swap a drive for another.

Then Detach removes it completely from that zpool.

But Offline is always weird to me. Does that mean a hot-spare will come and save it? What does that really do?

 

[sretalla]

I'm assuming Replace allows you to swap a drive for another.

It does... most specifically to replace a failed or failing disk with a good one.

Then Detach removes it completely from that zpool.

Where that can happen, yes. (not always an option, particularly not in RAIDZ VDEVs)

But Offline is always weird to me. Does that mean a hot-spare will come and save it? What does that really do?

Offlining a disk takes it out of pool operations for a time...

You can then either bring it back (online) or replace it (replace).

Why would you do that?

Well, first reason I can think of is you want to retire a disk that hasn't failed yet. So would subsequently replace it.

Second, if you wanted to do a destructuve or more extensive test on a disk (perhaps if you saw some issues reported by SMART), but don't want the pool to be impacted (too much) while you do that.

Third, if you're following the recommended process to increase a pool's size by replacing each disk in turn with a larger one... offline, replace, rinse, repeat.

If you want a spare to kick in, offlining a disk won't get you there... you also need to damage the disk (like destroy its partition table... incidentally one of the other reasons offline is needed, since you can't do that with it still online in a pool) and bring it back online for the pool to see the need for a spare to be invoked.