Convert 2 way mirror vdev to 3 way - or Add disk to one mirror'ed vdev only?

Status
Not open for further replies.

Rand

Guru
Joined
Dec 30, 2013
Messages
906
Hi,
so I recently acquired both a ZeusRam and a small P3700 and decided to have some fun running a few benchmarks.

I have a pool consisting of 12 mirror'ed Toshiba DT01ABA300's which I have used for about 3,5 years now.
After running benchmarks on the pool for about a day I saw an email that one of the disks had a bunch of SATA errors.

Darn.
Ok, no problem, I have a replacement drive ready. So I plug it in, resilvering starts ... SATA error count on *newdev* increased ... Grrr.

Still no problem, I have a second replacement drive ready (I moved one of the drives to my desktop and didn't want to add another vdev and be without spare...)
Resilvering starts - SATA error count on *newnewdev* increased - now this is getting ugly :(

So I decided to move the third disk out of my desktop again where it was running fine (so I thought) and after much pain trying to clone it I just reinstalled the desktop to finally free up the third Toshi drive ...
I should have heeded the indications - as soon as I plug in the drive and resilvering starts... Pending sectors's on the third drive...

Needless to say that my offsite-backup had some issues recently (zfs snapshots not running as expected for 3 months and now its totally out of sync since I can't copy 4 TB over my internet connection in any reasonable time)

So, now I am looking what to do.
The pool was due for a makeover anyway, but that was not planned for *now* ;)

Options:
-Simply buy a new disk size 3TB+ and be done with it, better €80 less than loosing data (I'd rather spend the money on the new pool to be honest)
-Add additional protection to impacted vdev (only) to be reasonably sure I can keep the data save for a month or two until new pool is ready.

So after a long intro now the question - how would I go about option two?

I know I can have 2 or 3 or n-way mirror vdevs, but can I have 5 2-way mirrors and one 4way mirror?

This is my current pool :


Code:
zpool status main
  pool: main
state: ONLINE
  scan: resilvered 3.30T in 5h43m with 0 errors on Tue Feb 14 18:02:58 2017
config:

  NAME  STATE  READ WRITE CKSUM
  main  ONLINE  0  0  0
  mirror-0  ONLINE  0  0  0
    gptid/3370ec45-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/36356fe1-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-1  ONLINE  0  0  0
    gptid/94740bc0-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/9797151c-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-2  ONLINE  0  0  0
    gptid/b03ec4f1-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/72897344-f2a7-11e6-94ed-005056844136  ONLINE  0  0  0
  mirror-3  ONLINE  0  0  0
    gptid/0b0df3cb-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/0db36f27-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-4  ONLINE  0  0  0
    gptid/2325c1ad-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/254b5c2b-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-5  ONLINE  0  0  0
    gptid/3bef9fe5-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
    gptid/3e00141a-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  logs
    nvd0  ONLINE  0  0  0


What I would want is to add one or even two additional mirror drives to mirror-2 only because one of the drives in that vdev has shown a bunch of SATA errors. Those are not increasing under light read load, but I don't really trust it.
The other drives are fine, Smart is fine as well and no issues whatsoever (actually had next to no issues with these drives before).

Thanks:)
 
Last edited:

snaptec

Guru
Joined
Nov 30, 2015
Messages
502
Yes you can do that.

I would also change the sata cable if you used the same slot or check if the plug has good connection.
3 hard drives getting worse that worked till now is really not normal.


Gesendet von iPhone mit Tapatalk
 

Rand

Guru
Joined
Dec 30, 2013
Messages
906
Good idea re cables.

Edit: Found it - its zpool attach.

zpool attach <pool> <existing device to mirror> <new device to add to mirror>

Code:
 
~# zpool attach main gptid/72897344-f2a7-11e6-94ed-005056844136 gptid/f5dbbc16-f2a5-11e6-94ed-005056844136
~# zpool status main
  pool: main
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
  continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Wed Feb 15 15:02:39 2017
  90.5M scanned out of 10.0T at 11.3M/s, 257h38m to go
  14.5M resilvered, 0.00% done
config:

  NAME  STATE  READ WRITE CKSUM
  main  ONLINE  0  0  0
  mirror-0  ONLINE  0  0  0
  gptid/3370ec45-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/36356fe1-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-1  ONLINE  0  0  0
  gptid/94740bc0-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/9797151c-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-2  ONLINE  0  0  0
  gptid/b03ec4f1-1a7c-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/72897344-f2a7-11e6-94ed-005056844136  ONLINE  0  0  0
  gptid/f5dbbc16-f2a5-11e6-94ed-005056844136  ONLINE  0  0  0  (resilvering)
  mirror-3  ONLINE  0  0  0
  gptid/0b0df3cb-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/0db36f27-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-4  ONLINE  0  0  0
  gptid/2325c1ad-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/254b5c2b-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  mirror-5  ONLINE  0  0  0
  gptid/3bef9fe5-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  gptid/3e00141a-1a7d-11e6-b6d4-000c2989f189  ONLINE  0  0  0
  logs
  gptid/857bc521-2ca1-11e6-9dc7-000c29e88918  ONLINE  0  0  0



:)
 
Last edited:

fta

Contributor
Joined
Apr 6, 2015
Messages
148
IMO, this is how replaces should always be done with mirrors, i.e. turn a 2 way mirror into a 3 way, then take the bad disk out of the 3 way. The reason is if you have a read error on the non-bad disk, it's possible you could still get the data from the "bad" disk for populating the third mirror. This means it's safer than removing the bad disk and then putting in the new drive.
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
this is how replaces should always be done with mirrors
It's simpler to do a replace without setting the 'bad' disk to offline first. No need to reach for the CLI. See the documentation for replacing drives to grow a pool.
 

GBillR

Contributor
Joined
Jun 12, 2016
Messages
189
It's simpler to do a replace without setting the 'bad' disk to offline first. No need to reach for the CLI. See the documentation for replacing drives to grow a pool.
Agree... However, to answer his initial question:
Add additional protection to impacted vdev (only) to be reasonably sure I can keep the data save for a month or two until new pool is ready.
The answer provided is the only way to increase a mirrored vdev by one drive... correct? I do not see a method in the documentation to accomplish this via the GUI. Just want to be sure I am not missing it. The documentation states that if you use the replace option, the "replaced" drive will be offlined once the process is complete.

Thanks.
 
Status
Not open for further replies.
Top