Degraded Pool - Any chance to save data?

[bcolvin71]

I have (had is more likely) a pool with 7 WD Red 3.5 TB standard drives (non SSD) in riadZ2. I had been hearing a clicking sound ever so often and haven't had time to nail down the issue. However, on the 23rd, I got a message that one drive had bad sectors, so I remove that drive to send out for repair. After the replacement was re-silvered and back in the system, I heard the clicking again so I was in process of investigating the other drives. My card is hot swappable, so I unplugged 1 drive and waited to hear the sound again. When I didn't hear it, I plugged the drive back in and unplugged another one. Apparently I didnt wait long enough for the system to show the 1st drive back online. I know, not smart on my part. While I had now 2 of the of the 7 offline, of course I had a power failure. Unfortunately now, I have 4 of the 7 drives showing as "degraded". Tried running a scrub on the pool which did not yield any change to the status.
What options do I have, if any, of saving the pool in my system?

 

[jgreco]

Is your data still there? If so, the degraded tag is simply a notice that there's been a lot of errors, and it may be that "zpool clear <yourpoolname>" is all that's needed.

 

[HoneyBadger]

7 WD Red 3.5 TB standard drives

Would you mind humoring an old Badger and posting the model numbers?

I'm really hoping they aren't shingled (SMR) drives.

 

[jgreco]

Didn't all the drive manufacturers stop making oddly sized HDD's awhile back?

 

[HoneyBadger]

Didn't all the drive manufacturers stop making oddly sized HDD's awhile back?

I think a number got dropped somehow there, and it's intended to be written as '7 WD Red 3.5" X TB standard drives' where X is an even integer.

The 'WD Red' part was what caught my eye though, as adding SMR drives to the mix in a resilvering pool - especially WD Reds that might experience a firmware bug - is a caution point for sure.

 

[bcolvin71]

Is your data still there? If so, the degraded tag is simply a notice that there's been a lot of errors, and it may be that "zpool clear <yourpoolname>" is all that's needed.

Yes, the data is still there. I will try the zpool clear command to see what happens.

 

[bcolvin71]

Would you mind humoring an old Badger and posting the model numbers?

I'm really hoping they aren't shingled (SMR)

 

[bcolvin71]

Sorry. They are 3 TB, not 3.5. Took those prior to installing so I could easily see the serials

 

[HoneyBadger]

Okay, the WD30EFZX are the "Red Plus" which aren't SMR - so that part is safe. After a zpool clear, do a scrub to verify that the data is safe.

 

[bcolvin71]

What a relief. The clear command worked and post scrub, all data seems good. Much appreciated jgreco & HoneyBadger.

On a side note, what's the proper procedure to restore a snapshot?

 

[HoneyBadger]

On a side note, what's the proper procedure to restore a snapshot?

Actually rolling back the dataset using zfs rollback is destructive and can cause data loss. It's an option, but you can also clone the snapshot, mount it elsewhere, and then copy data back in, as described in:

https://www.truenas.com/docs/scale/scaletutorials/storage/creatingsnapshots/#rollback

Replications use the existing snapshot when doing an incremental backup, and rolling back can put the snapshots out-of-order. To restore the data within a snapshot, the recommended steps are:

1. Clone the desired snapshot.
2. Share the clone with the share type or service running on the TrueNAS system.
3. Allow users to recover their needed data.
4. Delete the clone from Storage.