Recovering data

[gearbox]

Im new to zfs and think i seriously messed up freenas box

I originally setup a pool with raidz using 4 disks. Feeling good about everything i decided to add another USB drive to the pool. I connected with ssh, plugged in a USB drive, did zpool add...

So far so good. i guess while moving though the USB cord got wrapped around my foot, USB disk went flying through the air and smashed hard on the concrete floor.

The pool will not import now.

I quickly ran to best buy and bought a replacement drive, freenas webui is telling me that it cannot find enought free space for the pool.

is it possible to get the data off the drives in the raidz and start over? or is that data gone now? (yes, I should have backed it up before i started messing around but i didn't)

zpool status shows: (after having to issue zpool import -F Forge to get that far)



freenas# zpool status
pool: Forge
state: UNAVAIL
status: One or more devices could not be opened. There are insufficient
replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-3C
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
Forge UNAVAIL 0 0 0 insufficient replicas

raidz1 ONLINE 0 0 0

gpt/ada0 ONLINE 0 0 0
gpt/ada1 ONLINE 0 0 0
gpt/ada2 ONLINE 0 0 0
gpt/ada3 ONLINE 0 0 0​

da1 UNAVAIL 0 0 0 cannot open​

 

[sjieke]

By adding the USB drive to your pool you lost the redundancy of your 4-disk RAID-Z1, so 1 disk lost == lost pool
A little bit more info:

When you created your 4-disk RAID-Z1, freenas created a Raid-Z1 vdev (virtual device) and a pool 'Forge'. It then added the vdev to the zpool. So your pool 'Forge' has 1 disk for redundancy.

Then you added the USB drive to the same volume. For this Freenas created a new vdev with only 1 disk. So this vdev doesn't have any redundancy. Then it added this vdev to your existing pool 'Forge'. This results in all data being striped (like RAID0) accros the 2 vdev's.

Then you lost the USB drive, which resulted in a loss of the second vdev (as it didn't have any redundancy). Loosing an entire vdev of a pool equals in the loss of the entire pool due to the striping.

So moral of the story: If you want to add disks to a pool (add a new vdev), use the same redundancy as the vdev's already in the pool.

 

[gearbox]

Thanks sjieke, So basically, lesson learned, data lossed start new and do it right.

 

[sjieke]

Indeed.

The positive side of the story is that you won't make the same mistake again :)

 

[ProtoSD]

I also added a link to this thread to the FAQ question about adding disks. Hopefully the final version of 8.01 will have some kind of error checking / warning about this. I think there have been a couple of other people here that have been bitten by this also.

 

[lyzanxia]

Does this also apply to adding a cache device later on to the pool? Or is this not crucial to the pool surviving and is the raidz still redundant as before adding the cache device?

 

[sjieke]

I think this doesn't apply to the cache device, it is (or should be) just a cache for reading. But I'm not 100% sure, so better dubble check somewhere.

I do think I read somewhere that removing/losing a dedicated LOG device (ZIL aka ZFS intend log) will make the pool unusable for ZFS versions prior to v19. FreeNAS is currently at v15 I think.

 

[matthewowen01]

loosing your cache will not affect anything, it should keep running even if you tear it out while the system's live.

loosing your zil I'm not quite sure on. i know you will not loose or corrupt existing data but you may loose data that was in the process of being written. i do not know if zfs will automagicaly say it doesn't have a zil and recreate it on the pool like it would if the zil's location was never designated.

if you're a home user, you don't need a cache drive, you won't see any improvement and your money would be better spent on another 8 GB of ram.