pool UNKNOWN but likely recoverable

[eeldna 2nd]

Hi,

Aim: I purchased 4 new hard drives to upgrade my NAS. I wanted to remove each current disk, one by one, so the new disk would become part of the pool.
Reality: I put one zfs pool disks offline via the GUI. Then, another disk decided to stop working.
Outcome: the zfs pool (HPNAS) is now UNKNOWN because 2 of the 4 disks are not online.

I have tried to find an equivalent to my issue and have not had luck. In short, I wanted to upgrade my raid-5 (zfs) NAS storage and I took a disk offline. Now the system won't let me do anything because 2 of the 4 disks are not online. If I can put one disk back to "online" from its current "offline", I think the system will be fine.

Motherboard make and model/CPU/Hard disk controllers/Network cards:
- It's a HP N54L
RAM quantity:
- 8GB RAM
Hard drives, quantity, model numbers, and RAID configuration, including boot drives:
- boot from an 8gb USB stick.
- 4 WD Hard Drives (3TB each) in a raid-5 configuration. No encryption.
- The pool name is HPNAS
Freenas version:
- FreeNAS-11.3-U2

Cause:
- I put apa0 "offline", then removed the "first disk" (the left most disk) to put in a new larger disk
- When I took the disk out, it turned out the "first disk" was not apa0 and now I have one disk "offline" and one disk "unavail"

Diagnostics:
"zpool import" shows the following

root@freenas:~ # zpool import
pool: HPNAS
id: 10342560618609615770
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing devices and try again.
see: http://illumos.org/msg/ZFS-8000-3C
config:
HPNAS UNAVAIL insufficient replicas
raidz1-0 UNAVAIL insufficient replicas
1056674614550561172 OFFLINE
17914818884535021667 UNAVAIL cannot open
gptid/b79788f7-7b83-11e1-bd89-e83935209a ONLINE
gptid/1c8ff3ea-5535-11e5-abbc-e83935209a ONLINE

"Storage / Pools" shows the following
HPNAS UNKNOWN

Note1: If I select "Add" and attempt to "Import" a pool, at Step 3 (Select Pool to Import), there are no pools (ie. it shows -- in the drop down list)
Note2: I looked at the URL given (above) and after reading it twice, it contained nothing that stood out to me.

My Plan:
If I understand correctly, there are two things I need to do.
1. Get disk "1056674614550561172" back online
2. Put those disks back into the HPNAS pool

In short, I just want to get the zpool back up and running. I tried to run "zpool online HPNAS <disk>" and "zpool online raidz1-0 <disk>" and both cases it said "no such pool". I also tried to "extend" the pool (Storage / Pools / Extend Pool) but I had no idea what I was doing and it threatened to wipe the disks.

Any help would be appreciated. Thank you

 

[eeldna 2nd]

Here are some screenshots that might give context.

 

[Redcoat]

From the linked file:

"If the device has been temporarily detached from the system, attach the device to the system and run 'zpool status' again. The pool should automatically detect the newly attached device and resume functioning. You may have to mount the filesystems in the pool explicitly using 'zfs mount -a'. "

 

[eeldna 2nd]

From the linked file:

"If the device has been temporarily detached from the system, attach the device to the system and run 'zpool status' again. The pool should automatically detect the newly attached device and resume functioning. You may have to mount the filesystems in the pool explicitly using 'zfs mount -a'. "

Thanks for the suggestion. I have all of the original disks back in the system.
If I run "zpool status", I receive the following.
root@freenas:~ # zfs mount -a
root@freenas:~ # zpool status
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:07:42 with 0 errors on <date>
config:
NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

How do I put "1056674614550561172" back online? I tried "zfs mount -a" and I didn't notice any response.
I'm repeating this for context. My zpool is HPNAS and it has 4 disks. It's a RAID5 array, meaning that 3 of 4 disks is enough to recover the system.

Thank you!

 

[eeldna 2nd]

Thanks for the suggestion. I have all of the original disks back in the system.
If I run "zpool status", I receive the following.
root@freenas:~ # zfs mount -a
root@freenas:~ # zpool status
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:07:42 with 0 errors on <date>
config:
NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

How do I put "1056674614550561172" back online? I tried "zfs mount -a" and I didn't notice any response.
I'm repeating this for context. My zpool is HPNAS and it has 4 disks. It's a RAID5 array, meaning that 3 of 4 disks is enough to recover the system.

Thank you!

Doe this help? See attached image. You can clearly see HPNAS exists.
I typed in "zpool online HPNAS 1056674614550561172" and it says "cannot open 'HPNAS' : no such pool or directory".

Can someone even tell me if "1056674614550561172" is part of any pool?
Thank you.

 

[eeldna 2nd]

After spending the better part of a day looking for more information, I have found the following (I think so...).
1. When you take a disk offline, it's online status can only be determined by having three other workings disks.
2. The disk that is UNKNOWN was quick formatted by mistake. This means the content is on the disk, but, the filesystem is wiped.

so, I tried "gpart recover ada1" (the disk I think was quick formatted). It told me the disk didn't need to be formatted.
I also answered my question 'is "1056674614550561172" is part of any pool'? The answer is no. Under Storage / Disks, you see the list of disks. Each disk will tell you if it belongs to a pool. My four drives all list "Unused", meaning these disks aren't part of a pool.

I'm still trying to work out how to move forward. I think my best bet (if possible) is to recover ada1.