Hi,
I have a strange problem with one of our backup FreeNAS boxes. A Couple of months ago we had an unrecoverable error in our zpool so we had to build a new pool from scratch. The box ran fine since, but yesterday we had to reboot it (probably for the first time since the new pool had been created).
What happens now is that FreeNAS tries to import the old (broken) pool. After booting running zpool status I get:
freenas# zpool status
pool: store1
state: UNAVAIL
status: One or more devices could not be opened. There are insufficient
replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-3C
scan: none requested
config:
NAME STATE READ WRITE CKSUM
store1 UNAVAIL 0 0 0
raidz1-0 UNAVAIL 0 0 0
9465009962783256376 UNAVAIL 0 0 0 was /dev/mfid6
3777795644875467493 UNAVAIL 0 0 0 was /dev/gptid/690fb07a-9610-11e2-bd8f-14dae9c4f110
2091297130569300125 UNAVAIL 0 0 0 was /dev/gptid/6982955e-9610-11e2-bd8f-14dae9c4f110
12910823325045097081 UNAVAIL 0 0 0 was /dev/gptid/69ecfb45-9610-11e2-bd8f-14dae9c4f110
664568350415246497 UNAVAIL 0 0 0 was /dev/gptid/6a225b3c-9610-11e2-bd8f-14dae9c4f110
10625246729079390574 UNAVAIL 0 0 0 was /dev/gptid/6a5cf01a-9610-11e2-bd8f-14dae9c4f110
1657281061953720003 UNAVAIL 0 0 0 was /dev/gptid/6a98ee7b-9610-11e2-bd8f-14dae9c4f110
70234070040747385 UNAVAIL 0 0 0 was /dev/gptid/3b599ab0-c45c-11e2-80e2-002590ae58de
So, when I export that broken pool and run a "zpool import" so see which pools are actually available I get only the new (clean) pool:
freenas# zpool import
pool: store1
id: 8712011636306578773
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
store1 ONLINE
raidz1-0 ONLINE
gptid/9ae1827c-eeb5-11e2-b81c-002590ae58de ONLINE
gptid/ebd60f04-ed54-11e2-b81c-002590ae58de ONLINE
gptid/a87f5b8b-efa4-11e2-b81c-002590ae58de ONLINE
gptid/b75bee78-f04d-11e2-b81c-002590ae58de ONLINE
gptid/f5b82166-f430-11e2-b81c-002590ae58de ONLINE
gptid/4d4b22fb-e94c-11e2-b81c-002590ae58de ONLINE
gptid/dafa4ec0-ea51-11e2-b81c-002590ae58de ONLINE
gptid/c2c814ac-ead5-11e2-b81c-002590ae58de ONLINE
That's the correct (and clean) pool. When I detach the broken pool and use the auto-import feature of the FreeNAS GUI it imports the clean pool just fine and all datasets become available.
But when rebooting the machine I have the same problem all over again.
I already tried the following without success: I imported the new (clean) pool under a different name, ran a "zpool destroy" of the old "phantom" pool and re-imported the clean pool under the correct name (store1). That seemed to work at first, but after the next reboot FreeNAS finds and imports the old "phantom" pool again instead of the correct pool.
Where does FreeNAS store it's metadata about the old "phantom" pool and how do I get rid of that? I assumed FreeNAS would use the pool ID for import but if that were the case it would find the correct pool. It somehow seems to get confused that the old and the new pool have the same name.
What's to do in this case? Any help appreciated!
Regards,
Joern
I have a strange problem with one of our backup FreeNAS boxes. A Couple of months ago we had an unrecoverable error in our zpool so we had to build a new pool from scratch. The box ran fine since, but yesterday we had to reboot it (probably for the first time since the new pool had been created).
What happens now is that FreeNAS tries to import the old (broken) pool. After booting running zpool status I get:
freenas# zpool status
pool: store1
state: UNAVAIL
status: One or more devices could not be opened. There are insufficient
replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-3C
scan: none requested
config:
NAME STATE READ WRITE CKSUM
store1 UNAVAIL 0 0 0
raidz1-0 UNAVAIL 0 0 0
9465009962783256376 UNAVAIL 0 0 0 was /dev/mfid6
3777795644875467493 UNAVAIL 0 0 0 was /dev/gptid/690fb07a-9610-11e2-bd8f-14dae9c4f110
2091297130569300125 UNAVAIL 0 0 0 was /dev/gptid/6982955e-9610-11e2-bd8f-14dae9c4f110
12910823325045097081 UNAVAIL 0 0 0 was /dev/gptid/69ecfb45-9610-11e2-bd8f-14dae9c4f110
664568350415246497 UNAVAIL 0 0 0 was /dev/gptid/6a225b3c-9610-11e2-bd8f-14dae9c4f110
10625246729079390574 UNAVAIL 0 0 0 was /dev/gptid/6a5cf01a-9610-11e2-bd8f-14dae9c4f110
1657281061953720003 UNAVAIL 0 0 0 was /dev/gptid/6a98ee7b-9610-11e2-bd8f-14dae9c4f110
70234070040747385 UNAVAIL 0 0 0 was /dev/gptid/3b599ab0-c45c-11e2-80e2-002590ae58de
So, when I export that broken pool and run a "zpool import" so see which pools are actually available I get only the new (clean) pool:
freenas# zpool import
pool: store1
id: 8712011636306578773
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
store1 ONLINE
raidz1-0 ONLINE
gptid/9ae1827c-eeb5-11e2-b81c-002590ae58de ONLINE
gptid/ebd60f04-ed54-11e2-b81c-002590ae58de ONLINE
gptid/a87f5b8b-efa4-11e2-b81c-002590ae58de ONLINE
gptid/b75bee78-f04d-11e2-b81c-002590ae58de ONLINE
gptid/f5b82166-f430-11e2-b81c-002590ae58de ONLINE
gptid/4d4b22fb-e94c-11e2-b81c-002590ae58de ONLINE
gptid/dafa4ec0-ea51-11e2-b81c-002590ae58de ONLINE
gptid/c2c814ac-ead5-11e2-b81c-002590ae58de ONLINE
That's the correct (and clean) pool. When I detach the broken pool and use the auto-import feature of the FreeNAS GUI it imports the clean pool just fine and all datasets become available.
But when rebooting the machine I have the same problem all over again.
I already tried the following without success: I imported the new (clean) pool under a different name, ran a "zpool destroy" of the old "phantom" pool and re-imported the clean pool under the correct name (store1). That seemed to work at first, but after the next reboot FreeNAS finds and imports the old "phantom" pool again instead of the correct pool.
Where does FreeNAS store it's metadata about the old "phantom" pool and how do I get rid of that? I assumed FreeNAS would use the pool ID for import but if that were the case it would find the correct pool. It somehow seems to get confused that the old and the new pool have the same name.
What's to do in this case? Any help appreciated!
Regards,
Joern