Pool Degraded disk unavailable Unable to replace disk

[Martin Jones]

Build FreeNAS-11.1-U2
Platform Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
Memory 16311MB
4x 3TB WDRED's Raid Z2-0

OK I'll start by saying that "One is born every minute" and I'm the one. What does every moderator say? Don't do anything you don't understand!!!!!!!!!! welcome to my world!

This started as a simple boot failure, but recreated and replaced the boot drive. Then I made the mistake. Took the additional iso drive out of the USB port put it on the table and rebooted the system. I then tried to mirror the boot before importing the pool. Unfortunately I'd forgotten to add the additional flash drive and without realising, created the mirror from one of my pool disks ADA0. So when I imported the pool, of course, got the following error;

CRITICAL: March 28, 2018, 8:32 a.m. - The volume Systems state is DEGRADED: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state.

SO, I went looking for how to replace failing drive, went to Volume Status expecting to be able to offline the drive but no offline button existed. Tried to replace the disk and got this error;

Replacing disk 10426849769521457601
Force: Disk is not clear, partitions or ZFS labels were found.

So did some further looking and found the "Mirror" of the boot drive as a stripe da0p2, with the flash drive still on the table in front of me.

Now I'm completely confused. How can I, if at all, resolve this without losing 2.5TB of movies and music.

 

[nightshade00013]

Please use a SSH session to find the output of

Code:

zpool status

and then put the output in code tags.

 

[Martin Jones]

Code:

		  raidz2-0									  DEGRADED	 0	 0	
 0																			
			10426849769521457601						UNAVAIL	  0	 0	
 0  was /dev/gptid/79b80100-273d-11e6-b667-3464a99adae4						
			gptid/7abac781-273d-11e6-b667-3464a99adae4  ONLINE	   0	 0	
 0																			
			gptid/7bbc3374-273d-11e6-b667-3464a99adae4  ONLINE	   0	 0	
 0																			
			gptid/7cbfe54b-273d-11e6-b667-3464a99adae4  ONLINE	   0	 0	
 0																			
																				
errors: No known data errors													
																				
  pool: freenas-boot															
 state: ONLINE																
  scan: none requested														
config:																		
																				
		NAME		STATE	 READ WRITE CKSUM								
		freenas-boot  ONLINE	   0	 0	 0								
		  da0p2	 ONLINE	   0	 0	 0								
																				
errors: No known data errors													
[root@freenas ~]#

 

[nightshade00013]

Ok so let me get this straight, you replaced your boot drive with a drive that was in a pool. Correct.

So to fix it you will have to do a fresh install onto a USB drive that you were supposed to use. After that you need to boot with it in place and import the configuration. Then take the drive you flubbed up on and re-silver it back into the pool. But until you get the boot volume off of that disk you can't do anything with it.

So first step, install a fresh copy of FreeNAS on to the drive it should be on. It may be a wise idea to do this in a different computer that does not have a HDD hooked up. If you are wanting two drives mirrored install to them both at the same time. Trying to add it later already got you into the problem you have now.

Second step make sure you have a config backup made of your current FreeNAS. https://www.45drives.com/wiki/index...tions_in_FreeNAS_in_the_event_of_boot_failure will help you through the backup as well as restoring in a later step.

Now shut the FreeNAS down. Insert the USB drive and make sure the system is set to boot from it and not a HDD. If it boots from the HDD you will have to go into the BIOS and either remove all HDD's from being bootable or make sure that the USB drive is higher in the list.

Boot the system and once you are sure it is on the USB drive import the config. No need to go through the setup wizard, just cancel, skip or close it. Then import the config as the directions above give.

Now you should be able to place the drive back into the pool and have it re-silver. Once it is done you should be up and doing just fine.

If you have an issue with putting the drive back into the pool then you will need to figure out which one it is https://forums.freenas.org/index.php?resources/identify-your-drives-by-serial-number.64/ and then probably pull it and dump it to get rid of the partitions that are causing an issue.

Just go slow and make sure you know what you are doing before you do it. This is a measure three times cut once type of deal right now. If you are not 100% sure ask questions.

 

[Martin Jones]

OK the comment in the first line is incorrect. The creation and replacement of the BOOT drive was successful. The problem occurred when I tried to mirror the BOOT and by mistake used a 3TB disk in the zpool (ADA0). I'm sure the end result is the same, ADA0 has been overwritten with BOOT data thereby making it unavailable not failed, if I'm correct. Which is why I can't replace it.

I cannot get the zpool to resilver automatically so should I do one of the two below?

1. Pull ADA0 (I know the number) and format it/wipe it and put it back?
2. Wipe it using the "wipe button" in view disks.

 

[danb35]

So from your zpool status output above, it looks like ada0 is no longer part of that pool, right? So you've got your boot pool on a single device (da0), and you've got a disk that used to be part of your main pool (ada0), that you inadvertently removed from there and added to your boot pool, that you've now removed from the boot pool, and want to add back to your main pool?

If I got that right, a full wipe shouldn't be necessary. Post the output of gpart show ada0 in code tags to see if there's a partition table on that disk.

 

[Martin Jones]

[root@freenas ~]# gpart show ada0
=> 40 5860533088 ada0 GPT (2.7T)
40 1024 1 bios-boot (512K)
1064 5860532056 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

 

[danb35]

In code tags, please. But yes, that's showing that there's a GPT partition table on that disk. You should be able to do zpool labelclear /dev/ada0p2 followed by gpart destroy ada0, and then do the replacement through the GUI.

 

[Martin Jones]

AM I missing something here first entry gives error response

Code:

failed to read label from /dev/ada0p2

 

[danb35]

failed to read label from /dev/ada0p2

That's OK, it means the ZFS pool label has already been cleared.

 

[Martin Jones]

Also tried Same response

Code:

failed to read label from /dev/ada0

 

[danb35]

There wouldn't be a label on ada0 in any event; it would have been on ada0p2, but it's apparently been cleared already. You should still be able to do the gpart destroy.

 

[Martin Jones]

Code:

gpart: Device busy

 

[Martin Jones]

OK since The above response was not very useful I'm gonna wipe the whole drive and see if that works. Thanks for the input and I'll come back for more advice

 

[Martin Jones]

wiped ADA0. Replace worked and now its resilvering. Took 2 hours and done.