replacing drive with cold spare

Status
Not open for further replies.
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
Looking around I can't seem to find a process for replacing a drive with a cold spare. I off lined the drive and put the new one in but it won't see the new one it just let's me replace the drive with the old one. I tried turning it off and booting with the new drive but for some reason that totally removes my zpool and all the disks show up fine but not in a raid group. Booting back up with the old drive everything is good except I'm still in degraded mode. Any commands I can do to like rescan drives when I hook the new one in? Bit of help would be great. Raidz1 4 drives.
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,402
From a SSH session as root output of:
Code:
zpool status -v

camcontrol devlist

glabel status

gpart show


Use [code][/code] tags to preserve the formatting.
 
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
Here are the outputs you need with my current drive in offline and everything is working in degraded mode.

Code:
[root@freenas] ~# zpool status -v
  pool: Vol0
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: scrub repaired 0 in 6h40m with 0 errors on Sun Dec 30 06:40:58 2012
config:

        NAME                                            STATE     READ WRITE CKSUM
        Vol0                                            DEGRADED     0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/338243b8-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0
            gptid/33c3355c-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0
            gptid/342046fb-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0
            gptid/34a6ad54-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0
          raidz1-1                                      DEGRADED     0     0     0
            2088074764308451236                         OFFLINE      0     0     0  was /dev/dsk/gptid/518cbece-e7fc-11e1-8afc-00045fb309e8
            gptid/7b92c455-2bca-11e2-ad71-00045fb309e8  ONLINE       0     0     0
            gptid/5270dda3-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0
            gptid/52e8cd24-e7fc-11e1-8afc-00045fb309e8  ONLINE       0     0     0


Code:
[root@freenas] ~# camcontrol devlist
<SAMSUNG HD154UI 1AG01118>         at scbus0 target 0 lun 0 (pass0,ada0)
<ST2000DL004 HD204UI 1AQ10001>     at scbus1 target 0 lun 0 (pass1,ada1)
<ST2000DM001-9YN164 CC4H>          at scbus3 target 0 lun 0 (pass2,ada2)
<ST1500DL003-9VT16L CC32>          at scbus4 target 0 lun 0 (pass3,ada3)
<ST31500341AS CC1H>                at scbus5 target 0 lun 0 (pass4,ada4)
<ST2000DL004 HD204UI 1AQ10001>     at scbus6 target 0 lun 0 (pass5,ada5)
<ST2000DL003-9VT166 CC3C>          at scbus7 target 0 lun 0 (pass6,ada6)
<SAMSUNG HD154UI 1AG01118>         at scbus8 target 0 lun 0 (pass7,ada7)
<Corsair Voyager 1100>             at scbus9 target 0 lun 0 (pass8,da0)


Code:
[root@freenas] ~# glabel status
                                      Name  Status  Components
gptid/34a6ad54-e7fc-11e1-8afc-00045fb309e8     N/A  ada0p2
gptid/338243b8-e7fc-11e1-8afc-00045fb309e8     N/A  ada1p2
gptid/342046fb-e7fc-11e1-8afc-00045fb309e8     N/A  ada2p2
gptid/52e8cd24-e7fc-11e1-8afc-00045fb309e8     N/A  ada3p2
gptid/5270dda3-e7fc-11e1-8afc-00045fb309e8     N/A  ada4p2
gptid/33c3355c-e7fc-11e1-8afc-00045fb309e8     N/A  ada5p2
gptid/7b92c455-2bca-11e2-ad71-00045fb309e8     N/A  ada6p2
gptid/518cbece-e7fc-11e1-8afc-00045fb309e8     N/A  ada7p2
                             ufs/FreeNASs3     N/A  da0s3
                             ufs/FreeNASs4     N/A  da0s4
                            ufs/FreeNASs1a     N/A  da0s1a


Code:
[root@freenas] ~# gpart show
=>        34  2930277101  ada0  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>        34  3907029101  ada1  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  3907029101  ada2  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  2930277101  ada3  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>        34  2930277101  ada4  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>        34  3907029101  ada5  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  3907029101  ada6  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834696     2  freebsd-zfs  (1.8T)
  3907029128           7        - free -  (3.5k)

=>        34  2930277101  ada7  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>      63  15950529  da0  MBR  (7.6G)
        63   1930257    1  freebsd  [active]  (942M)
   1930320        63       - free -  (31k)
   1930383   1930257    2  freebsd  (942M)
   3860640      3024    3  freebsd  (1.5M)
   3863664     41328    4  freebsd  (20M)
   3904992  12045600       - free -  (5.8G)

=>      0  1930257  da0s1  BSD  (942M)
        0       16         - free -  (8.0k)
       16  1930241      1  !0  (942M)


I will post again with same after I put the new drive in a boot up.
 
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
Here is with the new drive in and my volume is gone, all drives are listed to make a new Vdev.

Code:
[root@freenas] ~# zpool status -v
no pools available


Code:
[root@freenas] ~# camcontrol devlist
<SAMSUNG HD154UI 1AG01118>         at scbus0 target 0 lun 0 (pass0,ada0)
<ST2000DL004 HD204UI 1AQ10001>     at scbus1 target 0 lun 0 (pass1,ada1)
<ST2000DM001-9YN164 CC4H>          at scbus3 target 0 lun 0 (pass2,ada2)
<ST2000DM001-1CH164 CC24>          at scbus4 target 0 lun 0 (pass3,ada3)
<ST31500341AS CC1H>                at scbus5 target 0 lun 0 (pass4,ada4)
<ST2000DL004 HD204UI 1AQ10001>     at scbus6 target 0 lun 0 (pass5,ada5)
<ST2000DL003-9VT166 CC3C>          at scbus7 target 0 lun 0 (pass6,ada6)
<SAMSUNG HD154UI 1AG01118>         at scbus8 target 0 lun 0 (pass7,ada7)
<Corsair Voyager 1100>             at scbus9 target 0 lun 0 (pass8,da0)


Code:
[root@freenas] ~# glabel status
                                      Name  Status  Components
gptid/34a6ad54-e7fc-11e1-8afc-00045fb309e8     N/A  ada0p2
gptid/338243b8-e7fc-11e1-8afc-00045fb309e8     N/A  ada1p2
gptid/342046fb-e7fc-11e1-8afc-00045fb309e8     N/A  ada2p2
gptid/5270dda3-e7fc-11e1-8afc-00045fb309e8     N/A  ada4p2
gptid/33c3355c-e7fc-11e1-8afc-00045fb309e8     N/A  ada5p2
gptid/7b92c455-2bca-11e2-ad71-00045fb309e8     N/A  ada6p2
gptid/518cbece-e7fc-11e1-8afc-00045fb309e8     N/A  ada7p2
                             ufs/FreeNASs3     N/A  da0s3
                             ufs/FreeNASs4     N/A  da0s4
                            ufs/FreeNASs1a     N/A  da0s1a


Code:
[root@freenas] ~# gpart show
=>        34  2930277101  ada0  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>        34  3907029101  ada1  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  3907029101  ada2  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  2930277101  ada4  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>        34  3907029101  ada5  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

=>        34  3907029101  ada6  GPT  (1.8T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834696     2  freebsd-zfs  (1.8T)
  3907029128           7        - free -  (3.5k)

=>        34  2930277101  ada7  GPT  (1.4T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  2926082703     2  freebsd-zfs  (1.4T)

=>      63  15950529  da0  MBR  (7.6G)
        63   1930257    1  freebsd  [active]  (942M)
   1930320        63       - free -  (31k)
   1930383   1930257    2  freebsd  (942M)
   3860640      3024    3  freebsd  (1.5M)
   3863664     41328    4  freebsd  (20M)
   3904992  12045600       - free -  (5.8G)

=>      0  1930257  da0s1  BSD  (942M)
        0       16         - free -  (8.0k)
       16  1930241      1  !0  (942M)
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,402
You pulled the wrong disk. You offlined gptid/518cbece-e7fc-11e1-8afc-00045fb309e8, which is ada7 not ada3. In pulling ada3 you are missing 2 drives from one of your raidz1 vdevs which is one too many. Replace the correct disk. ;)
 
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
hmmm have to check again my label shows that drive as ADA7 must have labled wrong? lol
 
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
Another question... It seems I went ahead and did the same damn thing again just clicked offline and realized quick it was the wrong one to offline as I am replacing another drive. So I do have the option to repalce the disk with the same one that is already listed but it throws and error
Code:
Jan 11 07:33:19 freenas manage.py: [middleware.exceptions:38] [MiddlewareError: Disk replacement failed: "invalid vdev specification, use '-f' to override the following errors:, /dev/gptid/39c5789e-5c04-11e2-a059-bc5ff4365648 is part of active pool 'Vol0', "]


Code:
[root@freenas] ~# glabel status
                                      Name  Status  Components
gptid/9ebf2708-5b6d-11e2-9efd-bc5ff4365648     N/A  ada0p2
gptid/338243b8-e7fc-11e1-8afc-00045fb309e8     N/A  ada1p2
gptid/342046fb-e7fc-11e1-8afc-00045fb309e8     N/A  ada2p2
gptid/52e8cd24-e7fc-11e1-8afc-00045fb309e8     N/A  ada3p2
gptid/33c3355c-e7fc-11e1-8afc-00045fb309e8     N/A  ada5p2
gptid/7b92c455-2bca-11e2-ad71-00045fb309e8     N/A  ada6p2
gptid/263683e3-5ab5-11e2-a20c-bc5ff4365648     N/A  ada7p2
                             ufs/FreeNASs3     N/A  da0s3
                             ufs/FreeNASs4     N/A  da0s4
                            ufs/FreeNASs1a     N/A  da0s1a
gptid/39c5789e-5c04-11e2-a059-bc5ff4365648     N/A  ada4p2
gptid/39b44479-5c04-11e2-a059-bc5ff4365648     N/A  ada4p1


Code:
[root@freenas] ~# zpool online Vol0 14713318185702787415
warning: device '14713318185702787415' onlined, but remains in faulted state
use 'zpool replace' to replace devices that are no longer present


So does this mean that drive just failed? All I did was click the offline button lol :(

Thanks everyone for helping out this NUB I am learning lots though.
 
C

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Every time you offline a disk from the GUI you need to do a scrub to ensure complete redundancy of the zpool.

Edit: I mean if you offline the wrong disk and then try to put it back in the zpool.
 
M

Miniwehats

Explorer
Joined
Feb 29, 2012
Messages
70
Well yes I have been punching my self most of the morning. So let me understand this correct. If I offline a drive do I need to Scrub the VOL BEFORE i put a new drive in? Or just after it is done reslivering? Also if I offline the incorrect drive do I need to scrub first before attempting to online the drive?
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "replacing drive with cold spare"

Similar threads

Logikgear
  • Locked
Hot Spare vs. Cold Spare
Replies
8
Views
10K
Arwen
Arwen
G
  • Locked
Adding Two Spare Drives
Replies
16
Views
3K
Grenik
G
StephenFry
  • Locked
Replaced disk, pool still says 'replacing' and is degraded
Replies
12
Views
4K
sivabalan
sivabalan
mattlach
  • Locked
Best Way to Upgrade from 6 Drive ZFS to 8 Drive without additional backup?
Replies
7
Views
3K
mattlach
mattlach
D
  • Locked
SOLVED Replacing 'failed' drive
Replies
5
Views
25K
ProtoSD
P
Top