Properly Replace a Dead Disk from CLI

[T14]

My FreeNAS box has 6x2TB drives in it, all configured as a RAIDZ volume.
When a disk dies, I know how to replace it from the web interface, but I'm having a bit of trouble finding the proper way to do it through CLI.

It seems that a lot of people are doing this from the CLI to replace a dead drive:

Code:

zpool replace test_vol /dev/gptid/5eb4c0a6-d556-11e3-9741-000c292f28a4 /dev/da2

The problem with that is:

1. After it completes resilvering, the output of 'zpool status' shows gptid labels for the old drives, and a GEOM label for the new drive:

Code:

freenas# zpool status
  pool: test_vol
state: ONLINE
  scan: resilvered 941G in 10h4m with 0 errors on Sun May  4 02:04:08 2014
config:
 
    NAME                                            STATE    READ WRITE CKSUM
    blackhole                                      ONLINE      0    0    0
      raidz1-0                                      ONLINE      0    0    0
        gptid/8a4fffb2-33bc-11e3-957d-000000000000  ONLINE      0    0    0
        gptid/8b7af956-33bc-11e3-957d-000000000000  ONLINE      0    0    0
        gptid/8c46b5b8-33bc-11e3-957d-000000000000  ONLINE      0    0    0
        da2                                        ONLINE      0    0    0
        gptid/8e26eecc-33bc-11e3-957d-000000000000  ONLINE      0    0    0
        gptid/8edfc5cb-33bc-11e3-957d-000000000000  ONLINE      0    0    0

2. Much worse, the new drive does not appear to have a GPT partition table, or any GPT partitions, unlike the other drives:

Code:

freenas# gpart show da2       
gpart: No such geom: da2.
 
freenas# gpart show da1 da3 da4
=>        34  3907029101  da1  GPT  (1.8T)
          34          94      - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834696    2  freebsd-zfs  (1.8T)
  3907029128          7      - free -  (3.5k)
 
=>        34  3907029101  da3  GPT  (1.8T)
          34          94      - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834696    2  freebsd-zfs  (1.8T)
  3907029128          7      - free -  (3.5k)
 
=>        34  3907029101  da4  GPT  (1.8T)
          34          94      - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834696    2  freebsd-zfs  (1.8T)
  3907029128          7      - free -  (3.5k)

I know that it's possible to manually use gpart to create the proper partitions on the new drive like so:

Code:

gpart create -s gpt /dev/da2
gpart add -t freebsd-swap -s 2G da2
gpart add -t freebsd-zfs da2

and then do this:

Code:

zpool replace test_vol /dev/gptid/5eb4c0a6-d556-11e3-9741-000c292f28a4 /dev/gptid/5eb4c0a6-d556-11e3-9741-000c292f28b5

But that involves manually creating partitions on the new drive. While it seems fairly simple, the lines above don't take into account partition alignment at all, and also might be causing other problems I don't know about. Is there any way to tell FreeNAS from the CLI to partition a new disk like it normally would (give it a GPT partition table, align the partitions, create a swap partition based on the "Swap Size" setting in FreeNAS's web interface, etc)? Or is doing it manually, like I did above, the only way?

Also, if the way I've seen many people doing it:

Code:

zpool replace test_vol /dev/gptid/5eb4c0a6-d556-11e3-9741-000c292f28a4 /dev/da2

isn't creating a GPT partition table, what is it doing? Is it bad to do it that way?

 

[Yatti420]

Why not just replace via the GUI? Their are commands posted on the forum regarding this but the manual recommends GUI.. http://doc.freenas.org/index.php/Volumes#Replacing_a_Failed_Drive

 

[T14]

Why not just replace via the GUI? Their are commands posted on the forum regarding this but the manual recommends GUI.. http://doc.freenas.org/index.php/Volumes#Replacing_a_Failed_Drive

I don't want this thread to become a discussion of the advantages and disadvantages of using the GUI/CLI, but these are my reasons:

1. Curiosity
2. The terminal is where the action happens. The web interface is the simplified version that, while making the system more accessible to people, often hides what's actually going on behind the scenes.

Sometimes it's useful to know how to change the oil in your car, and why, and not just which medal to step on to make it go.

I'm sure the FreeNAS project exists because someone wasn't satisfied with simply using FreeBSD as it was intended, learned how it worked under the hood, and made it do what they wanted. I may not have the time/patience to learn how FreeBSD/FreeNAS works down to that level, but I can't see it being a bad thing to know a few things about what it does behind the scenes.

 

[cyberjock]

T14,

There's threads that discuss how to do the change from the CLI. The problem is that it has changed in the past, and may change at any point in the future, without warning.

You may not see it as a bad thing to know a few things, but the reality of it is that doing it from the CLI can be quite destructive because you may make the assumption you know best and then do something that turns out to be the cause for data loss. I've seen it many times on the forums.

In short, if you want to know how stuff is being done from the CLI, the source code for FreeNAS is available. If the WebGUI doesn't tickle your fancy enough, consider going to full fledged FreeBSD. The whole point of FreeNAS is to avoid the CLI wherever possible, not embrace it.

I do understand your thirst for knowledge, but it's an extremely slippery slope. Even when I'm helping people with completely fubared designs and I'm doing things I'd never admit to in the forums, I always end up treating the system like a FreeBSD machine. I may or may not be able to get your data back, but whatever pool I may be working on will never be suitable for FreeNAS again.

 

[T14]

I get what you're saying cyberjock.
I guess my choices are:

1. Stick with the GUI
2. Use the CLI at my own risk
3. Follow the FreeNAS project closely, reading developer notes, hanging out on developer IRC channels (if there are any) and sign up to a mail list (if there is one) so that I am aware of the proper way that FreeNAS interacts with the FreeBSD/ZFS tools, and changes to those ways introduced in future versions
4. As you said, use FreeBSD with ZFS, and do whatever I want with that

I guess I was hoping there was some script that FreeNAS runs that sets up the drives that was easily accessible through the CLI that would change whenever the procedure changes. I guess there isn't one.
I might dive deeper into FreeBSD one of these days, but for now I guess I'll stick to the GUI way of doing it.

Thanks

 

[cyberjock]

Well, regarding #3, the best way to find out is to read the code. If you ever ask how to do it from the CLI, you're going to get an answer just like I gave you. It may be more pointed and be more like "stfu and use the GUI or get off our lawn". One way to really anger the more experienced users is to ask about doing thinks like you are asking from the CLI. We take the stance that 'we got a manual.. there's no reason not to do it the right way, so why are you wasting our time telling how else to do it?'

You aren't the first to ask these questions, and you won't be the last. But we're so tired of fielding them the typical answer is 'the source code is available.. have a ball'.

 

[T14]

Maybe one of these days I will check out the code - thanks.
BTW for anyone who actually did use the following way to recover from a dead drive:

Code:

zpool replace test_vol /dev/gptid/5eb4c0a6-d556-11e3-9741-000c292f28a4 /dev/da2

and now has the new drive show up as a GEOM name in the output of 'zpool status', and 'gpart show /dev/daX' not showing a proper GPT partition table on the new drive, the fix is pretty simple, but does require a resilver so it will take a while:

Use the GUI to replace the disk with itself:

Code:

Login to Web GUI
Go to: Storage --> Volumes --> View Volumes
  Highlight the Volume (NOT A DATASET)
  Hit "Volume Status"
  You should see one that's different than the rest (mine said daX when all the others said daXp2)
  Highlight it
  Hit "Offline" to take it offline
  Hit "Replace"
  Do an inplace replace

As always, you can use 'zpool status' to see the progress of the resilver