Adding Two Spare Drives

[Grenik]

I am trying to add two spare drives. I had posted about it, but was trying to figure it out while waiting for an answer. I think I caused more harm than good. (I replaced the old post with this one)

I was trying to add one drive at a time and somehow ended up adding a (live?) drive to the pool.



I try to take ada10p2 off line and get the following:

"cannot offline ada10p2.nop: no valid replicas"

How can I get this out of my pool?

Thank you in advance

 

[Grenik]

I am guessing from the lack of responses that it is not possible to remove this drive from the pool? Later today I will start over and reset everything back up. All the data is (obviously) backed up elsewhere, it is just a pain to have to go back through it and it takes a couple days to copy it all back over.

Just thought I would ask one more time before heading down that path if there is a way to get back to the two RAID-Z2 system I had before I accidently added the other drive.

Thank you.

 

[bollar]

You have ada10p2 and ada11p2 sitting together as striped? It doesn't look to me like they're part of the pool. What does the Zpool (storage: active volumes) screen look like? Perhaps they're listed as a dataset called spares and you can safely delete them there.

 

[Grenik]

I had originally had that setup, but I was concerned that it was stripe, so I was playing around trying to add one spare at a time. (I have deleted that picture from the post above.)

The attachment shows my current configuration, and it looks like I accidently moved one from spare to being in the pool. My storage capacity is about 2T higher than it was before. I try to take the drive offline and get the error:

"cannot offline ada10p2.nop: no valid replicas"

Thank you.

 

[cyberjock]

If you read my guide you'll see what you did wrong. Link is in my sig.

 

[Grenik]

I understand that it was wrong (I have have read the guide a couple times), the question I have is there a way to undo it without starting over. I have not written any new data to the device since I did it, can I somehow remove the disk I accidently added?

 

[bollar]

Ah, I see the attachment. No, there's no way to undo that that I know of.

 

[Grenik]

Thank you Bollar for taking the time to respond. I will restart it and reload everything back up.

I do find it odd that I get a big red screen asking if I want to reboot, but no warning when I am accidently about to destroy the redundancy of my system. Ultimately, I was the one that accidently added it to my pool instead of as a spare, but a suggestion would be to consider beefing up the information on adding spare drives and their capabilities. The manual is rather vague on the topic - providing only the buttons to select, but no information on if the drive will automatically be used if a drive fails or how multiple spares might be handled.

Cheers and Happy Holidays

 

[bollar]

Thank you Bollar for taking the time to respond. I will restart it and reload everything back up.

I do find it odd that I get a big red screen asking if I want to reboot, but no warning when I am accidently about to destroy the redundancy of my system. Ultimately, I was the one that accidently added it to my pool instead of as a spare, but a suggestion would be to consider beefing up the information on adding spare drives and their capabilities. The manual is rather vague on the topic - providing only the buttons to select, but no information on if the drive will automatically be used if a drive fails or how multiple spares might be handled.

Cheers and Happy Holidays

I agree and think I'll see if a ticket has already been written on it. It would be ideal if a new user wouldn't have to make such an "undoable" decision without complete information -- and I do think the GUI should give you a "are you sure you want to do this?" warning.

 

[bollar]

Looks like this has been addressed for an upcoming version:
Code: Warn the user about different vdev type or num of disks on volume add
Feature request: warn before adding one single disk to existing pool

 

[cyberjock]

I agree and think I'll see if a ticket has already been written on it. It would be ideal if a new user wouldn't have to make such an "undoable" decision without complete information -- and I do think the GUI should give you a "are you sure you want to do this?" warning.

Uhh.. complete information is YOUR responsibility as the administrator of the server. If you don't know what you are doing no amount of warnings and whatnot will fix that. FreeBSD is extremely unforgiving if you don't know what you are doing. While a GUI fix looks like it is in the works, it just means that people that don't know what they are doing will start making a different big mistake now.

At some point you have to accept that the admin either must have a clue or suffer the consequences. That's why I wrote the guide. Unfortunately, too many people still don't read the stickies and pay for it.

In fact, I can predict what will happen if it gives you the warning when adding one disk. People will start buying 2 disks so they can avoid the warning. No warning = must be okay. Yes, some people are THAT stupid.

 

[bollar]

At some point you have to accept that the admin either must have a clue or suffer the consequences. That's why I wrote the guide. Unfortunately, too many people still don't read the stickies and pay for it.

I hear you, but you're presupposing that new users read the forum and I don't think that's a reasonable assumption. We know that the documentation is not the easiest to understand and it's possible for a competent person to read that section and still misunderstand how to apply it.

But in this case, the OP read your guide and understood it, but still misapplied his knowledge when it came to actually adding the spare into the zpool. The change to the GUI will save the type of user we both care about.

 

[cyberjock]

I hear you, but you're presupposing that new users read the forum and I don't think that's a reasonable assumption. We know that the documentation is not the easiest to understand and it's possible for a competent person to read that section and still misunderstand how to apply it.

I'm not presupposing that new users read the forum. I'm presupposing that the administrator of a FreeBSD/FreeNAS server actually DO THEIR HOMEWORK. The manual is very well written, I understood it fairly well before I had even created the first server. I tested it in a VM machine to verify that the limitations imposed were correct and that what I was trying to do would work. Do not come to this forum wanting to use what quite possibly is the most reliable file system on the planet and then blame the file system for user error. That's like demanding warning stickers on your hard drive bays when removing drives to verify that you aren't removing the wrong drive.

Yes, I acknowledge that I may be smarter than your average human. But I also spent almost a week experimenting with FreeNAS in a VM doing very bad things to it and almost 3 weeks just reading the forums to see what problems people were regularly having so I could avoid all of them. Then I bought hardware(all of which worked the first time because I did my homework). Then I ran more tests once I had a zpool with a few files copied just to see what would happen. I unplugged drives, I tried to add spares, I deliberately wrote zeros to a drive, all sorts of nasty stuff JUST to see what would happen. I kept a lot of notes of things that didn't work like I had thought. I had zero issues with my UPS, bought enough RAM, didn't add drives as stripes, I managed to avoid EVERY recurring issue people had, and I did it for cheap!

My only mistake is my own. When I created the zpool I didn't check the box for the 4k sector size for "future-proofing" when I thought I had. Now I'm days from creating a new zpool and doing a data copy as a consequence of my inattention to detail.

This is NOT Windows. It's not designed to cater to the majority of the planet. This is FreeBSD/FreeNAS. There's a reason why there is no ZFS implementation in Windows.

We have tried to made it completely clear what the consequences of adding a single drive to a zpool is. Yes, the OP made a mistake and knew better. That's why when you are making significant changes to a system you run through the steps in a second system or VM. Then you know EXACTLY what is going on. If you get a funny error then you know you didn't do something wrong and know to ask those with more knowledge.

Do I feel bad for the OP. I truly do. Nothing sucks more than knowing better and still making a mistake. But that's why you do a run through on a second system or VM and observe the results. If you don't get the results you are expecting, then you stop and start asking questions.

If someone handed you a new OS and said "learn it" how long would you study it before you decided to start trusting it with your data. I'd say if you spent less than 2 full weeks with FreeNAS with no prior FreeBSD experience then you are seriously overconfident and very lacking in your knowledge. Unfortunately, there are PLENTY of people in this forum that have a FreeNAS server setup and running in a few days.. and they spend too much time in the forum later and lose far too much data. All of those "Why is my NIC not working", "Why is my system so slow", "Why is my UPS not working", "Why am I missing GB of space" are all clear signs you didn't do your homework. Those people put FreeNAS on a USB, put it in a server and when it didn't "just work" started complaining.

Should you be getting a warning from FreeNAS to do backups every? How about it automatically choosing to do a file system check for you no matter how inopportune a time it is? Do you really want that level of simplicity? If so there's an OS that does that.. it's called "Windows". I hear its used all over the planet and includes a fairly reliable file system.

You better not leave Linux/Windows/whatever you normally use and come to FreeBSD/FreeNAS and expect easy spoonfed answers. The whole key to why Unix is so powerful is its amazing flexibility. There's almost never a question with a "Yes" or "No" answer. There's lots of "yes, kinda...", no, conditionally...", and "yes, but.." and nobody to better tell you how to do what you want than YOU.. with the proper knowledge. In this case the question "Can you add a single drive to a zpool" answers with "Yes, but you will lose redundancy from that single disk failing".

Bottom line, do your homework or go home. PERIOD. FreeBSD/FreeNAS is NOT for everyone. It's not. And I have only seen a small handful of people that came to the forum, asked some questions, then decided that FreeBSD is not for them. I applaud them because at least they know when they are in over their head.

* - Some of these comments are NOT indicative of my opinion of the OP but only related to the comment from bollar.

 

[bollar]

Bottom line, do your homework or go home. PERIOD. FreeBSD/FreeNAS is NOT for everyone. It's not. And I have only seen a small handful of people that came to the forum, asked some questions, then decided that FreeBSD is not for them. I applaud them because at least they know when they are in over their head.

I'm not here to joust with you on this. I appreciate the effort you to through every day in this forum to help people succeed with their FreeNAS systems. The exceptional amount of time it must have taken you write your extremely detailed reply to me reminds me how strongly you feel about this. Fundamentally, I agree with you that FreeNAS is not an appliance and requires advanced skills to setup and administer. Like you, I've created and blown away many zpools as I tried to figure out how FreeNAS implements ZFS -- and like you, I'm about to blow away a production zpool because of the 4K issue.

At the same time, I do think that FreeNAS should warn users that they're about to make an irrevocable mistake -- IMO, this is what a GUI is well suited for and really one of the key reasons one would choose FreeNAS over FreeBSD. And in this case, the CLI command "zpool add tank spare /dev/dax" makes a lot more sense to me than the way it's handled in the GUI, so I do believe there's room for improving the user experience. I don't see making sure the UI/UX is clear to be dumbing down at all.

And as for where to draw the line with this handholding -- I'm also very aware that you can't save people from being stupid. I've seen hundreds of messages on many NAS sites from people who have lost their arrays and don't have backups. We both agree this, and the many other RTM questions you field every day, is the epitome of stupid. There's nothing we can do to save these people from themselves.

 

[bollar]

I understand that it was wrong (I have have read the guide a couple times), the question I have is there a way to undo it without starting over. I have not written any new data to the device since I did it, can I somehow remove the disk I accidently added?

BTW, the one thing you could do to restore redundancy is to mirror ada10p2 with an additional drive. This may create performance issues, but it'll at least protect the array until you're ready to rebuild the zpool.

 

[cyberjock]

There's no way to "undo" it. :( That's specifically why I mention it in my guide. It's a one way ticket to single-point-of-failure.

Edit: Someone recently added a single drive, then before he added any data to the server he shutdown the server, pulled the "stiped" drive and zeroed it. Now he's on page 3 of trying things that aren't likely to work(and he's trying the last option now). He had no backups. I tried to find the thread but I couldn't find it easily.

 

[Grenik]

It clearly was not my intention to start this sort of a back and forth, but since I did, I will point out a couple things.

First, it appears that the "marketing" of FreeNAS is to a large group of "generalist" administraters. Just look at the web page that brings people in. It is not written for the professional administrator, it does not even imply that only that type of people should be using this software.

It is an Open Source project that at least appears to be trying to gain acceptance and recognition for the great work that goes into it. With that goal in mind, the manual sucks. Ifuess it makes sense once you have done it a couple times, failed, restarted, etc, but for a new user, it is barely adequate. It does not have much information about "why", it just provides the minimum information about "how". The guide however that you have in your signature is much, much better. That is what a manual should be like. It teaches some of the "why" along with the "how". It only covers a small part of the setup and operation, but I still found it very valuable.

For a project like this, the setup and operation cannot be so difficult and ambiguous.

As I stated, I accidently added another drive to my pool when trying to add a spare. It was an accident and I posted a suggestion that would make the system better for other users in the future. Noone is asking to make it idiot proof. In my case I have backups off all my data and I am in the process of recovering it. It was a good learning.

I guess my point is that not everyone should have to learn all the lessons themselves or there is no value in a community project like this. Keep making improvements to the manual and the package so that people can learn from the information before they start using it and while they are setting it up.