Replacing a failed drive. No 'Replace' button in GUI

[Phlod]

Today when I added a drive to my FreeNAS, everything worked for a while, and then it just disappeared. Trying to reboot after this would just hang at Mounting local filesystems. Removing the drive allows me to boot all the way, but now my Volume is Unavailable. The GUI, inexplicably has no 'Replace' button anywhere. In Drives I have Edit, and Wipe, and that's it. Nothing is quite as it shows in the manual, and now I'm getting frustrated. I'll admit to being a newb at FreeNAs, but I know my way around the command line.

Thank you in advance!

--Phlod

 

[Stephens]

More detail might help others help you. What kind of array did you start with? You said you "added a drive" to your FreeNAS. Do you really mean you had a failed drive, so you replaced it with a new one? What steps did you follow to do this replacement? If you go to the shell, what does "dmesg" show? What does "zpool status" show? What version of FreeNAS are you running (the whole string)? The more detail you can provide, the better the experienced guys here can help you.

 

[William Grzybowski]

The replace button would be in Volume Status screen... not View Disks screen...

 

[Phlod]

More detail might help others help you. What kind of array did you start with? You said you "added a drive" to your FreeNAS. Do you really mean you had a failed drive, so you replaced it with a new one? What steps did you follow to do this replacement? If you go to the shell, what does "dmesg" show? What does "zpool status" show? What version of FreeNAS are you running (the whole string)? The more detail you can provide, the better the experienced guys here can help you.

I apologize for my lack of information. I was very tired last night, and had been at this problem for at least 12 hours. Anyway, I don't know what kind of array I started with to be honest. Actually, let me start with some background. I'm running this on a quad core Xenon server, on Windows Server 2008r2, and using VMWare Player. FreeNAS-8.2.0-RELEASE-p1-x64 (r11950)

I have been amassing drives for years now, and had been keeping them as NTFS, but I hated having multiple drives with multiple filesystems for organizing my media. My Media server died recently so I decided to move all the drives to a FreeNAS. I had been copying drive contents to the NAS, and then wiping and adding the drive to the NAS. All was going swimmingly until the drive I just added (3rd total drive, second non-virtual drive) to the NAS failed. I mean it disappeared from Windows as well as VMWare barfing. After that FreeNAS refuses to boot past Mounting local file systems, and eventually VMWare complains that the operation on the Physical Drive failed.

Since I couldn't get the system to boot at all, I removed the failed drive, but now Nothing Works... Sorry. Being hyperbolic. The Virtual machine will boot without the drive, but the FreeNAS GUI shows a status of Unknown, and zpool status says no pools available. When I do zpool import -f I get:

pool: Storage
id: 12192648677279661521
state: UNAVAIL
status: One or more drives are missing from the system
action: The pool cannot be imported. Attach the missing devices and try again.

I have only the one Volume -- "Storage". And I can't seen any way to remove the failed drive in the Volume section of the GUI. That's as far as I've gotten so far. I have installed a new hard drive to replace the failed one, but I haven't added it to the FreeNAS virtual machine yet. If you need any more info, just let me know.

--Phlod

 

[cyberjock]

Did you have a RAID1 or RAIDZ zpool on FreeNAS with the different logical vmware drives on different physical drives?

If you didn't it appears that one of your drives have been corrupted and you have lost your zpool because of the corrupted drive.

 

[Phlod]

Did you have a RAID1 or RAIDZ zpool on FreeNAS with the different logical vmware drives on different physical drives?

If you didn't it appears that one of your drives have been corrupted and you have lost your zpool because of the corrupted drive.

Honestly, I wish I knew what type of zpool I had. I don't remember FreeNAS asking during setup, and all I did was give it drives and have it format them ZFS and extend the volume. I actually tried to figure this out before anything bad happened, but I couldn't find anywhere that would tell me.

Please tell me, by the by, that I haven't just hosed what data I *did* move onto the NAS. Is there anything to be done?

--Phlod

 

[cyberjock]

My thought is there is nothing you can do unless you can fix your zpool by restoring the drive. This is why it's imperative that you have a good plan if you try to use FreeNAS on a virtual machine to store important data. It's great for experimenting, but I'd never trust it to store data.

 

[Phlod]

Wow! A disk fails and I lose 1.5TB of stuff on the drives that didn't fail?

You're absolutely right, I should have had a better plan, one that stayed the hell away from ZFS and FreeNAS...

 

[survive]

Hi Phlod,

Allow me to be blunt....

I wouldn't be so quick to blame FreeNAS or ZFS for this problem you created.

The problem you are having right now is because you were adding individual drives (single drive virtual devices) to your pool so you had no redundancy (at all) to protect your data when you did lose a drive....you were essentially growing what is analogous to a RAID-0 volume in the hardware RAID world. This is a perfectly valid configuration if you are aware of the risks and have takes steps (backup) to manage that risk.

Ignoring the choice of operating environment to focus strictly on the storage I have to ask what exactly did you think would happen when you did (inevitably) lose a drive? What did you think would happen to your data that was on the other drives? That every third file would go missing?

Before you make another attempt at making yourself a filer I would encourage you to take some time and educate yourself on what RAID is, how it works and what the various risks & benefits the many options you have available offer you. A little time and planning of the front-end will save you hours of work and heartache when fate moves to expose any weaknesses in whatever you decide.

If you want to give FreeNAS another whirl there are many users available here and on the #freenas (IRC channel) who are more than willing to offer advice and help prior to implementing your chosen solution. They can certainly assist in pointing out any weakness that your plan might have and how to overcome them.

-Will

 

[cyberjock]

Wow! A disk fails and I lose 1.5TB of stuff on the drives that didn't fail?

You're absolutely right, I should have had a better plan, one that stayed the hell away from ZFS and FreeNAS...

If you look at the guide in my signature you will see that what you did is exactly what I showed in slides 21 and 22. The reason why I put that example in the guide is because that is a mistake alot of noobs make and then they get upset and blame FreeNAS. It's not FreeNAS's fault you didn't fully understand what you were doing. I added that example to hopefully stop people from making that mistake and losing data.

What's different from FreeNAS is that it is very powerful, but it fully expects you to know what you are doing.

 

[Phlod]

You are reading a lot into a very short post. No blame was laid at the feet of FreeNAS nor ZFS. It is in all ways a true statement to say that I should have had a better plan, and I should have stayed away from FreeNAS for doing what I wanted to do. In truth there probably isn't something that will do what I wanted. Live and learn.

However, losing terabytes of data is a traumatic thing... You might not want to descend quite so jackal-like when you *think* someone has besmirched your pet project. When I first read your responses I just saw red.

noobsauce80: Please try to do something to get your guide to actually show up on Google searches about FreeNAS. It sucks to find out you could have avoided a huge data loss by reading a guide that feels hidden away in some remote corner of the internet. I let the glowing reviews from people who did the same thing as I did (minus the failing drive), and ignored that they seemed to not have a plan. I researched for hours trying to find what the down sides were to my plan, all that I came up with was people with not much technical knowledge putting together FreeNAS servers with relative ease.

Not saying FreeNAS doesn't deserve the glowing reviews either. It worked great until it didn't.

Anyway, for what it's worth, thanks again.

--Phlod

 

[cyberjock]

I have lost TB of data in the past. I've learned NEVER to do something until you are willing to stake your data on it. After all, you are doing exactly that. I took some old hardware and built a few ZFS pools and did all the bad things to them that you'd expect from normal operation as well as admin errors. Until I was comfortable that I was able to recover from them ALL or understand the limitation thoroughly I didn't migrate to FreeNAS.

I'm sure you saw red. Who loses their data and DOESN'T see red if someone says anything like "it was your own mistake". It still sucks just as much as before but you've learned something. The waste would be to NOT learn from this and just go back to what you used to do to store data. It is almost certainly less reliable(especially if it's Windows based). You've learned a lesson the hardest way possible. The best thing to do is get your FreeNAS server back up and...

DO BACKUPS OF YOUR MOST IMPORTANT DATA.

I have alot of data, but a 2TB drive has a backup of all of my data and is kept off-site so if my house burns down I'll at least have my most important files.