Strange drive issues after upgrading.

[Bret Whittacre]

I ran the upgrade that was on the interface for FreeNAS. I made a serious mistake, and am hoping that someone can help me with my issues.

First, I have three drives in the system.
ada0 = 279 gigs
ada1 = 3.7 TB
ada2 = 3.7 TB
I had them set up so that ada0 was the system drive, and ada1 and ada2 where part of a zpool that would give me almost 8 TB.

After reboot, the only drive that show as available in the volumes GUI is ada2. I was looking around to see where ada1 went, and it looks like it somehow was attached to the boot. If I look at system -> boot -> Status It shows that there are two drives there ada0 and ada1 and that they are mirrored. This is not what I had before the upgrade.

Also, if I look at the volume that is there (media) under volume status, it shows it as a striped volume as ada2p2

So my question is this. How can I get ada1 back into the disk group so that I can see the data that was on the drive? I would really like to have it back.

Thanks in advance for any help that you may offer!

 

[Robert Trevellyan]

I'll warn you now that this looks bad, but to have any chance of receiving tangible help you'll need to provide much more detail about your system. Take a look at the forum rules for what's expected. In the meantime, a couple of basic observations and questions.

I ran the upgrade that was on the interface for FreeNAS

From exactly which version to exactly which version?

ada1 and ada2 where part of a zpool that would give me almost 8 TB

That was a serious mistake.

I was looking around to see where ada1 went, and it looks like it somehow was attached to the boot.

I don't know how that could possibly be the outcome of a simple upgrade. Did you do anything else around that time, e.g. running some kind of storage wizard, or using the volume manager?

How can I get ada1 back into the disk group so that I can see the data that was on the drive?

You probably can't and will almost certainly have to rebuild the pool and restore the data from backup.

 

[Bret Whittacre]

Robert..fair enough. I will try to provide more information. For starters, I believe that the old version was a FreeNAS stable 9.2. I am not completly sure on the exact version though. The new version is FreeNAS-9.3-STABLE-201412090314 . The system is a custom built box that has three drives in it. Two are large drives, and the third was for the os. I have attached a screen shot of gpart zdb and spool status. I am not sure about what kind of ram the system has, but there is two gigs. I attached a screen shot to help on that front also.

Also, the system has been up and running for over 8 months with no issues till today.

I realize that having two drives in one pool could be considered a mistake. I was shooting for more space, and it did work for some time.
About one hour before the system upgrade, I installed the Bacula system. Other than that, I did not alter any of the volumes or touch anything in the volume manager at all. That is what is driving me nutz. I also don't get how the system decided to put the other drive with the boot. I am attaching several screen shots to see if that can help.

Do you know of any way that I can remove it from the system boot? Since I will most likely loose everything, I am willing to try any command line options that come to mind.

 

[rogerh]

You know FreeNAS 9.3 is not supposed to work properly with less than 8GB of RAM? From the above it looks as though your data could be intact on ada2, is there any data in /mnt/Media/? The only way I can see that you should get the mirrored boot drives and the single disk pool is if you installed FreeNAS 9.3 rather than updated, and then set up a new zfs pool on ada2.

 

[Bret Whittacre]

ada2 did mount under /mnt/Media, but nothing is there. I believe that all the data is on ada1. I did not do a reinstall it was a upgrade, but I agree that it looks like it did a reinstall.
Do you know how to remove ada1 from the boot group, and just mount it so that I can look to see if there is any data on it? I am not sure of the commands to do this. Also, where does zfs store all the information about mount points etc.

 

[rogerh]

ada2 did mount under /mnt/Media, but nothing is there. I believe that all the data is on ada1. I did not do a reinstall it was a upgrade, but I agree that it looks like it did a reinstall.
Do you know how to remove ada1 from the boot group, and just mount it so that I can look to see if there is any data on it? I am not sure of the commands to do this. Also, where does zfs store all the information about mount points etc.

I expect it is fairly easy to remove ada1 from the boot mirror. I can't tell you how, but there are some threads on this forum discussing this. Or you could read man zfs. However, ada1 will have been repartitioned and will have the OS on it. I really doubt if anyone but a professional recovery firm could get anything from it, and they would probably need ada2 as well if they were striped for any length of time. See what the experts here think.

 

[Robert Trevellyan]

Did you previously have two storage volumes, one called Media and one called something else? If so, my best guess is that you had two single-disk volumes, aka stripes, rather than one two-disk volume, and somehow you lost one to the mirrored boot volume. I have no idea how that happened, but if you post a debug file we might be able to figure it out.

What are you doing that tells you there's nothing on Media now? What do you see in the Volume Manager?

If you did not previously have a storage volume called Media, then ignore all the above, because I would have no credible explanation for what you're seeing.

As noted above, 2GB of RAM is one quarter of the minimum required for FreeNAS 9.3. The number of months that the system previously ran without issues is irrelevant to this fact.

I also concur that splitting the boot volume mirror isn't going to reveal the data you used to have on the other large drive.

 

[Robert Trevellyan]

FreeNAS-9.3-STABLE-201412090314

Is there a reason why you didn't upgrade to the latest 9.3-STABLE?

 

[SweetAndLow]

Is there a reason why you didn't upgrade to the latest 9.3-STABLE?

I agree with this question. There are way to many strange things that can't happen that the op is explaining.

During upgrade you can't just overwrite a zfs disk. There is code the explicitly stops this from happening.

With the little amount of ram you are running in sure you have never read any documentation. So the way you thought stuff worked was probably incorrect.

 

[Bret Whittacre]

I know that the system is lacking in RAM. I also did not read any of the upgrade documentation. If I would have, I would have backed up the system. Kinda hard when there is 8TB to back up.
I believe that the system was a strip drive. Also, I hit the upgrade button that took it to the latest stable version.

I find all of the comments very interesting. As I read them again, I can see that people want to blame the lack of ram as the issue. I know that the system lacks ram, I also knew that when the original install happened. The issue has little to do with the lack of ram. It has to do with the way that file system was moved around, and how to move it back. I am not a zfs expert, it shows, so you don't have to fry me. I was just hoping that someone might know some commands that can move things around. I have been able to do that for years in Linux and ext3 without any issues. Since it looks like I am going to loose everything, I am going to try to set the box up from scratch, and just bring the drives back in the way that they were. I will post the results latter.

 

[Robert Trevellyan]

I can see that people want to blame the lack of ram as the issue

I don't think it's that as much as it is just a symptom of a lack of proper setup. It might indicate that FreeNAS, which requires a lot of study, not to mention outlay, to do right, may not be the right solution for you.

I'm still curious to see a debug file and answers to the other questions. There's just a chance that things aren't quite as bad as they appear.

I am going to try to set the box up from scratch, and just bring the drives back in the way that they were

If you're starting over, at the very least put the two 4TB drives in a mirror. Better would be to get a 3rd and create a RAIDZ1 vdev. A stripe is the worst of all possible configurations unless performance and capacity are more important than the actual data (this can be valid with a suitable backup strategy). RAIDZ1 is far from ideal for large drives, but nowhere near as bad as a stripe. In any case, things won't end well if you don't upgrade your RAM to at least the minimum requirement of 8GB.

 

[Bret Whittacre]

Robert, I am going to follow your advise. New hard it is. The old stuff is ...well old. I did try to do a reinstall, so I don't believe that I will be able to get you a debug file. I am going to get one more drive and then upgrade the ram. :) Thanks for all the posts.

 

[Robert Trevellyan]

New hard it is

You'll be glad you did.

In addition to the general hardware recommendations, many people with modest needs do just fine with an off-the-shelf entry-level server and a suitable RAM upgrade. Options include the Dell PowerEdge T20 and T110ii, and the Lenovo TS140 and TS440, all of which are available at great prices if you shop around. Check my signature for what I run ;)

 

[cyberjock]

What is the output of "zpool history freenas-boot"?