Panic when I try to import an encrypted pool

[NewtonRob]

I have an encrypted pool with 4, 4TB Western Digital Red drives. My system sees the drives. I can manually decrypt them. But when I try to import them my system panics. I have reinstalled FreeNAS, I have pulled the disks out and used a different system, but still have the same results. So I don't think it is a Hardware issue.

How this started. One day my NAS rebooted and the pool was locked. There was no way to unlock it. I have tried zpool import -F option from the command line. System panics.

Any suggestions? I'm even considering trying to import it with just 3 of the disk. I really don't want to loose the data, so I would really like some advice.

After the panic, the system reboots, but he dashboard is empty. If I look at Storage --> Pools, I see the pool, but it is locked. No way to unlock it.

Newton

 

[NewtonRob]

An update. I was able to mount the pool read only.
First attached the drives
geli attach -p -k file.key /dev/da2p2
Note. -p so it does not ask for a key phrase. I did this for all four drives.
Then I mounted the pool
zpool import -o readonly=on -o altroot=/mnt Red1

It mounts Red1 and I can access the files. This is GOOD!

Next problem, I can't share Red1/Plex because FreeNAS GUI says it is not a volume. I get "The path must reside within a volume mount point".

I have tried several options, but so far I can't get any of them to work.

1. I tried to connect a WD 10TB USB external drive to the FreeNAS box. I could see the device, but could not mount it. I basically tried all the fstypes including msdosfs, ntfs, etc. I also tried from the GUI and got errors.
2. Unable to SMB share from FreeNAS
3. I connected the 10TB drive to my Windows system. I shared it. I tried mount_smbfs -N -I <ip> //<user>*<computer>/FreeNAS /mnt/Windows. I got connection errors. Some variations it asked for a password, but still errored out.

Any other ideas?

 

[Samuel Tai]

How to: Backup to local disks

How to: Backups to local disks Overview: This subject comes up often enough that I thought I would start a post of how I do it. Specifically I back up my TrueNAS to a hot-inserted, locally attached disk, then when done, hot-remove and take...

www.ixsystems.com

 

[NewtonRob]

I put my NAS back together. I mounted another 8, 1TB disks to copy the data. Mounted the problematic pool as read-only. Everything looked good.

I started a copy, and after a minute or two the system panicked and rebooted.

Also above. The command to mount the pool as read-only was missing a -f.
zpool import -f -o readonly=on -o altroot=/mnt Red1

I must have errors on the disk. How can I fixed them? I doubt it can be done when mounted as read-only.

Can I identify a problematic disk? Then just mount the other 3?

Need some advice.


Thank you Samuel. I will keep your article in mind.

 

[CHL]

Ok, old thread, but still not solved.

Same issue here. Had it a long time ago on a testing machine, but today again in the production environment. Encrypted pool with 4 drives on 11.3-U5. This night, the system just rebooted without leaving any traces in the logs. Ok, maybe a power issue, but there are some other systems in the same room that did not reboot. Strange...

So I entered the key-pass and FreeNAS started to encrypt the disks. All fine, but when importing the pool, the system just panics. Gone back to 11.3-U4.1 - same thing. Updated to TrueNAS 12.0-U1 and voila: I could import the pool.

But now the question: What was/is going on here? I don't think it's only a hardware issue and all disks are fine. Maybe some kind of compatibility problem. Where to start searching for any hints? I can not sleep very well, because there are some other machines with the same setup up and running...

 

[SweetAndLow]

Ok, old thread, but still not solved.

Same issue here. Had it a long time ago on a testing machine, but today again in the production environment. Encrypted pool with 4 drives on 11.3-U5. This night, the system just rebooted without leaving any traces in the logs. Ok, maybe a power issue, but there are some other systems in the same room that did not reboot. Strange...

So I entered the key-pass and FreeNAS started to encrypt the disks. All fine, but when importing the pool, the system just panics. Gone back to 11.3-U4.1 - same thing. Updated to TrueNAS 12.0-U1 and voila: I could import the pool.

But now the question: What was/is going on here? It is not a hardware issue and all disks are fine. Where to start searching for any hints? I can not sleep very well, because there are some other machines with the same setup up and running...

Start by creating a new thread for your problem. Provide hardware specs and smart output for your drives.

 

[NewtonRob]

Ok, This will be a long explaination, but it might be useful for others.

I have an Intel 4790K (excellent chip!) with 32GB RAM running on a Gigabyte Ultra Durable GA-Z97X-UD7 TH motherboard. I also have 8x mix of WD blue/green, 4x WD Red, Intel 80GB SSD, and a PNY 120GB SSD.

Problem: System panic and reboot. Volumes could be manually mounted but copy caused panic and reboot.

Encrypted volumes not recoverable even under UNIX. Basically, because they were encrypted. I reinstalled FreeNAS. I was able to manually mount but panicked when ever I tried to copy the data off. I wiped the disks and lost the data. Most was backed up, so not a terrible loss. Very disappointed as everything I read said that ZFS was practically indestructible. I saw demo of pulling power from server, pulling power from drive. All were recovered with no issues.

I loaded UBUNTU and setup a RAID. Everything was working fine, then the system hung up.

What I did to fix the problem:

• Tested the CPU under load for hours.
• Test RAM for hours. Also tried with one set of RAM then the other.
• Bought SAS cards and Disabled SATA on the motherboard. Great addition, but did not solve the problem
• Bought external Network Adapter. Did not solve the problem.
• Bought a kick-ass 1000W Platinum power supply. Good for bragging rights. Did not solve the problem.
• I have 3 4-bay hot swappable drive bays in the case. I pulled them out and tested drives without them. Still had the problem.
• Wiped the disks and ran tests on all of them. NO ISSUES AT ALL!
• Bought a GeForce GT 710 Video card with 4 HDMI ports. Claimed to be good with Linux. I had seen some problems with built in graphics and decided to give it a try.
• During this process I reverted FreeNAS to earlier version, loaded on a new SSD, and tried everything I could find in the Internet concerning OS and software.

None of these addressed the problem. At this point I had turned every built-in capability of the motherboard off. I still had the problem. I had given up on FreeNAS and ZFS. I had switched to Ubuntu and software RAID. If I hadn’t started having problems I would have never worked with FreeNAS/TruNAS again.

At this point I was like, the only thing I have not changed or tested was the case!

With a shrug of my shoulders and a what the hell, I went into the BIOS and underclocked my CPU to 3GHz, which also underclocked my RAM.

EVERYTHING WOKRED! I have now loaded TureNAS 12 and applied 12U1. Everything is working. I am running Plex and an Ubuntu VM. Under the MV I’m running docker. Everything is back up and running without a problem. It is either the CPU, RAM, or Motherboard. But these parts are still expensive and I'm not going any deeper in this rabbit hole.

I’m disappointed that I’m not able to run the CPU at the published 4GHz specifications. But 3GHz is more than enough for this NAS. I am very happy to be running TrueNAS. I had given up on it. So, what should you do first. If possible underclock your system. Could save you a lot of money and time. If that doesn’t work, then you have a journey ahead of you. I have a case full of very cool stuff, but the project was to build a cheap kick-ass NAS for almost nothing. I even traded old computer parts for some of the drives. In the end, it cost me much more than I wanted. But to be very clear, still much cheaper then any 12bay NAS.

One last thought. Ask yourself if you really need encryption? Is this personnel and on your home network? I have decided that I do not need encryption. If you don't, there may be more options for data recover if you do run into problems.