Bad DVA Cannot Import Pool into TrueNAS

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I am having trouble with our NAS, A couple days ago everything seemed to be running fine until I discovered our FreeNAS box was still powered on but not accessible by any web interface or ssh, or anything. I restarted the box, everything seemed fine, checked the SMART status on all disks, all showed OK, except for one disk. I was going to attempt swapping the disk with a spare (RAID 10 with 4 drives), but before I could do anything, the system booted me off the SSH session and was inaccessible.

I attempted to restart the system, but since it was completely unresponsive, I was forced to use the reset button. After which, the system would not even boot. I plugged the system into a monitor, and keyboard, and was presented with an error BAD DVA.

I read a few forum posts, many said to try to import the pool using the latest version of TrueNAS, and rule out any hardware issues. I had a gut instinct that my nvme ssd may have gone bad, so I replaced it with a Seagate 2TB Barracuda, and installed TrueNAS...

I attempted to import my pool, it was detected properly, and TrueNAS attempted to import the Pool... however, once again, the system locked up, and is currently inaccessible, despite being powered on, and a bit of activity is visible on my 4 8TB disks...

I am out of ideas guys... I don't know much about TrueNAS, and really think I should have setup a NAS with hardware/software that I understand better. I was sold on the idea that FreeNAS + ZFS was the way to go, but I am totally lost on what to do next.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
How was your pool constructed? You mention a 4-drive pool, configured as 2 stripes of 2 mirrors. What was the role of the SSD? Try putting the SSD back.
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
How was your pool constructed? You mention a 4-drive pool, configured as 2 stripes of 2 mirrors. What was the role of the SSD? Try putting the SSD back.
SSD was the boot disk for freeNAS. I pulled the SSD out, and installed a Seagate Barracuda 2TB as the new boot disk. And I believe that is correct 2 stripes of 2 mirrors (aka RAID 1+0 / RAID 10).
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I just did a reinstall of TrueNAS, so I can attempt the zpool import again...

This time I executed the command with a monitor and keyboard instead of using the GUI, and after I used # zpool import -f myPool

I saw a KBD Panic in the details, and the system immediately restarted.

I also was able to view my pool using zpool import, and everything showed HEALTHY... So, I am not sure where to go from here...
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
So is your system back up?
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I was able to get the system powered back up, so I can type in commands, but I still cannot import my pool. Whenever I attempt, the system restarts, and the pool was never added.
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I was searching other threads to see if I could poke around and get more info... I found this documentation:

I found in it that you can sometimes import a degraded pool as "readonly" so this is what I did:
zpool import -o readonly=on myPool

That worked without errors, and I can type zpool status, and I can now see the pool as ONLINE... but how do I access the data in this pool? I don't see it in the web interface or anything. Is there a mount point? If so, I would like to access my data, back everything up, destroy the pool, and create a new one.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Export your pool, and then try importing it again with zpool import -o readonly=on -c /data/zfs/zpool.cache -R /mnt myPool.
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I am able to read my Pool using readonly, and I can now see the data in /myPool EXCEPT when I try to mount an external drive on USB, I am unable to copy the data I need.
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
I am able to read my Pool using readonly, and I can now see the data in /myPool EXCEPT when I try to mount an external drive on USB, I am unable to copy the data I need.
I can import the drive in the web interface, and mount it to /mnt/usb2... but when I try to copy files, over I simply get a message "No space left on device" when this is a new device at 4tb in size.
 

trillobite

Dabbler
Joined
Dec 6, 2021
Messages
10
Ok, I figured out a method to get my data off of the pool... I imported the external 4TB drive, and created a single drive "pool..." Then I went to the command line and copied my files from the readonly damaged pool to the new one this way:

cp -r /myPool/dirName /mnt/backup

After that, I created an SMB share on the backup pool... now I can see all of my data, and everything copied correctly.

Now, I can destroy the old damaged pool, and create a new one with replacement drives (if needed).
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
I can import the drive in the web interface, and mount it to /mnt/usb2... but when I try to copy files, over I simply get a message "No space left on device" when this is a new device at 4tb in size.

Yes, the drive import feature mounts the drive read-only for copying to your pool.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Now, I can destroy the old damaged pool, and create a new one with replacement drives (if needed).

Before you create your new pool, please run burn in tests on all 4. You'll likely find the bad drive that resulted in the bad DVA messages.

 
Top