Pool not importing (I/O error)

[klipsofkool]

Hello everyone.

Recently I have tried to import a pool into my VMware NAS, and it has always been unsuccessful for me:

Additionally, I have tried to forcefully import it using the shell command "zpool import -F KLIPS", to which it still doesn't import.

Is there a way to fix this IO error to successfully import the pool?

If not, is there a way to recover the data on the drives?

Both HDDs are in a RAID 1 configuration, and the pool and drives are both listed as online:

 

[Arwen]

It is helpful to list your exact hardware, how it is wired up, the TrueNAS version AND how VMWare is involved.

Their are specific configurations needed for reliably running TrueNAS as a VM / Virtual Machine. Failure to follow those suggested configurations can lead to data loss, and entire pool loss. But, let us start with the information above first.

 

[klipsofkool]

The hard drives are two Seagate Barracuda Compute 7200RPM 2TB drives. They are connected to the motherboard using SATA.
The CPU is an Intel Xeon E3-1230V3, running with 16GB of ECC ram.
The TrueNAS version is 13.0-U4

VMware is used to virtualise the TrueNAS environment.

The pool is contained in two VMDK virtual disks, running in a RAID 1 configuration:

I hope this is enough information to determine the root cause of my problem.

 

[Whattteva]

VMware is used to virtualise the TrueNAS environment.
The pool is contained in two VMDK virtual disks, running in a RAID 1 configuration:

Uh oh...

Please read the link in my signature under Useful Links -> Virtualizing TrueNAS the right way. Specifically, point #4.

 

[Arwen]

I have to agree with @Whattteva, you've implemented something that has been shown to cause data loss on occasion. I am no expert in VMing TrueNAS... so I will leave it up to others to make suggestions.


ZFS was designed for the data center, and expects direct access to storage. This means using in-order writes, and write barriers to make sure that data, metadata, and critical metadata are written in that specific order. Doing it that way makes ZFS' copy on write work perfectly. Even with hundreds, (or hundreds of thousands), of un-expected reboots or power cycles. (A battery backed UPS is desirable, but not needed for regular data integrity on a proper ZFS server. Of course, actual hardware failures can occur because of unexpected power faults.)

That said, in the data center, (and my work place), ZFS can reliably run on SAN LUNs that have their own RAIDing. This is because the EMC, Hitachi, etc... SAN array has lots of testing to make sure what goes in, comes out. VMWare on the other hand, is specifically shown NOT to do so in all cases with ZFS. For example, due to reboots of the VMWare host re-ordering writes, and then dropping disk writes that ZFS needed to be done in a specific order because of the reboot.

 

[Davvo]

The hard drives are two Seagate Barracuda Compute 7200RPM 2TB drives.

Those drives are SMR, and you cannot use SMR drives with ZFS: you just found out why.
Have an uneventful restore from backup.

EDIT: Oh dear, just looked at the virtualization part of the post.
There are tons of resources about it, please do read them.

 

[klipsofkool]

So I need to rebuild the system with CMR drives and without any virtualisation to ensure reliability of my NAS?

 

[Davvo]

You must use CMR drives.
You can virtualize if you do as follows. Generally, bare metal equals more stability.

Resource - "Absolutely must virtualize TrueNAS!" ... a guide to not completely losing your data.

[---- 2024/01/16: Still relevant. Virtualization really doesn't change much. Updates made as appropriate. ----] [---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some...

www.truenas.com