ZFS operations hang

[Vyacheslav]

Hello everybody.

I have a really weird problem, which happens occasionally approximate once a week.
The problem is that all operations to ZFS fail until reboot, but at the same time zpool status said that it is all right.
Additionally, at that moment it seems that Freenas works as a normal. I'm able to log in session (SSH or console) but when I requires directory list from the pools, the operation hangs, I can still type something in this session, but operation will never finished and consone will never return to command prompt.
for example : ls /mnt/bak-vol02/dep-mirror/VM_DEP_cluster1/
Other console session are working as normal, and also I can open new one and work there until access to ZFS pool.

I have 2 pools on server and both have the same behaviour.

I tried different configuration, but got the same issue.
I have 2 storages: ibm ds3400 and ds 3512 with dediceted LUNs which share to the Freenas. They are connected via FC HBA.
First hard conf. I tried is:
Vmware ESXi with LightPulse HBA 36Gb RAM. Freenas installed on vmdk and LUN mapped as a RAW disk. Server is IBM x3650-type 7979
Second hard cong. is: Freenas is running on phys. hardware IBM x3650-type 7979, freenas installed on internal ibm raid 1, storage connected via QLE 2562 HBA.

Logs keep silence.

I will appreciate any help and suggestion, which head me to further investigation regarding the issue, because I have no idea where to dig deeper.

FreeNas is used as a enterprise backup storage using iscsi, CIFS and rsync.

Thanks in advance.

Regards, Vyacheslav.

 

[cyberjock]

Not a clue.. but since you virtualized you can expect nobody with any experience to respond. We don't support or condone running in a VM because of the dangers involved. Far too many pools have just disappeared on a FreeNAS reboot.

 

[Vyacheslav]

But I have the same results on a physical environment.

> Freenas is running on phys. hardware IBM x3650-type 7979, freenas installed on internal ibm raid 1, storage connected via QLE 2562 HBA.

additional kernel module for this HBA has been installed.

 

[cyberjock]

Right. I understand what you are saying. But we've learned to immediately ignore someone the second they mention running it in a VM. We equate running in a VM to:

1. An advanced user that shouldn't be asking for help(or they shouldn't be virtualizing to begin with).
2. A whole mess of problem because of virtualizing that is often impossible to really handle over a forum setting without exhaustive time consuming efforts.

So we got the warnings not to do VMs and we leave people to their own volition when they virtualize. Too many of us have spent days trying to troubleshoot a problem that turned out to be a setting improperly set in a hypervisor. So to ensure we never get into that kind of multi-day mess again, we don't support it at all.

If one is virtualized and one isn't but they are on the same bare metal hardware then I'd take a look at what they are sharing in common with each other. Something is just not working out right.

The second thing that sets off red flags for experienced users is installing FreeNAS on a hardware RAID. Regardless of whether you can do it, it's something you shouldn't do. Just like with virtualizing, far too many problems have rolled back to something bizarre with the hardware RAID and FreeBSD.

Not to be rude, but most of us are going to look at your post, see your mistakes, and then run the other way because we don't even want to know what other mistakes you've made. We deal with so many users that do everything we say not to do in the manual we've taken up the attitude that if you couldn't read and follow the documentation we've provided on how to have a smooth running server we aren't obligated to help solve a problem that is quite possibly the result of user error anyway.

Sorry.

 

[Vyacheslav]

oh no... you've just broken my pink glasses. :D I got your point, and proceeding for further investigation on my own unless anybody wanna give some more hints.

thank you for your help.