FreeNAS 8 Freezing On VMware ESXi 4.1

[mwheeler1982]

I have a FreeNAS 8 VM set up on my home VMware ESXi 4.1 server. The VM host is a Dell PowerEdge 2850 with a PERC raid card, 3 drives in RAID5 configuration.

Through VMware, I present a single disk to FreeNAS 8. The volume is formatted with ZFS. I enabled CIFS and am accessing the volume over the network from a Windows 7 x64 desktop.

I noticed pretty quickly that if I left a torrent seeding over night, I'd wake up to FreeNAS having frozen. When this happens, I cannot ping, ssh, or http to the FreeNAS instance. If I hop on VMware and access the FreeNAS console, I see the 1-9 menu options. No matter what option I choose, after I press enter, nothing happens. If I do not have any constant data access to the FreeNAS volume, it stays up fine for days.

After forcing a reboot, I see the obvious "volume was not uncleanly mounted" message at boot. Log files in /var/log say nothing about any errors.

I did notice that in VMware, the FreeNAS VM's VMware tools status is listed as "unmanaged". Meaning VMWare tools is installed, but VMware isn't managing the updating of it. I cannot update VMware tools to the newest version because perl isn't packaged with FreeNAS.

I'm not sure if it's possibly an issue with FreeNAS and the SCSI drivers in VMware tools or not; maybe that's just a red herring to the actual problem.

 

[johnnyq]

Actually we had the same issue with running Freenas 8.0 on VMware ESXi 4.1 update 1. It not only froze with Windows 7 computers but also windows XP. We managed to fix the issue by not using ZFS and moving over to UFS for our filesystem. Worked like a charm no more freezes =] I think this may also have to do with the block size you choose for the underlying vmfs file system. Im going to play around with it and let ya know.

 

[William Grzybowski]

http://support.freenas.org/ticket/368

 

[Buhmi]

I've got the same problem, but not in a VM. I'm running Freenas 8 amd64 on an Asrock A330 ION, with 4GB RAM and 4xWD20EARS + 1xSeagate 7200.12 (1TB). The 4 WD drives are running in one ZFS Raidz Volume and the Seagate is formatted as UFS. Once in a while, usually when I'm working on data on the server or copy large amounts of data, the drive gets popped out in Mac OS X 10.6.7, my encoding-program crashes and the server isn't reachable anymore. Neither through AFP, nor via the webinterface. I can see that the server still shows the main menu of Freenas and it seems to be frozen. Any ideas? The only solution I can do then is do a hard-reset :(

 

[mwheeler1982]

William, That's the ticket that I opened because I got no response here. I'm not sure if it will get any visibility or not.

I encourage everyone having this problem to put in your "me too!" in the ticket.

 

[mikelorant]

I am having the exact same problem as well. Nice to know that ESXi doesn't need to be involved for the hang to happen. Rules out one major aspect of this problem.

Can we get some developer comments regarding how to debug this?

 

[shoon]

I'm having the same problem running on bare metal, here's my forum reference.

It's very difficult to diagnose when the webui, console, and ssh all seem to lock up simultaneously. I'm going to delete the ZFS volume and see if that removes the instability.

My current findings leads me to believe there is something wrong with the broadcom driver (ifaces with bce)

 

[mikelorant]

Changing to 1 CPU has resolved this problem.

 

[Buhmi]

But on an Atom 330 with only 1.6 Ghz thats not really an option :(

 

[skywise]

I was having the same issue with esxi 4.1, dropping to a single vCPU solved it but I would like to give it 2 for performance reasons.
Hope a solution can be found.

Greg

 

[Nexes]

I was having the same issue with esxi 4.1, dropping to a single vCPU solved it but I would like to give it 2 for performance reasons.

Same here. (with the HP Proliant Microserver + esxi 4.1)

Was it always like that? Or is it possible to install a previous version of freenas to make it work?

Greetings from Munich

 

[TravisT]

Glad I saw this issue. I rarely see high cpu usage with just 1 vCPU, but I'm also running VMWare ESXi 4.1. No problems here, and I'm using raidz on three disks and the OS on a separate disk. Subscribing to see if there is a fix for this in case I decide to allocate another vCPU in the future.

 

[ptmixer]

I wish I had found this thread earlier. This issue still exists. I'm on ESXi 5.0.0 using virtual hardware version 8 (the newest) and running FreeNAS 8.0.2-RELEASE. I was posting over here http://forums.freenas.org/showthread.php?511-System-freezes-when-writing-data&p=14612&viewfull=1#post14612 (page 4) but none of those fixes helped my situation, but changing to 1 vCPU seems to have done!

 

[TravisT]

Does anyone think performance would increase dramatically if you allocated another vcpu? I don't see too much CPU strain on my build. Seems that RAM is the biggest performance factor for these boxes. What's everyone else's opinion.

 

[ptmixer]

Does anyone think performance would increase dramatically if you allocated another vcpu? I don't see too much CPU strain on my build. Seems that RAM is the biggest performance factor for these boxes. What's everyone else's opinion.

Depends on your setup. I'm running a three drive RAIDZ with compression on. While copying data to the system the CPU is pretty much maxed out. So, for me I think it would help a lot. Running 2 vCPUs was not problem with Freenas 0.7.

 

[TravisT]

Good point. I was thinking that since dedup is not included in the ZFS ported over to the freebsd base that freenas is built on, that cpu wouldn't have much effect. Didn't think about compression though. I'd be interested to see what performance gains I get from adding a vCPU, as my current ESXi host has plenty of power to shell out another vCPU to freenas.

Hopefully this gets looked into further so I can give it a go.

 

[YTKColumba]

can anyone tell me what kind of provisioning you are using for your disks? I tried using Lazy Zeroed and I sometimes ran into an issue of when copying large files, it says the drive is not accessible, but when I click Try Again, it will continue and will eventually finish. I was wondering if using Eager Zeroed would help since it preallocate space and it will not take performance hit on first write.

I have another post over here: http://forums.freenas.org/showthread.php?5443-Error-During-Copy-To-FreeNAS-ZFS&p=19228#post19228

I allocated 2 vCPU to FreeNAS, do you think that could be the problem?

 

[jtmink1]

Me TOO!

I'm having the same problem on ESXI 5.0

Single VCPU fixes it