Virtualized storage for ESXi

[no_connection]

This is for an assignment at a University in a LAB environment, so no production or great danger to data. Or at least nothing that can't be restored from backup.

We currently have access to one FreeNAS based NAS for storage with four 500GB drives and 8GB RAM. Not exactly the fastest thing around, especially with 18MB read speed. (no idea why though, but I'd rather not mess to much with it)
So I'd like something better as storage for our group, and to spread the load.

We have a four HP servers with 24GB RAM and one Xenon 1.8GHz quad core (without HT). Four 1Gb ports.
Each equipped with two 146GB 15k SAS drives.

I have used one server as storage with FreeNAS before equipped with all 8 drives making the other nodes diskless, but this time we can't spare one host just as NAS.
We need it as a ESXi 5.5u2 host in the cluster.

Any advise how to best use the resources?

My plan was to run FreeNAS as a VM with the controller in pass though. 12GB might be enough leaving us with 12GB for other VMs.

I'm open to other OSes or configurations.

 

[depasseg]

I would start with the sticky about If you really must virtualize FreeNAS in ESX. I think it was written by jgreco. And I think it's right below the Never Virtualize FreeNas sticky.

 

[no_connection]

Thank you for the reply.
I should probably have said that I'm not unfamiliar with those threads. Which is why it's in the off-topic as I didn't want to clutter the normal FN forum or give others the idea that it is a good idea.
Think of it more like getting the job done rather than good idea in this case.
But giving FreeNAS enough amount of RAM means taking up around half the host memory, which is something I'd like to avoid. (still better than all RAM though)

So that is why I'm open to other storage choices. Any suggestions? Or is FreeNAS still the best choice, even as a VM?

 

[jgreco]

What are you proposing to use the FreeNAS instance for? There's a vast difference between trying to store basic documents and other files with a modest access profile, and VM disk storage, which is very stressy.

By way of comparison, the office documents fileserver here is on an 8GB FreeNAS VM with 4 x 6TB drives in RAIDZ2 on VT-d and it works fine. The FreeNAS box serving VM's has 64GB and is on bare metal with a shelf of drives.

 

[no_connection]

It is for VM storage.
The current NAS is basically not enough to keep up and it's getting noticeable. It's an i3-3220 with 8GB RAM Running FreeNAS 9.2.1.1. Four 500GB drives in mirrors sharing through NFS.
That's what we got for our LAB-kit.

Our HP servers are DL360e G8 with 24GB RAM and E5-2403
We got four of them to work with.

And given the 24GB of ram and only four threads we need all working as ESXi nodes to keep up with all machines needed.

*edit* how useful would for example Ubuntu server with NSF share compared to FreeNAS?

 

[jgreco]

Your boxes simply don't have enough RAM to do this job well. Fundamentally, ZFS is a CoW filesystem and part of that is that CoW filesystems are never awesome at VM storage. They fragment, usually rapidly, and the method used to combat that is to throw obscene levels of resources at it. Until you do that, with ZFS, performance will range from meh to poor to "wtf did my fileserver go hibernate for the winter???!?!"

What you might want to consider is using an older version of FreeNAS, skipping ZFS, and using device extents. This ought to work on a smaller FreeNAS VM - I'd be guessing when I say a 6GB VM would be fine, but I'd put $20 on that being the case. (I was the guy who bumped the min spec from 6 to 8 so I might have some special insight into that.)

http://olddoc.freenas.org/index.php/ISCSI#Extents

I'd suggest trying the last release version of FreeNAS 8.

Performance won't be as nice as ZFS with lots of resources, but if you've got a decent RAID controller providing access to the storage, it might work better than ZFS on an undersized VM.

 

[no_connection]

What about open-e?
http://www.open-e.com/products/data-storage-software-v7/lite/

Considering we max out at 8x146GB drives the 2TB limit is not an issue.
And it is for learning so I might just try both. The RAM requirement does look appealing too.

Fun fact: Remote access to the LAB environment won't do you much good if you unplugged all drives. One host refused to boot from USB so I pulled all drives and forgot about it. Took me a little while to figure out why no drives showed up in FreeNAS.

 

[jgreco]

No idea. I've gotten to that point in life where I don't mind throwing resources at a problem to get a better solution.

 

[no_connection]

I wish the Uni could do that as well, but RAM is apparently too expensive new, and they can't just buy it ether due to rules.
At least we learn a lot, which is the point of education.

 

[no_connection]

An update regarding open-e.
Registration and key is a PITA. And general behavior seems less than impressive. GUI gives me certificate warning so I have to import it multiple times. Not to mention being slow and unintuitive.
I will see tomorrow if I can get it running with disks, but if it's half the hassle of just getting it running then it won't have a future.

I wonder why there isn't any small iSCSI appliance that just act as a broker between disk and NIC.

 

[Ericloewe]

An update regarding open-e.
Registration and key is a PITA. And general behavior seems less than impressive. GUI gives me certificate warning so I have to import it multiple times. Not to mention being slow and unintuitive.
I will see tomorrow if I can get it running with disks, but if it's half the hassle of just getting it running then it won't have a future.

I wonder why there isn't any small iSCSI appliance that just act as a broker between disk and NIC.

Because there's no market, basically.