BUILD New Homeserver (ESXi with FreeNAS)

[Sc0rc3d]

Hi folks,

I'm using actual and HPN54L as my Home Server with WHS 2011 (build the system 3 years ago and should go now to my parents). And i want something more powerful to play with. :)

The following hardware should be used:

• Fractal Design Node 304
• Corsair RM Series RM450 450W (Gold PSU - should be quote what's important to me)
• Intel Xeon E3-1231 v3, 4x 3.40GHz
• ASRock Rack E3C226D2I (90-SXG180-A0UAYZ)
• Kingston ValueRAM DIMM 16 GiB (2x 8 GiB), DDR3-1600, CL11, ECC
• 1 x 3 TiB WD RED + 1 x 512GiB SSD

4 x 3 TiB WD-Red (1 x 2 TiB Seagate HDD) and a M1015 (in IT-Mode) are already here.

What I want to do:

-> I want to use a 16 GiB USB-Stick for the ESXi Host. Use the 512 GiB SSD for my Server-VMs and the 2 TiB Seagate for some Storage for the VMs (Webspace, Apps, etc.). All pluged in the Mainboard. The 5 x 3 TiB WD-Red-Drives i would connect over the M1015 via PCIE-Passthough to the FreeNAS-VM. I would use a Raidz1 (+external Backup once a month to USB3 drive). My Main-"Problem" is the RAM, i guess (cap. at 16 GiB on ITX).

Servers i want to run:

• FreeNAS (For Filestorage) - 8 GiB RAM (at max. less would be great). :(
• Webserver (MySQL, Apache, PHP (As a test environment - only run's if needed) - 2 GiB RAM
• Game Server - ArmA3/DCS (aonly run's if needed) - 4 GiB RAM
• Video-Render-Server (MeGUI@Windows 7) - (only run's if needed) - 4 GiB RAM
• OwnCloud-Server (should be enough i think - actual i use a RasPI and works great) - 2 GiB

-> The FreeNAS-VM should run the hole time and is most important to me. Web Server/Owncloud should run the "most time". Other Servers only if needed. In that case i can shutdown the Owncloud/Web Server-VM to get more resources for the Render-Server or the Game Server. If something required more/less RAM i can shutdown the VM and give it to it.

What is important to me, is that i reach/utilization the 1Gbit/s Network-Connection (~100MB/s) to my Computer (sure via Ehternet). Actual it works great with my HPN54L (WHS 2011) with 4 GiB of RAM. I want no deterioration. So lets see what you guys think. Advice & Feedback needed :o

 

[Fuganater]

If you plan on running any jails on the FreeNAS VM you should do at least 16GB RAM. Also FreeNAS says you should have 1GB RAM per 1TB HDD.

 

[Sc0rc3d]

Hi, no Jails. Did you read the part about the ESXi? ;)

I'm not sure if i should use the Owncloud Plugin (then i don't need an extra VM for Owncloud) and can get 2 GiB more RAM to the FreeNAS-VM. But i am not sure whats the best way (Owncloud VM vs Owncloud as Plugin in FreeNAS).

 

[INCSlayer]

you do know its not really recommended to run Freenas virtualized right?
it can be done if you know what your doing but unless you know exactly what your doing its not a good idea.
also im not sure if esxi will passthrough the drives to freenas as native hardware and if it doesnt then its an outright bad idea to use freenas in esxi

https://forums.freenas.org/index.ph...nas-in-production-as-a-virtual-machine.12484/

https://forums.freenas.org/index.ph...ative-for-those-seeking-virtualization.26095/

 

[depasseg]

Hi, no Jails. Did you read the part about the ESXi? ;)

I'm not sure if i should use the Owncloud Plugin (then i don't need an extra VM for Owncloud) and can get 2 GiB more RAM to the FreeNAS-VM. But i am not sure whats the best way (Owncloud VM vs Owncloud as Plugin in FreeNAS).

I would imagine that running as a jail will take less RAM than running as a VM.
Have you done any testing to confirm that your other VM's will perform as expected with that amount of resourcing?

And your backup solution is flawed. How do you plan to pass a USB3 device through ESX?

 

[Sc0rc3d]

@INCSlayer: Yes i know. But i don't see a problem in my Szenario and i think i know what i am doing (its not my first ESXi - i'm working on work with it).

"also im not sure if esxi will passthrough the drives to freenas as native hardware and if it doesnt then its an outright bad idea to use freenas in esxi"

I use an IBM ServeRAID M1015 (and on it the 5 x 3 TiB WD-Red-Drives) what i passthrough ESXi to the FreeNAS-VM. That would work with no problem. I found a thread here
in this Forum who recommend that Controller. ;)

@depasseg: Yes, thats my hope so i can get more RAM into FreeNAS-VM (at last i need to test it). Yes, the other VMs has enough (CPU/RAM) Power. They should perform as expected. And in case there are Problems i can shutdown all VM (who i don't need in that moment). The only VM who runs all the Time is the FreeNAS-VM.

Why is my backup solution flawed? USB3 is supportet native in ESXi 5.5(u3 i use):
http://www.v-front.de/2014/11/vmware-silently-adds-native-usb-30.html

Or is there a Problem in FreeNAS with USB3 devices? :o

 

[depasseg]

So are you going to make the USB3 drive a VMDK datastore? Passing it directly through to FreeNAS might not be easy.

 

[Sc0rc3d]

No i want to passthrough the USB 3 Controller and Backup with rsync like this:
https://forums.freenas.org/index.ph...b-drive-using-rsync-for-offsite-backup.12353/

 

[jgreco]

Why would you need 2GB for an OwnCloud VM? 256MB is fine for moderate usage levels.

 

[depasseg]

USB3 is not well supported in FreeNAS (USB2 seems to work better). And I might suggest using local replication instead of rsync, but that's just me. Especially since the drive will be formatted with ZFS, right? And then there is the issue with backing up to a USB drive which seems to have some issues.

 

[jgreco]

Why would you need 2GB for an OwnCloud VM? 256MB is fine for moderate usage levels.

Also idly wondering about 2GB for an Apache/PHP ... you doing something unusually big? We start most web VM's off at 128 or 256 without trouble.

 

[depasseg]

We start most web VM's off at 128 or 256 without trouble.

What OS are you using?

 

[jgreco]

What OS are you using?

FreeBSD of course.

 

[Sc0rc3d]

Thanks for your answers. :)

@jgreco:Your right, my RasPI comes perfectly clear with 512 MB. So the Question is, needs the Jail/Plugin of OwnCload (in FreeNAS) more RAM (or takes more RAM for it - you can not control the RAM usage of Plugins, right?) or the extra VM with 256/512MB RAM. When i use 1 GiB RAM for the Apache/PHP (is use Ubuntu with nginx) then i can spend ~10 to 12 GiB RAM for the FreeNAS-VM.

@depasseg: Thats bad! Then i need maybe some Ghetto Backup. You have some suggestions for a good Backupmethod? About the Filesystem on the Backup Drive i am not sure. NTFS would be great, that i also can access it from my Windows machines. :(

 

[jgreco]

You can theoretically apply some resource limits to jails, but I don't know if or how this is handled in FreeNAS. We don't run crap on our filers here, but if you had to make a choice between running stuff on the filer itself or robbing the filer of memory in order to create a VM, the lighter weight approach would be the jail.

 

[anodos]

You can theoretically apply some resource limits to jails, but I don't know if or how this is handled in FreeNAS. We don't run crap on our filers here, but if you had to make a choice between running stuff on the filer itself or robbing the filer of memory in order to create a VM, the lighter weight approach would be the jail.

Not relevant to this discussion, but FreeNAS 10 will introduce iocage, which supports configuring resource limits for jails. See here: https://github.com/iocage/iocage