AiO ESXi FreeNAS homeserver configuration

[ondjultomte]

So I have jumped the ship and ordered most parts for my new AiO server.
I need some input on how to configure the server for best performance.

Hardware:
32GB ECC
Supermicro X10SL7-F
Xeon 1220v3 (not yet ordered)

I will started with a mix of drives and replace them in the coming months

1x some thumb drive to boot ESXi from
1x Intel X25-M G2 80GB for VMs.

Storage drives will run raidz2:
3x HGST 4TB NAS
1 2TB wd green
1 2TB Samsung
1 3TB wd , something, cant remember now.

The last 3 drives will obviously be replaced with 3 HGST 4 TB NAS drives later on.

ESXi
VMs:
FreeNAS
Windows
Ubuntu
pfsense



My movie and music collection will take up the most part and redundancy isn't really critical.
But my collection of dSLR raw files will also be on here, but its currently only about 1.5TB
but growing fast ( I got two kids at home :) )

Traffic to the server will be writes from my workstation when I backup newly imported images and movies from a local Hdd. Perhaps 8 to 16 GB now and then.


Most throughput will come from a windows VM running on the server with torrent downloads from my internet connection 525 Mbit , ~ 60 MBps dl in utorrent.
Otherwise there will mostly be random IO reads from torrents and FTP from the server.

Or the torrent client transmission in freenas with couchpotato.

I want Freenas to handle large DLs, 100GB+, at 60MBps with no problem, with some random reads from the zpool at the sametime.

I will also backup the family's documents, word and excel files . also use crashplan for relatives to backup images to the server.

so how do i setup the server for best performance?

18 GB ram for the freenas VM
6 GB for windows VM
4 GB for Ubuntu VM

How do I config the zpool for best performance?
Would a 100 GB ssd as l2arc be useful due to all the random reads? most data will be hot for a week or so then cold, so most reads would come from the ssd, reducing the load on the drives.

I will dedicated one GBE NIC for internet and one for the LAN. I might get a pair of QLogic QLE 2462 for the server and a 2460 for the workstation too transfer files faster via FC. These cards are so cheap on ebay. If there are 2460 drivers for windows 10.


Any tips and feedback on my build and what to think of configuring the system?

 

[ondjultomte]

No one?

 

[dlavigne]

Hmm, guess not. Which solution did you decide upon?

 

[ondjultomte]

Im ordering the cpu today, so I will probably be building the server next weekend, I'll update the thread when its time for the configuration

Skickat från min HTC One via Tapatalk

 

[ondjultomte]

I need to find out if Qlogic qle 2462 has Windows 8, 10 drivers

Skickat från min HTC One via Tapatalk

 

[jgreco]

Hmm, guess not. Which solution did you decide upon?

Yeah, the problem with people so often doing crummy AiO designs that involve bad decisions is that when some user finally comes along with what seems like a sensible plan, no one reads past the word "esxi".

And I'm sorry I missed this. As the resident virtualization goo-rue, I do try to look at these things, but I've been spread thin lately.

OP: Congratulations on what seems to be a well-considered plan. Please note that there's documentation on virtualizing FreeNAS on the forums. FreeNAS is intended to be a storage appliance, and we've seen so... many... cases... where people have made poor decisions when virtualizing and then had their data eaten, usually irretrievably, because they screwed something up (whether in design or operationally). The worst offenders are typically the ones who want to build an AiO virtualization lab on ESXi that has a FreeNAS VM storing their VM data on a 16GB RAM whitebox from Shenzhen Back Alley PC Corp.

The guy who comes in and says he's planning a 16+GB FreeNAS VM for general filesharing on a well-behaved and known-to-work board, well, that's a rare one. Please be sure to follow the guidance in the link provided. With virtualizing, you are much more on-your-own than if you install on bare metal. Make sure you're entirely comfortable with ESXi and understand fully what's going on. Part of the reason I discuss virtualization at a kind-of-vague abstract level is that I have no intention of handing a loaded gun to someone who isn't qualified; it is your responsibility to either make sure you already have the skills, or to acquire them. If you can read what's linked above and go "oh well that makes perfect sense" or (better yet) "duh that's obvious" then I deem that sufficient virtualization-fu that you're very likely to succeed.

So it seems you've got your six disks. I suggest RAIDZ2 is probably what you want. It is a good balance of resilience without wasting a ton of space. When you replace that last old 2TB disk your pool size will jump from 8TB to 16TB if you do RAIDZ2.

You don't have to boot ESXi from a thumb drive. You can install it to the SSD if you wish.

We usually consider a ~16GB platform to be too small to benefit from an L2ARC. A typical safe sizing for an L2ARC device is about 5x the size of your FreeNAS RAM. It can be removed if it turns out to be hurting you, so I don't think there's necessarily harm in trying it. But also remember that if your pool isn't 100% busy, it isn't buying you much of anything.

 

[ondjultomte]

Thanks for the comments and thoughts.

Good idea do make a bare metal FreeNAS usb version as backup.

I was thinking of adding a L2ARC, not for speed, but to offload the hdds, less heat and wear, hopefully increasing their lifetime.

 

[cyberjock]

I was thinking of adding a L2ARC, not for speed, but to offload the hdds, less heat and wear, hopefully increasing their lifetime.

Yeah, you won't accomplish that goal. You'll probably end up with MORE disk activity because your ARC is too small to support an L2ARC.

 

[ondjultomte]

Still waiting for the CPU, so I got a cheap 240GB ssd with the primary usage is to cache torrent DL . Where do I place it for best result and versatility ?Will the IO be limited if I put it on the internal controller and share it via ESXi ? I was thinking I could put both VMs and use it for torrent DL cache.

 

[jgreco]

Then add it to ESXi as a datastore and put your VM's on it. You can create virtual disks as needed and attach them to whatever VM you need them on. ESXi does not require all your virtual disks for a VM to be on a single datastore.

 

[ondjultomte]

I'll try that and pass a 30gb VM hdd to freenas as l2arc

Skickat från min HTC One via Tapatalk

 

[ondjultomte]

I bought these dimms, i hope they are okay.

Skickat från min HTC One via Tapatalk

 

[ondjultomte]

Dammit, no post screen when starting, five beeps from the mb. And The workstation is totally dead...

Skickat från min HTC One via Tapatalk


I shouldn't have removed my wks mb from the chassi.

 

[ondjultomte]

My workstation is up and runnign again. BUt I still got problems with the server

 

[ondjultomte]

Hm I seem to have bought RDIMM ... doh .

 

[jgreco]

Ugh, that's not good!

 

[ondjultomte]

Okay got udimm here now. I cant understand how i missed that. Needed a new psu too. Ill keep the thread updated when i buildt the box

Skickat från min HTC One via Tapatalk

 

[jgreco]

Okay got udimm here now. I cant understand how i missed that. Needed a new psu too. Ill keep the thread updated when i buildt the box

Skickat från min HTC One via Tapatalk

Easily missed if you don't do this for a living. It's why I advise people to use a manufacturer's memory selection tool.

 

[ondjultomte]

I shouldn't have missed that .. any how memtest86 is running now on the box.

I also bought 3x 16 GB SanDisk Cruzer Blade. 2 for WMware , Ill mirror the drive manually, since there is no raid1 for usb drives :) and one for free nas that will mirror the freenas installed in a SSD

 

[ondjultomte]

After the memtest I guess its time to flash the SAS controller.