Advice on clever configuration of my two servers for VM use

[Murac]

I recently built a supermicro machine. Parts are still coming in, and still prepping and testing everything for deployment, but I will be posting a separate write up of my experience with reducing noise and getting it set up. So stay tuned for that.

What I'd like to ask for is some advice in an optimal set up moving forward in order to get the most out of my hardware.

New build:
Chassis: Supermicro SC846E16 24-bay
Mobo: X8DTE-F
CPU: 2 x L5640 2.24GHz (12 core)
Memory: 96GB Samsung 1333 DDR3L
HBA: LSI 9210-8i
HDD (going to be): 2 x (6 x 4tb WD RED in raidz2)

Old build:
Memory: Crucial 16GB Kit (8GBx2) DDR3L 1600MT/s (PC3-12800) DR x8 ECC UDIMM
Cpu/mobo: ASRock C2550D4I Intel Avoton C2550
Hdds: 6 x WD Red 4TB NAS Hard Drive: 1 to 8-bay RAID Hard Drive: 3.5-inch SATA in RaidZ2
Case: Silverstone Tek Premium Mini-ITX DTX Small Form Factor NAS Computer

While I see it's possible to virtualize freenas, I'd rather not. Initially, I was going to get rid of the old box, but perhaps it can still serve a purpose. The drives will be in the supermicro chassis.

Can I efficiently run 10-20 VMs inside of freenas without killing performance? This is a personal server so the stakes aren't high. Currently the most intensive program I run is Plex inside of a jail. Also have the other usual plugins to go along with that. What I want to do is play around with all these cool softwares I'm finding. And some of them seem to be easier to setup and run inside of linux instead of a freebsd jail. Since the way I have things set up right now works just fine, I'll probably leave plex, sab, sonarr, etc all running in their own jails (assuming freenas lives in the SM box)

Correct me if i'm wrong but jails don't lock unused resources from the rest of freenas but VirtualBox will. And if I run a few VirtualBox VMs that are only lightly active, it could degrade plex's and freenas' general performance. Or do I just run one VirtualBox VM with everything I want to play with inside of that? Not exactly best practice. Is bhyve a good option in 9.10?

I want the free reign to explore every option available to me but for now here's what I definitely want to be able to do as a whole (lumping potential jail solutions as well):

-GitLab
-development environments (LAMP, MEAN, etc)
-web scraping
-centralized password storage (LastPass ain't going to last forever I fear)
-nextcloud
-guacamole or some other remote access solution
-curious about how much web hosting I could manage. And I don't imagine much traffic would be garnered. but say, a personalized homepage to keep track of links and notes
-monitoring dashboard: grafana i just learned about. but you guys probably see where I'm going with that. networking traffic, VMs status, server status, other metrics
-plexpy
-SabNZBd
-CouchPotato
-Sonarr
-Plex

I imagine a lot of what I'm trying to do will be simplified with the release of FreeNAS 10. But until then, what are my options? I'd also like to have the ability to use a reverse proxy or what have you to have some of these services available publicly like any other server. (GitLab, homepage, dev env, nextcloud, remote access, etc)

Is it viable to keep the old box as the actual freenas system while keeping the drives in the new chassis and just use a regular virtualization solution in the SM system? I.e. can I wire the drives via the backplane outside into the C2550 system and then use it as storage via network? Presumably putting the HBA card in there, and more RAM. Or even the other way around? Should I keep the old box in use for virtualization solutions not easily executed in a jail?

Can I do everything I've listed above all inside of FreeNAS? (my dream)

Apologies for the lack of clarity in some of my explanation, I'm just trying to wrap my head around all the possibilities. While also being as brief as possible. I've gotten pretty good at managing my current system, this next step is somewhat of a brave new world.

As always, thank you for the advice.

 

[Stux]

FreeNAS 9.10 actually supports the bhyve hypervisor, which should do what you want, but it won't be integrated into the GUI until FreeNAS 10.

 

[Murac]

FreeNAS 9.10 actually supports the bhyve hypervisor, which should do what you want, but it won't be integrated into the GUI until FreeNAS 10.

Okay then, maybe I'll go that route. Documentation looks sparse so far. I run iohyve in a jail, correct?

 

[Stux]

Okay then, maybe I'll go that route. Documentation looks sparse so far. I run iohyve in a jail, correct?

No, I believe it's built into the kernel, and you run it direct from bsd. Then the vms are sort of like jails

Cli all the way

I plan to start playing with it, but haven't yet, sorry.

 

[SweetAndLow]

It sounds like you need a full on virtual host and a Nas. Combining them with so many services isn't a great idea.

Sent from my Nexus 5X using Tapatalk

 

[Murac]

It sounds like you need a full on virtual host and a Nas. Combining them with so many services isn't a great idea.

Sent from my Nexus 5X using Tapatalk

Yup that's what I was afraid to hear. I'll have to make due with what I have until I save up some money..

 

[SweetAndLow]

You might be able to get away with building a single virtual host using bhyve on freenas and running docker containers inside of that vm. This could get you everything you need with minimal overhead compared to multiple vm's. I think when Freenas 10 comes out this is going to be my solution. You will be going down uncharted waters though since this is new for everyone including freebsd.

 

[Stux]

The benefit is having all your sub-vms having vastly improved bandwidth to the FreeNAS pool vs accessing it via gigabit. And an all in one.

FreeNAS basically doesn't use any CPU power, which means you can get better use of the CPU for Plex, MythTV, Zoneminder etc

 

[Murac]

Beautiful. I will definitely be doing research into bhyve. I just successfully got nextcloud installed in a jail which is cool. I think I will also throw an SSD into the soon to be less important C2550 server and run atleast some of the services in there. It'll be a good experience to check out proxmox and make a few things easier to do until I totally swat all the hornets in bhyve.