BUILD Downscaling my FreeNAS machine (Yet Another Silvermont Thread)

[Xiche]

First, a little background:

I was a very early adopter of the SM X10 + Haswell Xeon hardware. In fact, I was even the first one to post my success with the hardware on this very forum. It has been a rock solid system for me and I originally overspec'd it by a decent margin (see the build details in my signature) so that it could double as a VM server using the then-rumored upcoming/experimental VirtualBox support in FreeNAS. However, and I mean this in the kindest way possible as it relates to an awesome and *free* product I've been enjoying, the VirtualBox support in FreeNAS ended up being kind of half-assed and clunky. On top of that, they backed themselves into a bit of an ugly corner on the implementation as far as upgrades go due to needing the VBox kernel modules in the "host" FreeNAS OS to be synchronized with the version of VirtualBox in the FreeBSD jail. CyberJock raised a ticket about updating the VirtualBox jail, but that ticket has been pushed out as Future/Won't Fix. The general response in the ticket seems to be that they're going to be abandoning VirtualBox in FreeNAS 10 anyway in favor of bhyve so it's a very low priority to update it, if they ever do.

As a side note, the only complaints I've had about the hardware relate to the Super Micro board. They fall into two categories:

1. Documentation - Super Micro provides absolutely nothing in the way of Release Notes for the BIOS and BMC firmware updates. Supposedly if you directly contact them and are persistent, you might get some of this information, but I've not had any luck. Every other motherboard vendor I've ever dealt with has given some sort of Change Log or Resolved Issues information for updates. They seem to lean very heavily on the "if you haven't found the issue yet, you don't need the upgrade" mantra.
2. BMC Updates - I know the best practices for using a BMC involve having it live on its own isolated/protected network, but it would still give me a bit more of a warm and fuzzy feeling if there were more BMC updates, necessarily *with documentation* to handle the ever growing list of vulnerabilities in their particular implementation and the underlying Linux packages it runs on top of.

Anyway, long story short, I've decided to reclaim the server hardware for a dedicated VM server using Linux/KVM and build a much lighter weight server to be my FreeNAS box. Besides the VMs currently running on it, I don't really do anything more CPU intensive than sharing out a couple ZFS datasets via SMB and NFS shares to a couple client machines at a time. No Plex transcoding, no couchpotato torrenting, etc. To this end, I am investigating the Silvermont-based Atom boards from Super Micro (aside from what I mentioned above, I really do like the board that I have).

This is my current hardware list:

Case: Fractal Design Node 304
RAM: 2 x Kingston 8GB 204-Pin DDR3 1600 SO-DIMM ECC Unbuffered (KVR16LSE11/8KF)
PSU: SeaSonic SS-300ET Bronze 300W
HDDs: 4 x Western Digital Red NAS Hard Drive WD20EFRX 2TB

These are the motherboards I'm trying to decide between:

A1SAi-2750F ($356)
Pros:

• TurboBoost gives this board both the best single-threaded performance and overall performance.

Cons:

• The most expensive option.
• No QuickAssist support.

A1SAi-2550F ($276)
Pros:

• TurboBoost gives this board better single-threaded performance than either of the Rangely options.
• $80 cheaper than the 2750.

Cons:

• No QuickAssist support.
• Only 4 cores.

A1SRi-2758F ($333)
Pros:

• QuickAssist support could be useful in the future, especially if it ends up getting re-purposed as a pfSense box.
• $23 cheaper than the 2750 for some reason (the posted tray prices from Intel would have you expecting it the other way).

Cons:

• Slightly slower (~8%?) than the 2750 when the missing TurboBoost would kick in.

A1SRi-2558F ($250)
Pros:

• QuickAssist support.
• The cheapest option by $26-$106.

Cons:

• The same ~8% slower due to missing TurboBoost.
  
• Only 4 cores.

So, the question really is this: Am I really even going to notice the difference between the C2558 and the C2750 given my previously described workload of simple network file sharing. If so, can I go somewhere in-between (C2550 or C2758) and be OK?

As a note on the chosen power supply since jgreco has been heavily warning against picking potentially underpowered PSUs in recent threads, my calculations for peak power requirements look something like this:

• Case fan controller = 15W
• CPU + Motherboard + BMC = 32W
• HDDs (Spin-Up) = 4 * 12 * 1.73 = 83W

So, a total of around 130W peak draw, which seems safe with a 300W 82%+ power supply. It should even be fine if I end up adding another two drives down the road (making a 4 disk [RaidZ2] vdev was the biggest mistake I made with the last build). Still, I might end up reconsidering.

 

[cyberjock]

You didn't hurt my feelings. It wasn't the gravy train we all hoped it would be. Let me clarify a few things though.

So it is still on my list of things to do. There's some catches.. for one my box won't let me create jails. No clue what's wrong and as I'm a low priority right now (and you can update your Virtualbox jail yourself) it's not exactly a show stopper.

Virtualbox isn't as great of a virtualization product as many of us are used to (especially those that use VMWare Workstation). It's just not. It's open source, contributed to by a bunch of guys that volunteer. So Virtualbox has some things going against it right out of the gate. If there was a version of VMWare Workstation for FreeBSD I'd have a guide and such posted immediately. I'd be creaming my pants every night because it would be so awesome.

The kernel driver requirement has to do with how Virtualbox is designed. It must have that driver to function. FreeNAS' design as an appliance clashes with the design Virtualbox expects. So there's some problems to deal with, and I've provided the most reasonable experience I can with a pre-made jail template and the virtualbox driver being part of FreeNAS.

From what I've read about bhyve (I have not tried to use it) is that it might not be the gravy train we are still hoping for either. bhyve doesn't support Windows at all, and only some flavors of linux (along with most distros of BSD). FreeBSD is not a popular OS when you look at the number of systems that have licensed, paid-for software. So naturally all of the revenue that could be put into development of a great type 2 hypervisor product just isn't there. So that leaves us with what we have...

We have jails. If you're a BSD guy (or willing to become one) you can do *amazing* things in the jail. Of course, this requires supported software, which many people find distasteful. One person I chatted with a while back said something like "When Exchange is offered on FreeBSD then we'll use it". That's just silly because Exchange is a proprietary product made by Microsoft. There's about 0% chance of that ever being on anything except Microsoft OSes. There are *tons* of options that could work just as well as Exchange... if you are willing to take the time to learn how to use them and set them up. But nobody wants the work. They want to use what they want, at the price point they want (free), and they want it to be dead simple to setup and manage.

Unfortunately that's not reasonable. Everyone knows the phrases "time is money" and "money is power". That naturally concludes that time is also power. Time gives you the ability to learn how to do things, but people don't want the power in today's world. They want their time and money.

For the few of us that do want the power and are willing to spend time to learn *something* and spend money to have the opportunity to learn that something, you will be a step above your peers when it comes to getting the good paying jobs, getting stuff done at work, etc.

So back to regular programming...

Quickassist means *nothing* for FreeNAS. The OS doesn't support it, and probably never will. I wouldn't buy hardware if you are expecting FreeNAS to support Quickassist someday. That's something that might or might not come, but do NOT bet your bank account on it.

Before you go dropping money on this box at least look at the prices for building something like a Pentium G3220 system. Yes, you'll probably save 5-10w of power going with Avoton over the Pentium when idle. But the extra cost to go with Avoton has often shown that you'll never actually recoup the money in electricity saved. And with the Pentium you could always drop in a Xeon or some other beast if you really needed to. The Avoton has no upgrade potential except for RAM.

 

[Xiche]

Don't get me wrong - I love VirtualBox and have used it almost exclusively for my personal virtualization needs since about 2008. The support for it that you helped put together for FreeNAS (thanks!) works more or less OK for the couple VMs I have running on it, but was quirky enough to make me want to go down this path. I had never used phpVirtualBox before to manage a headless configuration and had some issues with it (wasn't working in Firefox, couldn't launch Java-based console in Chrome, etc.), but mostly worked around them. I know that it's only one guy developing phpVirtualBox in his free time and it works pretty well, and I certainly don't expect anything more from it - I contribute to open source projects both as a hobby and as part of my day job and totally understand and appreciate how volunteer open source software development works. The only issue that I haven't been able to work around is console redirection causing the VirtualBox process to core, which may or may not be related to the fact that FreeNAS 9.3 appears to be using kernel modules from 4.3.16 and the jail is still using VirtualBox 4.3.12. I see that 4.3.20 is available via pkg inside of the jail, so what would be the best way to compile matching host kernel modules for FreeNAS to load? I tried updating virtualbox-ose-kmod inside of the jail and copying the kernel modules out for FreeNAS to use, but all that got me was well acquainted with how to access single user mode in FreeNAS 9.3 to undo it :) (resulted in a system lockup with the kernel constantly spewing backtraces to the console).

On the hardware side, I am slightly priced into a mini-ITX solution since I jumped on the Fractal Node 304 case while it was on sale (almost half off what it is now at NewEgg). I understand that QuickAssist is not currently used in FreeNAS, but I was giving it weight because I might want to either repurpose the system into a pfSense router at some point or even run OpenVPN in a FreeBSD jail, which should soon be able to take advantage of the cryptodev support for Intel QuickAssist (according to the pfSense guys).

I am definitely taking your advice to heart, though, and am considering just starting over with an X10SLM+-F uATX board ($180) and Pentium G3220 ($59) since that combo ends up being cheaper than any of the Silvermont boards I was considering, and, like you said, gives a lot of future upgrade potential. I'm sure I'll build some other mini-ITX system someday to use the case with.

 

[cyberjock]

To be honest (and I haven't tried to dig into this to find out if its true) but rumor has it that the kernel driver hasn't changed since 4.3.8. I don't think 4.3.20 was out when someone told me this and I haven't really said anything about it because I hate people to take what I say as "well, must be true". This is unverified and could be wrong. But if true, upgrading to 4.3.20 from pkg-ng with the currently released virtualbox template shouldn't really break anything.

I have heard that phpVirtualBox kind of sucks. I've never personally had a problem with it. I'm inclined to think that either it sucks or its at least picky and I've somehow beat the odds because other forums have mention of phpVirtualBox being pretty unreliable at times.

Hehe. I ended up being a pro at single user mode when experimenting with the kernel driver. So I know exactly what you mean.

AFAIK QuickAssist requires drivers and such, none of which exist in FreeNAS. It's something that if you really wanted to have supported in FreeNAS you'd probably have to provide the patch yourself to the git repo for FreeNAS. I'd wager that they dev team isn't going to be quick to spend dev resources on it. Even when I was doing the Virtualbox stuff I had only minimal help with getting Virtualbox working. Probably 90% of it was done by me and a few other community people working on the Virtualbox jail.

 

[nidur]

Everyone knows the phrases "time is money" and "money is power". That naturally concludes that time is also power. Time gives you the ability to learn how to do things, but people don't want the power in today's world. They want their time and money.

For the few of us that do want the power and are willing to spend time to learn *something* and spend money to have the opportunity to learn that something, you will be a step above your peers when it comes to getting the good paying jobs, getting stuff done at work, etc.

Brilliant, made my evening.