Seeking advice on build: performance vs energy savings+silence

[Caspar]

Hi,
First of all great forum with lots of info (thanks!). The last couple of days I've been reading lots of pages on the forum and now I'm very excited to try and build my own FreeNAS server.

My intention for the server is to use it for home/personal stuff, and learning. Most important is a reliable storage solution at home (encrypted data), file (streaming) server and personal Git server. I have about 2TB data currently. Since I want the machine to be always available I would like it to be somewhat energy efficient (especially when running idle) and because we have a really small house the server must be located in the bedroom (my wife made me turn off my ReadyNAS Duo because of the noise level).

My priorities are: 1) Reliable 2) Low noise level 3) Performance/scalability 4) Energy efficiency 5) Price

With this in mind I've been doing some research and came up with the following 3 options:
1) I believe good performance:
* Mainboard: ASRock C2550D4I with embedded Intel Atom Quad core CPU see http://www.asrockrack.com/general/productdetail.asp?Model=C2550D4I#
* RAM: 2 x Crucial CT102472BD160B 8GB ECC (listed on ASRock memory QVL, should work)
* PSU: Seasonic Platinum FL2 400W see http://www.seasonicusa.com/Platinum_Series_FL2.htm
* Case: Fractal Design NODE 304 http://www.fractal-design.com/home/product/cases/node-series/node-304-white
* HDD: for now 2 x WD Red 3TB (which I would like to have in RAID, I guess RAID-Z2)

2) More performance, same as 1) except
* Mainboard ASRock C2750D4I with embedded Intel Atom Octo core CPU, see http://www.asrockrack.com/general/productdetail.asp?Model=C2750D4I#

3) More performance and more scalable I guess, same as 1) except:
* Mainboard ASRock E3C226D2I, see http://www.asrockrack.com/general/productdetail.asp?Model=E3C226D2I#
* Intel Xeon E3 1220V3, see http://ark.intel.com/products/75052/Intel-Xeon-Processor-E3-1220-v3-8M-Cache-3_10-GHz

Although I think I don't actually need the Xeon build, I probably would enjoy it most. I believe the Xeon is better suited to run VMs (which I might want to do in the near future). The price difference 1) vs 3) is about EUR 125 for the parts, that's no problem. What I am worried about is the extra energy consumption when running idle and (maybe) extra noise.

So my questions are:
* Is the PSU Seasonic Platinum FL2 400W a wise choice? I deliberately chose this one because it's fanless. But it might burn the Xeon?
* Should I buy extra fans for the NODE 304 case?
* I saw that the Xeon has a feature called 'Idle states': will that really make a difference for cutting noise and energy costs? I'm guessing the new server will be running idle for most of the time. The max. TDP for this Xeon is 80W but will consumption drop significantly when running idle?
* The Xeon has VT-d support, the Atoms don't. Is VT-d important for running VMs?
* Does anyone use parts from the above lists and if so, please share your thoughts!

Hope my questions make sense, if not let me know why. Thanks in advance! Haven't bought anything yet but once I'm confident about the parts I'll run (I mean surf) to the store:D and keep you posted.
Caspar

 

[JambaFun]

Hey there Caspar!

I just finished my first FreeNAS build using Node 304 and E3C226D2I so this seems like a good thread to give my thoughts on the system. I'm still in my burn-in-phase (having spent a couple of weeks learning FreeNAS in a virtual environment), so I can not attest for the reliability of the system. Although I feel fully confident that as long as you follow the best practices taught by the masters of this forum you won't have to worry too much about that.

Now my priorities looked a bit different than yours:

• Above all else it had to be a small form factor. The only place for server equipment at home is a small closet. Node 304 fit perfectly. More on noise levels later.
• Secondly, I don't indent for this system to do anything but store files. I will use it as a second off-site backup for my business among other things. This meant I could go for a less powerful CPU to save some money. I ended up with i3 4150 without really thinking about it too much. Call it personal preference.
• Last off, I really wanted to use 6 drives. It hits the sweet spot for allowing any 2 drives to fail plus having a good ratio between disks purchased and usable space. It also has the benefit of filling up the case which means I hopefully don't have to touch the system for very long. (I maxed out RAM of course.)

So back to you. First off, noise levels. If you really have no option but to put the NAS in your bedroom you have to rethink things considerably. I used to live really small (both at school and later on) so I've had to build near dead silent systems a few times. You need a dampened case, big tower coolers or water cooling with big slow moving fans plus really good airflow. This makes Node 304 a terrible case. Currently I have pretty decent temperatures considering it's scalding outside, cpu 47 ˚C (stock cooler) and drives 30 ˚C, but the system is loud as hell. There is absolutely zero room for a big cooler in there due to cables and all the fan outtakes leak sound very well. I'm sure I could build a system silent enough for the bedroom if I had to, but not with a case as small as this.

If however you could place the NAS somewhere else I strongly recommend Node 304. You won't need any extra fans or anything. I do regret using a Seasonic PSU. They have the power connector in a very bad spot for Node 304 meaning I had to choose between putting the PSU upside down with only two screws attached or (as I did) bend the power cable almost to the point of breaking. I will look into getting a replacement power cable where the end going into the PSU is mirrored. If it exists.

Regarding power consumption and what VT-d is for I think you could pretty easily find the answers through Google.

Cheers!

 

[joeschmuck]

Just to drop a little bit into the bucket... An Atom processor isn't going to make you happy with respect to all the tasks you currently think you will want to do and after you get it built up, you will be disappointed because it won't do more. If your priority is silence then you could purchase six 1TB SSD's (3.6TB usable storage) for the system which would actually make the system almost silent. A good quality Seasonic (buy the proper model for the case and wattage needed) is a good choice too. And a large CPU heatsink slowly moving air across it. This does put silence as #1, size at#2, and price as #5.

 

[danb35]

* HDD: for now 2 x WD Red 3TB (which I would like to have in RAID, I guess RAID-Z2)

RAIDZ2 is comparable to RAID 6--it's an array consisting of at least four disks, in which any two disks can be lost without loss of data. RAIDZ1 is similarly comparable to RAID 5--it consists of at least three disks, and any one can be lost without loss of data. A two-disk redundant solution would be a mirror, which outside of the ZFS world would be called RAID 1, but in the ZFS world it doesn't have a particular RAID label attached. With a two-disk mirror, either of the disks can fail without loss of data.

 

[Zou]

Hi Caspar

I'm not a FreeNAS expert but recently i've done some research on building an extremly quiet system. Keep in mind that's only theoretical reflections I never build it, use it or test it IRL.

* Is the PSU Seasonic Platinum FL2 400W a wise choice? I deliberately chose this one because it's fanless. But it might burn the Xeon?

If you hesitate it exist also semi-fanless PSU. At xx% load the fan begin working. Like the Corsaire RM450 PSU.

* Should I buy extra fans for the NODE 304 case?

The Fractal Silent Series R2 fans are quiet fans but not the quietest. If you want to change I can recommend you the Noctua NF-S12a ULN or Noctua NF-S12B REDUX 700 to replace the 120mm rear fan and two Noctua NF-R8 REDUX 1200 to replace both front fans. Beware that these fans favors quietness over efficiency, they are'nt the best quietness/efficiency ratio.

For the CPU cooler you can maybe think about a passive one. Like the Zalman's FX70 or Nofan CR-80EH or ...

* I'm guessing the new server will be running idle for most of the time. The max. TDP for this Xeon is 80W but will consumption drop significantly when running idle?

I don't know if you found it, but it exist a low power consumption Xeon only 18W TDP. Xeon E3-1220L v3. Certainly less powerful with only 2 core/4 threads @ 1.1Ghz/max 1.5Ghz.

There exist also several intermediate CPU between Atom and i3/Xeon. Like the Pentium G3220/3240/3420/3440 or their low energy equivalents G3220t/3240t/3420t/3440t (TDP 35W). These are recent CPU, so always check the MB compatibility first (don't forget the Bios version also because without a CPU you can't update a bios MB ...). I don't know their FreeNAS efficiency.

* The Xeon has VT-d support, the Atoms don't. Is VT-d important for running VMs?

I think yes.

 

[cyberjock]

Vt-d doesn't matter for running vms via the virtualbox jail.

 

[anRossi]

I'm glad I searched first before posting, because I am actually planning out a build that is scary-similar. My build is a home backup server. I want it small so in case of emergency I can grab it and my cat and walk out the door. I want it quiet so I can leave it on always.

Good to know that the Node 304 with anything warm (e.g. that E3-1230 V3 at 80TDP) in it is going to be loud.

I was looking at the octo-core http://www.asrockrack.com/general/productdetail.asp?Model=C2750D4I#
because it's quiet, but I worry about the 32-bit Atom cores (even with 8 of them, I don't think there's enough parallelism in my use case to make up for their slowness)

According to Intel's fact sheet (http://ark.intel.com/products/77987/Intel-Atom-Processor-C2750-4M-Cache-2_40-GHz) it supports 64GB of RAM (4x what the Xeon-based ASRock Mini-ITX boards support) and it has AES-NI, which will make encrypted workloads a decent speed. But it's still 32-bit and Atom...

Any other comments on the Atom, or are my fears pretty accurate?

 

[DJ9]

I don't get the entire small case deal. Most small cases are wider than a full tower, it's just a height difference.

Another thing, besides cramming everything in a small case vs something larger with room to breath, the space required isn't that different. The bigger case has better cooling options, more drive space, and it will be quieter.

This is why god invented ethernet cable, the damn server can be located anywhere. (and renters, or those using wireless.. really)

 

[cyberjock]

Umm, they aren't 32bit... lol. 32-bit would be pointless with 64GB of RAM.. right?

 

[anRossi]

Cyberjock, check the Intel fact sheet I linked to. It doesn't explicitly call it out as 64-bit, and it explicitly calls out that it supports physical address extension up to 36 bits.

It supports addressing 36-bits of memory, which gives ~68 GB of address space, just enough for 64GB of RAM.
The OS would have to be 32-bit and support PAE as well to be able to address that much memory on 32-bits. Basically, there's an extra level of table indirection in the page tables and some special hardware in the MMU that understands this.

Edit: If it were 64 bit, it would not need physical address extension. Also, if you check 64-bit CPUs, they only offer hardware addressing of something like 40 bits (more recent ones might be 42 or 48). The MMUs in CPUs don't fully support 64-bit addresses yet. There's no reason to put 64-bits worth of memory inside a computer at this point, so why have all the hardware for addresses you wont use until long after the service life of the CPU is over.

 

[cyberjock]

I looked at the intel sheet:

Instruction Set = 64-bit

What's not to understand about that?

I also looked at the link to the motherboard:

Supported OSes:

Microsoft Windows
- Server 2008 R2 (x64)
- Server 2012 (x64)
Linux
- RedHat Enterprise Linux Server 5.5/6.4 (x32 and x64)
- CentOS 5.5 / 6.4 (x32 and x64)
- SUSE Enterprise Linux Server 11 SP1 (x32 and x64)
- FreeBSD 9.1 (x32/x64)
- Fedora core 18 (x64)
- UBuntu 12.04/12.10 (x64)
Virtual
- VMWare ESXi 5.1 (Marvell SE9230 and Marvell SE9172 has no driver can support VMWare)



Sure as hell sounds like 64-bit. I also am using this motherboard so I do know better. ;)

 

[anRossi]

Oops, well allow me to open my mouth and insert my foot. I apologize for going off there.
I was looking at a comparison sheet with a couple other processors and the 64-bit field was in a different place for the atom than it was on the other two.
I've been reading spec sheets too long when I miss things like that. I'm embarrassed.


So... if you're using the board, would you recommend it? What are your experiences?

 

[cyberjock]

I won't lie.. I was *very* confused. I was like "did he have a few too many beers or does he need a cup of coffee?"

It is the board that the FreeNAS Mini uses. So I'd say it's a good board.

Read my review of the Mini (which focuses mostly on the performance of the CPU):

http://forums.freenas.org/index.php?threads/cyberjocks-review-of-the-freenas-mini.20724/

I think it's a very good board. It does have some performance limitations. If you plan to run a bunch of VMs and/or jails that are CPU intensive you *may* want to consider a full fledge Xeon instead. ;)

 

[Caspar]

Hi guys, wow lot of suggestions/ideas and good to know there's someone with more or less the same wish list. I guess I'll have to rethink the location of the server. I must say the more I read about the Xeon the more I want it. I found out that the E3 1230V3 also has hyperthreading (= more expensive ofcourse but not much).

I had thought about SSDs, I already use them in my desktop computer (256GB) and in my VU+ Solo2 (120GB in a satellite receiver) and they work excellent. I did say price was #5 right, but no, can't afford that much money for my 2 TBs.

I also need to reconsider the RAID options. I've been wondering why people would buy that many 1TB hdds, now I get it!
So thnxs, I'm heading back to the drawing table. I think I'll look into desktop cases or even rackmount-like cases, that might fit somewhere else (so probably no towers for me). Be back soon.

 

[anRossi]

After reading your review I think I'll probably go ahead with that. But the memory quality list is ... hard to find the parts for. Could you give the part numbers for the memory sticks in your mini?

 

[cyberjock]

If I remember correctly it's one of the 8GB sticks that's on the compatibility list. Unfortunately I really can't take the time to pull the mini out of the corner it's been placed in to look inside. :(

 

[anRossi]

I discovered that the pics you took in the review are high enough resolution that I could read the part number off the sticker. :)
They are samsung sticks, which aren't on the compatibility list. These, according to the sticker: http://www.superbiiz.com/detail.php?name=D38GE1600S

 

[cyberjock]

Hmm.. I wonder if they were taken off the compatibility list at some point. iX doesn't generally make a habit of diverging from compatibility lists.

 

[diedrichg]

If all you are doing is storing home encrypted data, file streaming and running it as a personal Git server; you may not need to full power of a Xeon. Check out the Intel Pentium G3420. It may suit your needs. It's cost effective, it has a low TDP and it's still powerful enough for a home server with only a few users utilizing it at one time for a few hours out of the full 24.

 

[anRossi]

Cyberjock,

would it be risky to trust the memory manufacturer's memory validation list?
http://www.kingston.com/us/memory/search?DeviceType=7&Mfr=ASR&Line=Server Board&Model=87323

I've never had a problem with a (consumer/desktop) system and I always use Kingston's memory search. With a server, I understand it's more picky with memory compatibility, so is the list provided by the memory manufacturer (in this case, Kingston) trustworthy?