BUILD Noobs first built - Info I used and thinking process

[Thorsten]

Hi Guys

I am in the process of building my first FreeNAS and so that other noobs might profit a bit from this experience and maybe some experienced users might comment on the selection of the parts for my built, I wrote down which sources I used for information, what I found out, the parts I selected and why I selected them.
As well as some questions I have left of things I am not 100% sure about.

Sources of Information:

I have read the forum post about hardware, http://forums.freenas.org/index.php?threads/so-you-want-some-hardware-suggestions.12276/ ,

the guide for noobs http://forums.freenas.org/index.php...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/ ,

and read threw most of the documentation concerning hardware at http://doc.freenas.org/index.php/Past_Releases .

Summary from the information I found:

To summarize what I took from these sources:

- I need a Xeon processor for ECC RAM and encryption.
- I need ECC to make sure my ZFS file system does not get corrupted
- I need at least 10 HDD as 6 HDD are not enough for me and I want to to do RAIDZ2 as RAIDZ1 will not be enough (too much risk when 1 drive fails and is being rebuilt).
- to find out how high the capacity should be of the drives (2 Tb or 3 TB) I used this calculator http://www.servethehome.com/raid-calculator/
- I need a proper NAS harddrive like the WD Reds as with the Green I would have to hack them a bit like adjusting the idle settings etc. and as I want t make sure I can use the warranty and don't feel like doing too much hacking so Red instead of Green seems to be the better choice.
- I need at least 1 GB RAM for each TB of HDD as ZFS needs a lot of resources
- I need to use a memory selector from a manufacturer to make sure the RAM is compatible with the motherboard
- FreeNAS should run from a USB device with a size of at least 4 GB as FreeNAS is about 2 something GB and with 4 GB the wear of sectors can be spread over the device. I also read the USB device should be run in READ only but not quite sure how that works maybe thats something that can be configured in FreeNAS.
- I don't need SSD as I don't need to do any of the fancy caching things like ZIL and L2ARC as for now the system will probably give me enough performance without caching
- I should use a server motherboard from i.e. Supermicro: Actually I am not sure why, but sounded good, especially having IPMI to access the server even in BIOS from another PC seems very useful, and apparently such a motherboards might use less Watts
- For the system to use less watts the PSU should only be running at 30% to 50% of capacity
- The efficiency of the PSU should be at least 80 Plus Gold or Platinum
- To make sure the data does not become corrupted when the power fails I need a UPS (Uninterruptible power supply).


So what do I want the server for?

I pretty much will have most of the time only 1 computer connected to the NAS, maybe more in the future but never more than a maximum 10 in the future with my family and each having about 2 devices.
I will want to do some Video and Audio streaming from the NAS and do backups to Crashplan.
Also I want to encrypt all the data on the NAS.

Built:

Knowing this I decided on the following components:

CPU: Intel® Xeon® Processor E3-1220 v3
Motherboard: Supermicro Mainboard X10SL7-F Single
Case: Fractal Define XL R2
USB: Kingston 4 GB
RAM: 32 Gb Kingston Server Premier ValueRAM DIMM 8 GB ECC KVR16E11/8KF
Harddrive: 10 x WD Red 3 TB
PSU: Fractal Design Newton R3 1000W
UPS: EATON Protection Station 800 USB
DVD-Drive: LG Drive
Anti-Static mat and wrist band for building

Thinking behind each device:

Intel® Xeon® Processor E3-1220 v3
I wanted a CPU which could use hardware AES encryption and support ECC RAM

Supermicro Mainboard X10SL7-F Single
A server motherboard which supports my CPU,
Has 8 SAS2 connectors, and 6 SATA connectors so it can support my 10 SATA drives
(I read I can connect SATA drives to the SAS connectors)
And has IPMI so I can connect to the NAS BIOS and running system from another computer, never used it but sounds awesome.
Supports ECC
Has a USB 3.0 connector (you never know if one might want to connect an external drive to transfer some data to the NAS).
Has Dual Gigbit Ethernet don't think I need dual but hey better 2 than none
And this way I don't think I need a NIC.

Case Fractal Define XL R2

Supports 8 drives
Offers noise reduction and placement of up to 7 fans
I can add more drives using 5.25" to 3.5" converters

USB: Kingston 4 GB

The reason I explained above.

RAM: 32 Gb Kingston Server Premier ValueRAM DIMM 8 GB ECC KVR16E11/8KF

I chose this RAM using the Kingston memory selector
http://www.kingston.com/en/memory/search/options/
And the Server Premier memory even though I believe its better than the normal ValueRAM is currently cheaper.

PSU: Fractal Design Newton R3 1000W

First I used the Watt calculator here http://extreme.outervision.com/psucalculatorlite.jsp to find out how many watts I needed. Which resulted in needing at least 358 Watts. Which means I needed at least a PSU with 720 Watts.
But the final reason why I chose this PSU was that I need at least 10 connectors for the SATA drives and this PSU offers 12. This was a very important point I almost looked this over and chose a PSU with not enough SATA connectors.
The PSU is from the same manufacturer as the case so I thought this makes sense
Also this PSU has an 80 Plus Platinum efficiency rating.
Other than that I wasn't sure what else I should look out for with the PSU.

UPS: EATON Protection Station 800 USB

This is a very lightweight UPS which has the battery built in and offers 500 W.
Of course this is not 1000 W as my PSU but as I calculated the system only used about 358 Watts I believe this should be enough.
Also this device has a USB connection which should be compatible with the Network UPS Tools (http://doc.freenas.org/index.php/UPS) which come with FreeNAS and such I should be able to configure FreeNAS such that it triggers a proper shutdown once the main power is gone.

Other things:

I DVD drive to upgrade FreeNAS, and might beusefull for other reasons.
Anti Static mat and wrist band to make sure the parts do not get damaged during the built.

Things I am not quite sure right now about this built:

Can I connect the SAS2 connectors on the motherboard with SATA WD Red drives?

Do I need need a NIC (Network Interface card) when the motherboard comes with Gigabit Ethernet?

Is the UPS enough for my purpose / is it compatible with FreeNAS so when the power is gone the UPS will trigger a proper shutdown of the system?

Ok I hope I didn't forget anything

Any feedback about the chosen parts especially about the things I am not quite sure about or what else I should think about is very welcome!

All the best

Thorsten

 

[Starpulkka]

You should add that hdds operating temperatures should not exeed over 35c. Your computer case has good fan places to hdds so its ok. Also mention to 40kg firewall would be good to put front of freenas.

As for psu and memory and all else i dont say nothing.
Also ok summarum but you do realise that this post will be on second page within weeks where no one looks anymore..

 

[Thorsten]

Hi Starpulkka

A 40 kg firewall? Whats that? :-D Well I guess you mean a strong one
Well the NAS will be in my home network, which has a router in front of it and the WLan has a long password so I think it should fine?

Yeah I know this will be second page but maybe people will find it by searching I also found other posts which helped me and where else could I have posted it?

 

[Rand]

>- I need a Xeon processor for ECC RAM and encryption.
For ECC an i3 is sufficient, for encryption you dont *need* a cpu which supports AES-NI but it helps;)

>- I need at least 10 HDD as 6 HDD are not enough for me and I want to to do RAIDZ2
You can run RAIDZ2 with fewer drives o/c, depends on how much space you need.

>- I need a proper NAS harddrive like the WD Reds as with the Green I would have to hack them a bit like adjusting the idle settings etc.
Selecting drives depends on factors like size, price, warranty, cooling options, noise; WD Reds are not the ultimate drive for anyone here.

>- I should use a server motherboard from i.e. Supermicro: Actually I am not sure why, but sounded good, especially having IPMI to access the server even in BIOS from another PC seems very useful, and apparently such a motherboards might use less Watts
The relevant point here is stability. Server boards tend to be tested more thoroughly, will support ECC Ram and IPMI is a nice bonus. Don't expect minimized power usage.
Supermicro is a known good brand which many people use. It's not necessarily the only usable brand.

>- For the system to use less watts the PSU should only be running at 30% to 50% of capacity
For the best efficiency you should use the chosen PSU at its highest efficiency value, thats not necessarily @30-50%. A way overpowered PSU might run at good efficiency at 50% full utilization but will run inefficiently when its idle.
Most systems idle most of the time & generally use way less power than you'd expect, so most people will be fine with a gold level 400W PSU.

>Can I connect the SAS2 connectors on the motherboard with SATA WD Red drives?
I believe so but why buy a SAS capable Mainboard if you dont need it?

>Do I need need a NIC (Network Interface card) when the motherboard comes with Gigabit Ethernet?
Depends on the NIC - if its a good brand (basically Intel) then no. If its Realtek - then its recommended for best performance.

 

[Thorsten]

Hi Rand

Thanks for your answers

>- I need a proper NAS harddrive like the WD Reds as with the Green I would have to hack them a bit like adjusting the idle settings etc.
Selecting drives depends on factors like size, price, warranty, cooling options, noise; WD Reds are not the ultimate drive for anyone here.

So what might other options be? Whats bad about the WD Reds? What should I be looking for? What would be the "ultimate drives" remembering that this is just a home NAS being used right now only be 1 user on weekends. I am not looking at a high performance server system here right now. Pretty much just one with lots of storage. And the drives being "ok" for a NAS.

>- For the system to use less watts the PSU should only be running at 30% to 50% of capacity
For the best efficiency you should use the chosen PSU at its highest efficiency value, thats not necessarily @30-50%. A way overpowered PSU might run at good efficiency at 50% full utilization but will run inefficiently when its idle.
Most systems idle most of the time & generally use way less power than you'd expect, so most people will be fine with a gold level 400W PSU.

Well the biggest reason for this PSU was so that I can power at least 10 SATA drives.
Maybe some smaller PSU also offers enough SATA connectors but I guess the question really is how much power my system uses while running and maybe I should get a smaller PSU when I realize it doesn't use so much power.

>Can I connect the SAS2 connectors on the motherboard with SATA WD Red drives?
I believe so but why buy a SAS capable Mainboard if you dont need it?

Well I looked at SuperMirco Motherboads for the LGA 1150 socket and this one is the only one as far as I could see which could suppoer 10 harddrives so this is the one I chose.

>Do I need need a NIC (Network Interface card) when the motherboard comes with Gigabit Ethernet?
Depends on the NIC - if its a good brand (basically Intel) then no. If its Realtek - then its recommended for best performance.

I double checked and it says in the motherboads specs "Dual Gigabit Ethernet LAN ports via Intel® i210AT" so since its Intal I guess thats fine.

 

[Rand]

Well on one hand you want to provide a generic guide for noobs which shouldnt point to specific components as ultimate choice if they are not;)
So for you its the fitting choice;)
WD Reds are ok, but not the only choice. I use some cheap Toshiba drives, when they break i get new cheap ones; works for my purpose with Mirrors and proper backups but might not be ok for someone else.
PSU - The relevant point is *not* the number of connectors - there are adapters for that, but total power and amperage for startup (provided you dont use an adapter with staggered spinup support)
You dont need all ports on the mainboard. There are adapter cards (HBAs SATA or SAS like M1015) for this
Yes most serverboards nowadays use intel nics

 

[ser_rhaegar]

You will most likely still need to modify the red drives with wdidle3 and a patch utility from WD. Search for cyberjock's thread on red and green drives.

Also I thought there was an incompatibility right now with Kingston 8GB ran modules and the X10 series motherboards. There should be a thread here on that as well with specifics.

 

[indy]

Things I would take a look at:
- the PSU is monstrously overpowered, dont choose it by the number of connectors
- 10 drives fit in the smaller Define R4 (minus the DVD-drive)
- DVD-drive can be emulated with IPMI
- if you do not need 4 cores the Haswell i3 series has AES and ECC as well
- 16 GB RAM might very well be sufficient

Regarding your X10SL7-F questions:
- The onboard SAS-controller works fine with SATA drives, flash it to IT-mode for best ZFS compatibility though
- Onboard NICs work great with FreeNas

 

[Thorsten]

- the PSU is monstrously overpowered, dont choose it by the number of connectors

Ok, yes it is overpowered, so do yo think I should choose a PSU with about 720 Watts since my system needs about 358 Watts?
Or how do you thik I should choose a PSU?

- 16 GB RAM might very well be sufficient

I am surprised that you are saying this. Isn't the rule: 1 Gb RAM fo every 1 TB HDD? If I choose 3 TB x 10 HDD I would have 30 TB and such need 30 GB RAM?
But I guess I could start with 16 GB and see how it goes.

 

[Rand]

Ok, yes it is overpowered, so do yo think I should choose a PSU with about 720 Watts since my system needs about 358 Watts?
Or how do you thik I should choose a PSU?

Your system won't be 'needing' 358 Watts, that calculator of yours is ****.... not very realistic.
I just configured a server mobo with only a cpu and it tells me I'd use 154 Watts? Not even with 90% load which they recommend??? Crap.

First of all, you need to know your maximum usage. If you say you can peak at really using everything 100% in your server then thats what your PSU needs to have as max (+ 10% give or take).
As i said, a home server is idle most of the time, so max load is ***not*** the most important value to save power.

WD Red 4TB - 4,5W average Power on r/w > * 24 = 108W - just make sure you have 24*1,75A surge covered on spin up - or got M1015 or similar which do support staggered spinup - really a recommendation if you plan on so many drives.
CPU, Board, Ram, Fans > Max 200W and thats very very high.

So where are the 360 Watts the calculator claims?

I currently run a E3 1230v3, 16 GB, 8x3,5 drives, 5x2,5", 5 heavy duty fans and 30W's worth of PCIe cards on 130W idle - i'd be suprised to see that go above 200W on load - and i am using a 300W PSU.

I am surprised that you are saying this. Isn't the rule: 1 Gb RAM fo every 1 TB HDD? If I choose 3 TB x 10 HDD I would have 30 TB and such need 30 GB RAM?
But I guess I could start with 16 GB and see how it goes.

It always depends on your workload. Start with 16 and if you really have a large working set go ahead and fill it up.

 

[Thorsten]

I have now tried to find a PSU with about 300 Watts, but I have a difficult time finding one with at least 10 connectors for harddrive (IDE / SATA/ whatever)?

Can anyone recommend a manufacturer / PSU or website which I can use to find a suitable PSU?

The search seems simple: PSU should offer about 300 Watts and be able to supply at least 10 SATA drives with power but the search seems more difficult...

Or is there another way to get power to these drives? Rand mentioned something about adapter cards?

PSU - The relevant point is *not* the number of connectors - there are adapters for that, but total power and amperage for startup (provided you dont use an adapter with staggered spinup support)
You dont need all ports on the mainboard. There are adapter cards (HBAs SATA or SAS like M1015) for this

Or did he just mean cards instead of connectors on the motherboard?

 

[Rand]

3 issues to consider (in order or relevance)
-Power usage at startup
This depends on the type of harddrive, typically one would assume 2A here per drive - in your case you can
a.) buy a power supply which supplies that much power on one rail or two rails (if you split the connectors up properly) + spare for other 12V components like add on cards
b.) Use staggered spin up to have the disks start in groups (eg 4 at a time) reducing the overall stress on the PSU at startup; this needs to be supported by your controller (sas onboard - no idea), sata/sas hba - usually yes
-Total Power usage
This needs to cover all components with some spare but that shouldnt exceed 300W usually unless you have lots and lots of addon cards and tons of old drives
-Nr of connectors
There are adapters to split a connector up in multiple connectors 1-2 or 1-5. Thats *not* an issue;)

 

[cyberjock]

I would never, ever consider 10 drives on a 300w power supply. This is probably why you won't find a power supply that small with that many connectors. The manufacturer doesn't WANT you to plug in that many drives. That wattage is pretty low and starting current could really do a power supply that is that low in. I would go with a 400w+ minimum. Even then, I would probably look for a 500w or 550w just to cover the bases.

I think he was talking about sata power splitters.

 

[Rand]

Well i dont have issues with 10 drives on a 300W supply, the system runs at 130W (power meter) (drives idle) so nowhere near max power even if the drives would run full power.
I am using staggered spin up o/c but i dont see a reason to use a larger PSU.
But everyone his own:)

 

[Windreaper]

PSU cabling is selected according to typical usage scenario - not what the rails can support. 300W is more than enough for 10 drives. Just buy some Molex->SATA adapters (or SATA power splitters). Mind how the PSU rails are configured to balance the load.

 

[Lordadmiral Drake]

Well, as Rand said: Staggered spinup is the word here. This way you can use a lower rated PSU while keeping it able to start the system. Point is, HDDs draw a lot more current on spinup than during operation.

 

[cyberjock]

Assuming the hard drive and controller supports staggered spinup. Plenty of them don't.. even if enabled on the controller. :) This is why storage aficionados never ever ever rely on staggered spinup. A simple firmware update can suddenly make that feature not work and suddenly you're over-stressing your power supply on every bootup.

 

[Thorsten]

Alright thanks for all the feedback!
Because of the points mentioned I have made the following changes to the built:

CPU: I have selected the Intel® Core™ i3-4130T before I checked the system requirements of the plex plugin https://plexapp.zendesk.com/hc/en-us/articles/200375666-Stand-alone-Server
and they recommend that one should use at least a Core 2 Duo 2.4Ghz and according to CPU Benchmark.net such a CPU gets a rating of 1,565 while the selected i3 gets 4,218 so I should be fine to transcode HD content with this CPU.
PSU: I have now selected the BeQuiet SYSTEM POWER 7 300W. It can even deliver up to 350 Watts. So I think I should be absolutely fine at start up and even when I transcode video as this i3 only uses a Maximum of 35 Watts.

Case: I changed the case to a Fractal DEFINE R4 Arctic White as it is a bit smaller and I don't need a DVD Drive

DVD Drve: No DVD drive since I can use IPMI for that

Does anybody think this selection does not sound solid? Especially the selection of PSU? I believe this should be powerful enough but feedback is appreciated

 

[Rand]

If you are happy with Bronze level then the PSU is ok

 

[Thorsten]

Well so far I couldn't find any Gold Level, with 300 Watts?
Do you have one or know of any?