New build power efficiency + decent speeds

[-Adam-]

First of all, I would like to thank everybody involved in development and support of FreeNAS. For the past month I have been testing this solution on a different hardware and right now I have a FreeNAS server running in the office.

I have read many articles, guides (including great dummy guide) and forum discussions. So much knowledge!

What I'm thinking right now is a new build for my photography business. I need a lot of space with safe configuration. I will be working on the local ssd of my mac or directly on FreeNAS and backup will be stored on FreeNAS overt the network. There will be another share for client's zip files for download and a share for Apple Time Machine. Most of the time FreeNAS will be running idle. Shares will be configured as AFP and there will be no streaming, no plex, just simple read/write process of the files.

Till now, I have been using Lacie quadra hardware raid1 FireWire 800 enclosures with 3TB (2x3TB inside) of space - when the volume was full, I purchased another one. Right now, I'm on the verge of filling up the 3TB volume and since FreeNAS is working well in the office I think of my own build only for photography. Another aspect for FreeNAS is that with wired 1Gbps or wireless AC network I can work without physically attaching drives, which will much easier for me. Also with FW800 I was getting max. 70MBps and currently configured Xeon FreeNAS can give me even 110MBps.

The most important points:
1. The build needs to be as much power efficient (max. 50W idle) as possible with guarantee of decent speeds (90-100 MBps) when needed.
2. The build needs to be quite quiet since I will have it in my apartment.
3. The price is also a quite important factor, but I know that good technology costs money.

What I have chosen and my comments:
MOTHERBOARD: MSI B150M PRO-VDH- up to 64gb of ram, a lot of room for future upgrade. This piece can be changed when is not compatible with FreeBSD.
https://www.msi.com/Motherboard/B150M-PRO-VDH.html#hero-overview

CPU: INTEL I3 6300 - I have been thinking of i3-6100t for the lower Tdp, but after reading hardware guide I'm not sure that this will have much influence on actual wattage. Both intel processors have ecc support.
http://ark.intel.com/pl/products/90731/Intel-Core-i3-6300-Processor-4M-Cache-3_80-GHz

RAM: 16GB ECC - I think that 16gb will be ok for start - I can always buy 3 more of these: HyperX Fury Black 16GB [1x16GB 2133MHz DDR4 CL14 DIMM]

ENCLOSURE: fractal design 804- less important. One I connect everything I don't plan to change drives (fingers crossed)

POWER SUPPLY: be quiet! 600W Straight Power 10 CM BOX - I think that this is very important piece - the more efficient (gold is fair trade), the better, but I can not decide how much wattage to take. I think that for 6-12 (12 maybe in a few years) drives 600W will be enough. I don't want to buy huge power supply and have it take 100W idle...
http://www.bequiet.com/en/powersupply/521

SilentiumPC Fera 3 HE1224 (SPC144) - the cooling is very important, since too much fans will have an impact on noise and wattage and too less will have an impact on the durability of the equipment. Ideal solution would be to have a passive cooling for the CPU and one fan in the enclosure + one fan in the power supply, but can I go passive with i3? I have read many articles saying that the better solution is to have huge passive radiator and fan with 400rpm to move the air even a little.

6x3TB WD REDS - I have bunch of them in the office and never had problems. 5400rpm will generate less heat and less wattage. 3TB in Poland have the best PLN/TB ratio. I plan to configure Raidz2.

2x 16gb USB 3.0 flash drives (I have SanDisk cruzers in the office)

I was thinking of Intel Atom, which has significantly less TDS, but I'm not experienced in this technology and as CPU is not super important, I think that to be on the safe side, you should have a decent processor. Xeon is too expensive at the moment.

Please feel free to share your ideas and comments - I think that it is not that easy to combine good power efficiency and speeds with consideration for money spent on the equipment.

Best regards,
Adam

 

[Ericloewe]

MOTHERBOARD: MSI B150M PRO-VDH

Doesn't support ECC.

HyperX Fury Black 16GB [1x16GB 2133MHz DDR4 CL14 DIMM]

That's not ECC.

but can I go passive with i3?

You could, but it's a silly idea. CPU coolers are quiet, it's the drive cooling you should worry about.

You should probably read the hardware recommendations guide, linked in my sig.

 

[Kevin Horton]

The most important points:
1. The build needs to be as much power efficient (max. 50W idle) as possible with guarantee of decent speeds (90-100 MBps) when needed.
2. The build needs to be quite quiet since I will have it in my apartment.
3. The price is also a quite important factor, but I know that good technology costs money.

ENCLOSURE: fractal design 804- less important. One I connect everything I don't plan to change drives (fingers crossed)

I've got a Fractal-Design Node 804. It can be a very quiet enclosure, if you replace the included fans with PWM fans and incorporate some sort of active control of the fan speed as a function of hard drive temperature. One example script. Another example script. And a third script.

Power consumption - your target of 50W at idle will be difficult to achieve. As a point of reference, my Node 804 system consumed about 60W at idle when I had 6 HD in it. That is with X10SL7-F motherboard, G3258 CPU and 16GB RAM. If you really need 50W at idle, you may need to look at a Xeon D system, but that is a lot more expensive, and I think the reason for low power target is to save money. I don't know how much power costs in Poland, but here in Canada, I calculated that an extra 10W of power consumption would cost me $80 over a five year system lifetime. That assumes the system is running continuously. Conclusion - big power savings are worth an extra up front investment, but small power savings are not.

 

[GBillR]

Doesn't support ECC.

So I would qualify this statement... the board does support ECC ram (at least it claims to) https://www.msi.com/Motherboard/B150M-PRO-VDH.html#hero-specification

However, is specifically states that it only supports it in non-ECC mode.

So essentially, you "can" use ECC Ram, if you really like, but it's not going to actually DO anything for you.

 

[-Adam-]

Thank you very much for very important info for motherboard - I need to investigate it deeper!

You should probably read the hardware recommendations guide, linked in my sig.

I did, that why I'm not sure about lower TDP CPU's - great tutorial!

I don't know how much power costs in Poland, but here in Canada, I calculated that an extra 10W of power consumption would cost me $80 over a five year system lifetime.

I think that extra 60W would be ok. 10W over 5 years in Poland would be approx. $43 (btw. I didn't know that we have 50% cheaper energy :).
Kevin, what speeds do you achieve in your setup?

 

[Kevin Horton]

Thank you very much for very important info for motherboard - I need to investigate it deeper!


I did, that why I'm not sure about lower TDP CPU's - great tutorial!


I think that extra 60W would be ok. 10W over 5 years in Poland would be approx. $43 (btw. I didn't know that we have 50% cheaper energy :).
Kevin, what speeds do you achieve in your setup?

AFP was good for about 800 Mbit/s when the AFP share didn't have much in it. But, the AFP performance has degraded significantly on one share that has a lot of files, and varied between 100 and 400 Mbit/s. I eventually switched that dataset to use SMB for the share, which restored the performance to about 400 Mbit/s. The speed with SMB was limited by CPU performance. SMB is single-threaded, and one thread was maxed out at 100% when doing 400 Mbit/s. You should get better speed if you have a faster CPU than my G3258.

 

[-Adam-]

AFP was good for about 800 Mbit/s when the AFP share didn't have much in it. But, the AFP performance has degraded significantly on one share that has a lot of files, and varied between 100 and 400 Mbit/s. I eventually switched that dataset to use SMB for the share, which restored the performance to about 400 Mbit/s. The speed with SMB was limited by CPU performance. SMB is single-threaded, and one thread was maxed out at 100% when doing 400 Mbit/s. You should get better speed if you have a faster CPU than my G3258.

Thanks for detailed explanation. Didn't know that AFP has lower speeds when packed with files...

 

[MatthewSteinhoff]

The most important points:
1. The build needs to be as much power efficient (max. 50W idle) as possible with guarantee of decent speeds (90-100 MBps) when needed.
2. The build needs to be quite quiet since I will have it in my apartment.
3. The price is also a quite important factor, but I know that good technology costs money.

If those are the most important points, get another LaCie. Their 8TB model only draws 20 watts under load, achieves 210 MBps with USB 3.0, is nearly silent and costs just $450 USD.

Cheers,
Matt

 

[-Adam-]

If those are the most important points, get another LaCie. Their 8TB model only draws 20 watts under load, achieves 210 MBps with USB 3.0, is nearly silent and costs just $450 USD.

Thanks for your input, but there is many advantages to go into FreeNAS:
• when i purchase lacie it will be my 3rd enclosure and when I connect all 3 of them I will get the same wattage (3x20 = 60W) when idle.
• in mbp from 2013 I have only 2 usb slots so no go for 3 enclosures at the same time
• having all the photographs in huge freeness raid lets me access data from anywhere without having to physically connect to drives
• having 3 mirrored strips is less secure than 6 or 12 disk raidz2 (and in raidz I have more space than in mirrors)
• in Poland Lacie quadra 2x4TB costs $700. For twice the price I have a decent freeness with i3, server motherboard with ECC and 6x3TB WD REDS with approx 11TB of secured space (RAIDZ2) not mentioning that when I want to expand I can always but only two single drives and make a pair...
• Speeds - 100MBps is more than enough for work with pictures. I think it is even enough to work with video...

But anyway - thanks for your comment!

 

[wblock]

I would avoid MSI and Asus and other consumer brands with the possible exception of Gigabyte for that X150M Pro ECC board. But for this, a micro-ATX Supermicro motherboard, either socket 1150 if 32G of RAM is enough, or 1151 for up to 64G. Or maybe go for one of those Dell T20 servers that are available for $250 or $200 sometimes lately, and upgrade the processor later if performance is an issue.

 

[SweetAndLow]

My first build in my signature, the asrock itx one, used 40w at idle. I measured it using a kill-a-watt.

Sent from my Nexus 5X using Tapatalk

 

[-Adam-]

exception of Gigabyte for that X150M Pro ECC board

This Gigabyte is 1/2 price of supermicro with socket 1151. Thanks for suggestion!

My first build in my signature, the asrock itx one, used 40w at idle. I measured it using a kill-a-watt.

Thanks, what was the transfer rate? Did you managed to reach 1000Mbps?

 

[-Adam-]

Gigabyte for that X150M Pro ECC board

I have come across this topic from Dec 2015. Is this motherboard's NIC recognizable by latest FreeNAS version? How about the combination with i3-6300? Will the setup run ECC correctly?

 

[SweetAndLow]

This Gigabyte is 1/2 price of supermicro with socket 1151. Thanks for suggestion!



Thanks, what was the transfer rate? Did you managed to reach 1000Mbps?

That's kind of a silly question but yes it does. Basically any FreeNAS system you build can do a single stream or more at 1gbps. My pool in that build could write at 450MB/s and read at 475MB/s, this was on a mostly empty pool and streaming workflow.

There are lots of variables to consider though and your own workflow should be tested to see if it meets your required performance needs.

And for someone who asked about power consumption you are choosing a terrible board. It has worthless audio and graphics equipment that is going to use extra power. On top of that no ecc and a nic that isn't supported and can't do 1gbps.

Just follow the hardware guide and stop making this complicated.

Sent from my Nexus 5X using Tapatalk

 

[wblock]

I have come across this topic from Dec 2015. Is this motherboard's NIC recognizable by latest FreeNAS version? How about the combination with i3-6300? Will the setup run ECC correctly?

ECC memory and NIC should work. I've been tempted to buy that board just to try, but not quite enough yet. No reason to think it wouldn't work, just untested by me.

 

[-Adam-]

Hello guys!

After a few days of reading forum and digging into ECC support (including great post by cyberjock - https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/, which clearly explained why zfs+ecc is a must be) I have come to understanding that ECC is crucial for long-term work of zpool, therefore I will go for supermicro, which as far as I can see is the most reliable here.

I think that I will go for

• X10SLL-F-O
• i3-6300
• 2x8GB ECC RAM (always expandable to 4x8GB)

The last question is strictly regarding the configuration of the pool. Taking into consideration potential problems with RAM that can destroy entire pool and frankly, I don't have any influence on that except using ECC, I think more of splitting my future 6x3TB discs into 3 mirrors. In case of memory failure I will have only one pool destroyed, but the others will work fine. When I will put all 6 discs into RAIDZ2 pool, in case of memory error, entire pool will be gone - right?

 

[SweetAndLow]

Memory error will not destroy your whole pool. It will if you ignore the problems but not if you fix things when they go wrong.

Sent from my Nexus 5X using Tapatalk

 

[-Adam-]

Memory error will not destroy your whole pool. It will if you ignore the problems but not if you fix things when they go wrong.

Sent from my Nexus 5X using Tapatalk

Understood. I'm thinking more like another member of this forum from this thread: https://forums.freenas.org/index.php?threads/can-freenas-manage-multiple-zpools.18112/:

Thanks for the reply. I have 2 concerns about your suggestion though:

1.) the Cyberjock's guide specifically advises against eSATA drive use and especially single drive vdevs because a single drive failure would cause a loss of the entire zpool.

2.) My question is, can FreeNas manage more than one ZPOOL at the same time (not vdevs. I understand the vdev expansions). The guide only refers to adding additional vdevs to "the" zpool. I understand I can expand the zpool but again, if any vdev is lost, the whole pool is lost. For safety, is there an ability to manage multiple zpools (composed of vdevs) with the same FreeNas build so that "if" one zpool was lost, all of the server data is not lost (only the affected zpool is lost)?

Example:
Let's say I want to setup my photo RAW files and export images in a zpool of their own so their integrity isn't impacted by a catastrophic failure of the zpool that serves my generic files, video streaming files, etc. Can 1 machine and FreeNas build manage them as different volumes (not just vdevs of the same zpool)? I would expect the spools to be on different physical arrays completely. Something like this:

FreeNas build:

Zpool-1 (i.e. Volume 1)

- 4 x 3TB Raid-Z2 (holding all photo files)​

Zpool-2 (i.e. Volume 2)

- 5 x 2TB Raid-Z2 (holding all HTPC medial files)​

My personal opinion is that having 3x3TB zpools (mirror of 2x3TB each) is safer than one 12TB (RAIDZ2). From what I read in this forum and in many guides, RAIDZ is more complicated to handle in case of any errors than mirrors. I don't have practically any long-term experience with ZFS and can not tell what is more likely to happen - disk fail of zpool fail in case when we use proper hardware configuration.

 

[Ericloewe]

From what I read in this forum and in many guides, RAIDZ is more complicated to handle in case of any errors than mirrors. I

You misread. There are no complications. RAIDZ is just a bit slower due to its structure.

I don't have practically any long-term experience with ZFS and can not tell what is more likely to happen - disk fail of zpool fail in case when we use proper hardware configuration.

Disks are guaranteed to fail, there's no doubt whatsoever (unless they get retired before they die). Pools are not supposed to break down - if they do, it's probably the admin's fault.

There's a lot of RAIDZ hate on the forums lately and I'd really want to know where people are getting that idea...

 

[MatthewSteinhoff]

There's a lot of RAIDZ hate on the forums lately and I'd really want to know where people are getting that idea...

I'm not sure it's hate.

What I'm feeling is drive prices are getting ridiculously low - 4TB for $120. (Old dude here: my first hard drive was 20 megabytes for $600.)

With storage prices that low, mirrors are really appealing. Four drives in a mirrored stripe have twice the performance of RAIDZ2 with the same amount of available space.

I also see FreeNAS as being easier to expand using mirrors. In the above example, I can go from 8TB to 12TB in the same pool just by adding a pair of 4TB drives. (Though, with six drives, RAIDZ2 looks a lot more appealing; in a home or small office environment, space is usually more important than performance.)

{shurg}

Cheers,
Matt