Newbie - Hardware FreeNas assembler suggestion request

[LightBroPT]

Hi,

I'm a newbie freenas "fan".
I would like to build my first stable freenas box with 4x3T HDD and 64GB ram.
I have a +-€1500 budget.
Could you help me in my decision?

Thank you

 

[icsy7867]

Is that +-€1500 budget with or without drives?

Not to be one of those guys that just posts a link to a thread...
https://forums.freenas.org/index.ph...ll-a-proper-home-freenas-setup-cost-me.28936/

But this post will explain stuff way better than I can! But I would recommend (Generalized)

A quad core Xeon (Especially if you are planning on running jails or bhyve VM's) ~$250
A SuperMicro motherboard, preferably with IPMI for remote/local system management $200-$300
64GB DDR3/4 ECC Memory. (Although for 12TB of raw space you will probably be fine at 24+ GB of ram.) ~$150-$200 (@3x 8GB)

You may want to do some more reading on FreeBSD/Freenas 10.3 with the new skylake processors and DDR4, last time I was reading about it, it was still a bit of an unknown. I am also a fan of the Fractal Design R4 cases (~$100). Plenty of room, plenty of fan space and built in dust covers!

3TB WD Red Drives are about $110 each (You could save some money and go Seagate if you wanted)
~$330

Totaled to Roughly
~$1180 (In USD though! Not sure what the exchange rate is)

 

[LightBroPT]

Thank you for your fast reply.

Converting to USD... around $1800 with drives

 

[Mirfster]

Are you partial to a particular form factor (Tower, Mini-Tower, 1U, 2U, etc)? What is your "Use-Case" going to be?

 

[Evertb1]

I don't know if you already started buying stuff but this was my shopping list for my build
Motherboard: Supermicro X10SL7- F € 254,00
Memory: Samsung ECC DDR3L-1600 8GB x 2 (just enough for the moment and I have 2 slots open for expansion next month) € 94,50 for the pair
Processor: Xeon E3-1226 v3 (outlet deal) € 189,00
HDU's: WD Red 2 TB 8 x € 680,00. My budget was not big enough for bigger drives and I wanted to build a pool with 8 drives (4 pairs in mirror)
Case: Sharkoon T9 Value blue edition € 64,90 (flimsy thing but I had my reasons)
Swap bays for HDU's: Icy Dock MB154SP-B x 2 € 159,80
Together a whopping € 1.442,00

Some considerations:
I bought the processor because it was priced very nice, but I am sure that any Core i3 with ECC support (and hyper threading) would be sufficient in a lot of use cases.

My choice for the motherboard was about three or four things. The brand has a good reputation. The board is equipped with a very nice SAS controller. The board has three NIC's, 2 of them Intel and one of them dedicated to IPMI (very nice to have). And the price was not over the top. By the way the SAS controller on this board has 8 SATA connectors on the motherboard so you don't need to buy break out cables. If you ever need to expand your controller you just buy a reverse break out cable and connect a SAS expander Card (that is not in my future I am sure )

Why the (for todays standards) small HDU's? My need for space is modest. So about a little less then 8 TB of space will keep me happy for the coming three years or so. The last couple of weeks I have done al lot of reading and decided that working with mirrored pairs was it for me. Four pairs in a pool offers a reasonable data safe situation (but still: back up, back up, back up). Buying 3 TB or bigger drives would be above my budget (say at least € 200,00 more).

The casing was a different story al together. I found it very hard to find a nice, modest sized, reasonable priced case, offering room to put in the swap bays I wanted. The Sharkoon offers 9 (!) 5,25 inch external slots in a modest package. I did not care for the fancy fans with blue led's and the case is a bit flimsy. But now I have done the build it's doing a pretty good job. I still had some Noctua case fans so I swapped them and offered the led ones to my son (happy smiles al around). I also had some dampening material that I used on the case.

 

[R.G.]

I have a running FreeNAS with a Xeon processor and Supermicro motherboard. It's been running essentially non-stop for nearly two years. It correctly told me when a disk in the array was dying and resilvered the disk properly, making a disk crash an interesting event, but no crisis and no data loss.

Where I differed from your approach was using seven 3TB drives in RAIDZ3 (I'm a little paranoid about disk failures) and using only 16GB of ECC memory, as well as a different case. The case was what got me to reply to this thread.

I bought a Fractal Node 804. I am very, very happy with it. It is one of the small cube-shaped enclosures, and has proven remarkably suited to a NAS IMHO.
It is internally divided into a motherboard chamber and a disk/power chamber. The disks hang in two clusters of four on slide-out holders. I used seven of the eight spaces for my disks. The manufacturer says there is space for two more 2.5" disks and two more 3.5" disks. I didn't need that many.

Although the disk mounting is quite compact, the disks run about 33C to 35C with 25C room temperatures as long as I clean the (included!) dust filters. It has space for ten (!) fans; I'm using four, one for the CPU side, three for the disk/power side. In spite of that, I can barely hear the fans when I'm in the room. It's quieter than the other computers I run. There's a lot more to it. Look here: http://www.fractal-design.com/home/product/cases/node-series/node-804 It cost me US$89 on sale, but seems to run about US$100 these days when not on sale. I don't know what your local conditions would do to that price.

I use my NAS primarily for backing up the other computers in the house, not for serving video or running VMs or other stuff. It's a data repository, so my concerns were for data reliability, not performance. To me that meant going for a reasonable amount of storage, with high tolerance for a single fault. That pushed me to RAIDZ3. With seven 3TB drives, I have 12TB of storage and can tolerate losing any three disks. Another variant would be seven 2TB drives for 8TB of storage in RAIDZ3, or six 2TB drives for 8TB in RAIDZ2. Right now, I can get 2TB drives for ~US$50 each, so the six-drive version would be ~US$300, and the seven-drive version would be US$350.

I have not seen any issues with using only 16GB of memory. More is probably better, but I can tell you that such a system runs well for me. Again, I don't have applications configured in jails, and I'm not performance constrained.

 

[philhu]

I just put together my first FreeNas using a Super Micro chassis SC847 and motherboard x8DTN+

The chassis has *36* disk slots, each supporting up to 7tb drives. Dual Xeon 4 core cpu E5560. The chassis and motherboard and card, all working with dual power supplies was $700, so deals are out there.

I bought 8 4tb drives (+ 3 on the shelf) so I can transfer from my QNAP NAS. When the data is done, I will take the QNAP drives (16x4tb) and put 11 in the new chassis as another vdev, when done, and ultimately, put a 3rd vdev of 11 drives so I will end up with one huge volume holding 3 11 disk pools in RaidZ3 mode on each, giving me about 84TB when done. So on 33 drives, 9 are redundant raidz3 drives, which is a bit overkill, but my data is so worth it.

I did add a usb3 4 port pci controller, an extra 2 1g nic slots on a pci/e card and a 2 port 10GBE pci/e card as well as a scsi card with my 24 slot tape loader, running Bacula in a Jail, which was running on the network to another box, now runs native/local so runs 3x faster! also, in the process of taking my 2 VM's from the old dell esxi server and moving them to the FreeNas box using virtual box. Plex/Pytivo server.

This machine is supposed to run all of this in under 500 watts! My old Dell 2950, used for vm use only, was 1650 watts! The new machine seems to hover at 650, as it is a newer energy saver model, and since it mostly idles most of the time

 

[Evertb1]

Where I differed from your approach was using seven 3TB drives in RAIDZ3 (I'm a little paranoid about disk failures) and using only 16GB of ECC memory, as well as a different case. The case was what got me to reply to this thread.

Being paranoid about disk failures is something I can fully understand. Been there doen that. And most likely it will happen again sooner or later. The reason for using the mirror vdevs was this article: http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

And about the case: I just like to keep the swap bays. Being able to disconnect my drives without to much trouble is nice. Just yesterday I flashed the bios of my diskcontroller and removed the drives before doing that(talking about paranoid :) )

 

[R.G.]

Each to his own taste. I have a couple of Norinco monster cases that I set up originally, and wound up condensing down into the Node 804. It's a highly limited solution in terms of number of drives, stops pretty solidly at ten. But my needs for number of disks stops at seven. No need for the monster's huge number of drive bays, and the Node is quieter. I'm good at 12TB for the moment, even with room for expansion. Others may have higher storage needs.

I read Salter's article a ways back. He makes some good points. Probably the best points are that you can increase your pool in steps of only two disks of any give size, not many. IIRC, the thing he glossed over a bit was the crux of the issue. He was emphatic about being able to rebuild a replacement disk in a mirror "really fast".

I'd have been a lot happier if he'd quantified "really fast". :) The critical issue of course is whether you'll get enough disk failures in the vdev to lose the data. For a mirror, this is easy; you estimate the probability of a disk failing over any time period. If one disk has failed, your chance of losing the entire Vdev is the probability of losing that many more disks. The article properly puts this at the probability of losing one (the only remaining!) disk in the time remaining to resilver a new disk. For a mirror, you're betting your data that the resilver time is shorter than the time to the next disk failing.

For a RAIDZx, the probability is the probability of losing a second disk before the first loss can resilver, times the probability of a second loss given that one disk has already failed for RAIDZ1. The resilver time is indeed longer for a RAIDZ1 than a mirror, and there are a number of disks that can fail, so the probability of failure is larger by a factor of both the number of disks and the longer resilver time. For RAIDZ2, that's the probability of a THIRD disk going down times the probability that two have already died, in the resilver time. For RAIDZ3, it's the probability of a FOURTH disk dying during one resilver time for the array, given that three disks must have already died. I would have welcomed a clearer statement of the probabilities.

All parts can have a simplified estimate of the probability of failure in a given time frame. Disks, by and large, have a quite low probability of failure in any given small time interval. The bigger the time interval, the bigger the probability, and vice versa. Also some realistic estimates backed up with real-world data on resilver times.

I haven't done the calculations (obviously), or I wouldn't be winging this. But my gut tells me that it's better to bet that three or four disks won't die on the same day, let alone the same "resilver time", than to bet that a second disk won't fail while I'm refilling a new one. I may have to go get educated on the topic and run my own calcs.

Meanwhile, I'm waiting on a replication of my RAIDZ3 NAS to a purely-backup RAIDZ2 NAS backup. Once the replication finishes, I'll have both a RAIDZ3 front end, and a RAIDZ2 "mirror" of that. :) What are the odds??

 

[LightBroPT]

Thank you all for your support...

Regarding some questions before, hera are something important for me:

- I was thinking in something like a 4 bay mini-tower like FreeNas Mini like
- 4x 3TB HDD WD Red
- The energy consumption is very important because it will be running 24/24 but at home.... energy cost is very important for me but a stable and good power supply to support it...too

Thank you again

 

[philhu]

My data is important to me

So I am using 2 sets of 11 drives, one set 4tb and the other 6tb, in RadiZ3

So, I end up with 8x4tb + 8x6tb = 32+48= 80TB. I currently have 47tb of data

My case has room for another set of 11, so in a year or so, maybe 8tb will be lower

So my massive case has 36 slots, and running full blast, it uses 280w or $23/month of electricity at my rates. It replace an old dell 2950, which used 1777 watts. And the freenas box did all the nas, as well as virtual box, and 2 jails, for software that was on the retired dell machine.

All in all, I am very happy

 

[Evertb1]

I am pretty much amazed about the massive amount of storage some people need. But I believe you can't do much wrong for private use if you build a Freenas Mini like box. It's a nice small form factor and there are cases out there looking pretty good (Chenbro ES34069T-120 with 4 swap bays or the SilverStone SST-DS380B -offering even 8 bays- just to name 2 of them).

The only problem is that most of them force you to a small form factor mobo (Mini DTX or Mini ITX) or even an non ATX form factor for the PSU.

But there are some good choices available out there. With your budget I am sure you can build yourself something nice.

I have build my own Freenas box with some new parts and parts I already owned. Would I do a complete new build, my shopping list would look something like this:
Case :SilverStone SST-DS380B 167,90
PSU :SilverStone ST45SF-G, 450 Watt 99,90
Motherboard :ASRock C2750D4I (integrated 8 core CPU) 499,00
Memory :Kingston ValueRAM 8 GB ECC DDR3-1600 (2 x) 96,00
USB Stick (OS) :Sandisk Cruzer Fit 16 GB 18,00
HDU :Western Digital Red, 3 TB (4 x) 415,60
UPS :APC Back-UPS BX1400U-GR 164,90
-------
Total 1461,30
As you can see with your budget there would even be room for a UPS (not specifically the one mentioned because there is choice enough). I feel you should always take an UPS in your budget if you don't own one already.
I am also sure that I could find some of the things for a lower price. But this prices are from a main web store (Alternate).

What ever you decide: success with your build

 

[LightBroPT]

Hi, Evertb1,

Already have a 1500VA APC UPS.

No problem with the Marvell sata ports? How about processor performance other than the main storage process? Video, VMs? I want to have some performance for it, too

 

[Evertb1]

Hi LightBropt,

Already owning a UPS is good. More room for you on your budget.

Now about that Motherboard. I would select it with my use case on my mind. It would be perfect fit for me. Especially because it offers a lot within a small form factor. One of the great things it has is the IPMI interface on a dedicated NIC. I have IPMI on my Supermicro motherboard and I love it. But with your use case in mind you could very well come to a complete other conclusion.

Those Marvel Sata ports: you here different things about them. Some good some bad. I don't own a motherboard with them one so I have no experience. On the other hand: the motherboard has also an Intel controller on board. Good for 2 x SATA3 6.0 Gb/s, 4 x SATA2 3.0 Gb/s. For my personal build (if I would start al over) I could also decide to buy an IBM M1015 controller and crossflash it to IT modus. They are not hard to find and reasonable priced. Flashing them is easy (I already did this for a friend and was done in 10 minutes). That would give you another 8 Sata ports with the help of some break out cables (they ar not expensive and ar nice for cable management as well). This card is very popular in the Freenas community. Again, this is what I would do for my use case.

With video I guess you mean doing the Plex thing and transcoding etc. The processor is rated for around a 3900 passmark score . I know, not the most powerful CPU there is. But to transcode a 1080/10 MBPS stream you need around a 2000 passmark score for each stream. So if you don't want to do this for an whole household at once, you should be good.

I have no clue about running VMs and if this processor would be sufficient to run a bunch of them. I hardly run them myself. I have a dedicated HTPC and if I have a need to run some OS to play around with I have a spare desktop.

I would like to make another point. You want to have 64 GB memory. The 1150 platform is pretty popular within the Freenas community I think. But as far as I know most of the mobo's for this socket support 32 Gb max. The Asrock supports 64 Gb.

But keep in mind: After all is said and done it is your use case that should be your lead in building your Freenas box.