NAS on a dime; Build review

[Chris Moore]

@Chris Moore I really don't want to offend you because you've been a great help and I know your intentions are good, but I honestly feel like you are insisting on a lot of features that really are just overkill for a family sized datatank. I really don't care for professional server grade options like IPMI or SAS controllers. Frankly this box is just going to be a glorified mediaserver that happens to do some NAS tricks because it can. Do I really need all that expensive stuff just for that? I know its not 'professional' to run a basicbirch 'gamer' board on a NAS, but will the arrays really care if there's an audiochip present? And is it really necessary to look for exuberantly expensive IPMI boards when I've got a screen/mouse/kb sitting there anyway?

I've been doing it 'wrong' for decades, with a spareparts scrap computer running under my staircase with windows 10 of all things, and its served me perfectly fine sofar. I just wanted something nicer and more purposely built this time around. Doesn't mean I'm looking to build IBM central down there though. So can't we gear this thread down to 'EcoNAS' speed like the title suggests?

Do what you want, it doesn't affect me.

 

[spotcatbug]

Part of the reason for the cost is the IPMI remote management that is built into the system board. It allows you to remotely manage the system over the network without ever needing to connect a monitor, keyboard, mouse or CD/DVD drive. It is a really nice thing to have.

I wish I had spent the extra $50 to get the one with IPMI. Live and learn. Cost-wise, you save yourself the cost of a screen and keyboard. That's gonna be more than $50, so at least a wash. Even if you already have those things (let's face it, on this forum, we all do), it's a pain when you need it, to hook them up. Where are you going to put your server? I stuffed mine in a closet. Sure, you set it up at your desk or someplace convenient, but then you remove the keyboard and screen and stuff it away. Well, believe me, you will have to put a screen and keyboard on the thing more than you think. Definitely worth $50.

 

[Sokonomi]

Do what you want, it doesn't affect me.

I know it doesn't, but it does affect me.. I'm just trying to make things work with what I have and can afford. Sometimes that's not ideal, but that's just how it is. You can afford a +20 disk NAS, good on you, but I cant. Try and see this from my perspective too.

@spotcatbug Why do you need it so much though? Normally you just manage the thing through the FreeNAS front end or SSH or something. So if you need to get down and dirty outside of that, there's probably some shit hitting the fan and that NAS box needs to come out anyway, right?

My box is going to sit on a shelf above my workstation, so if I need to, its a 2 minute cable swap.

Ive noticed you are sporting the setup I was currently looking at, hows it working out for you, aside from the apparent need to have IPMI?

 

[garm]

Ask yourself if you care about the data you and your family will store on this machine. Then ask yourself what the acceptable downtime is. When you know what you want you can start selecting hardware. If you don’t want to spend money on reliable hardware you should atleast be aware of the issues you will likely face. If you accept the server to be offline without being able to remotely manage it during a vacation then skip such features. The important part is to know the hardware you invest in. How it will perform with FreeBSD and FreeNAS. How reliable it will be for ZFS. The forum is full of people who lost their data due to ill informed or negligent hardware selection/configuration.

Sins you live in the Netherlands you have access to the entire EU market for no additional cost. You also have fairly cheap access to the US market. If you can’t understand why people here will bash your hardware selection you need to ask yourself if you are right or if they might be right. If you have the experience or if they do. Use the search function, find all the hardware related issues and get second hand experience or trust in others

 

[Sokonomi]

The NAS will not be the sole keeper of the data. It will simply mirror the data on all family devices into one central place so its backed up and easier to access from other devices. With the exception of some TV shows and movies which I can redownload in a weekend. So if the NAS goes down, all it means is data is trapped on individual devices for a while until it syncs up again. No loss, just inconvenience.

And if my crappy win10 i7 920 gaming box with a haphazardly slapped together RAID can manage an uptime of 873 days and counting, how bad can it really be if I ran a groomed box that has an OS specifically built for this job?

I do NOT have cheap access to the US market, are you joking? Have you seen the shippingcosts? 40 bucks to have a motherboard sent to me.. And last time I checked, I didn't speak chech or swedish or polish or russian, so good luck with those European markets I guess?

It is not a matter of distrust, its a matter of people throwing enterprise guff at me, even when I specifically mention my budget and what I have to work with. Imagine going to a car dealer and asking for a cheap car to get around in, and all the salesman says is "Ferrari". To every single question or suggestion you make, "Ferrari". It would drive you nuts, wouldn't it? Would it really hurt that much to pretend for a moment that I cant afford 'the good shit', and to just help decide what parts are most suitable for the money and parts at hand? My budget is 600 euros plus some spare parts. Its not 800, its not 700, so whats the point in suggesting parts that bring it up to that price. I'm asking what I can afford, not what the most ideal build would be, and that idea apparently rubs people the wrong way around here..

But it seems most of you are married to one brand and are not willing to stoop down to help a lowly peasant such as myself to even glance at more budget friendly albeit less ideal alternatives, so I guess ill just go off and try and do this thing to the best of my ability and some help from google. A few others, like that EcoNAS guy seem to manage alright, so ill try that I suppose. It wont be perfect and ironclad, but its probably good enough for an average joe like me.

 

[garm]

I’ve built a few servers sins I started using ZFS and my present one cost me about €400 ex drives. It’s not that I’m Scrooge or anything, my workstation and laptop both cost me more then €2000. I just like getting the most possible value for my kronor. It’s an obsession..

I have 6 hard drives in mirrors for about 6 TB of net storage, an E3-1225, 12 GB of RAM, LSI9211-8i and an additional Intel Network card for a total of 3 Ethernet ports. The server in a Dell of-the-shelf bought in Sweden, the network card is $10 from the US and the LSI is from China. I buy all my hard drives from Germany as the price and VAT is lower then Swedish.

The CPU is waay more then I need and I’m borderline on RAM. I have a few web applications and two databases running in jails so I occasionally see a few MB of page file.

I never put untested gear in production (trust it with my data). I extensively stress all components and I have never lost any data to ZFS. Granted I haven’t lost anything on my Mac either, but Windows and NTSF has let me down over and over again.

This machine has never rebooted on its own. When I first got it I moved over drives that where almost 30k hours old. Those have been replaced and that’s about the only time this machine has been offline.

The machine itself is about two years old and never misses a beat, the oldest data on it is almost 20 years old.

You don’t need a certain brand, you need gear built to different quality then what the gaming kids expect. You need server grade hardware to reliably process and store data over time.

 

[garm]

Here are some websites for you to look for parts or systems

https://pricespy.co.uk/
https://pricespy.ie/
https://www.amazon.de/?language=nl_NL
https://ebay.de/
https://www.mullet.se/
https://www.dell.com/nl-nl/work/shop/servers/sc/servers
https://www.hpe.com/nl/en/servers.html
https://www.alibaba.com/
https://m.nl.aliexpress.com/

 

[Sokonomi]

I’ve built a few servers sins I started using ZFS and my present one cost me about €400 ex drives. It’s not that I’m Scrooge or anything, my workstation and laptop both cost me more then €2000. I just like getting the most possible value for my kronor. It’s an obsession..

Yes! Bang-for-buck is my modus operandus. I enjoy the finer things in life, and to me its a sport to get them as cheap as possible. I'm not a fan of shelf bought boxes, Ive dealt with compaq, dell and medion, all of em disappointed me because they offer very little leeway. In those three boxes the disks were poorly ventilated and there was only room for the ones preinstalled so I had to get weird with some aluminum bars and grommets. Though it proves things can be done pretty ok with just 'desktop' grade stuff?

Speaking of bank-for-buck; this second hand 9 months used i3-6100 for 50 euros peeked my interest. Those are well over 100 euros normally, and seem to have plenty of grunt. A little more clockspeed than the G4400, double cache, and hyperthreading, plus it even has a lower TPD. Nice deal, yes?

I never put untested gear in production (trust it with my data). I extensively stress all components and I have never lost any data to ZFS. Granted I haven’t lost anything on my Mac either, but Windows and NTSF has let me down over and over again.

Stress testing is key, memtest, CPUburn and HDD sector testing. I do these standard with any build, to catch any crap in the early build stages. Wont be any different this time. I found a nice guide in the FreeNAS resource pages about disk burn-in. I always test disks but up until now I didn't know you could stresstest these, too. The more you learn.. :D

I wont lie and say windows softraid is perfect. ;) It can behave pretty ratty when setting it up, though (knock on wood) once I have it set up it hasn't flaked on me yet. My current media box has been on for +800 days in this config with 4 disks (no important data on that thing though!!). Still runs like a charm.

you need gear built to different quality then what the gaming kids expect. You need server grade hardware to reliably process and store data over time.

I understand server machines deal with more prolonged stress than gaming rigs, though there are plenty of extra bells and whistles I can do without. IPMI, SAS cards, ECC memory, while all very cool stuff, do I really need these features for some closet box?

Here are some websites for you to look for parts or systems

https://pricespy.co.uk/
https://pricespy.ie/
https://www.amazon.de/?language=nl_NL
https://ebay.de/
https://www.mullet.se/
https://www.dell.com/nl-nl/work/shop/servers/sc/servers
https://www.hpe.com/nl/en/servers.html
https://www.alibaba.com/
https://m.nl.aliexpress.com/

The site ive been using (tweakers.net) is a culmination of multiple IT shops, amazon.de included. Ebay is alright for cheaper stuff, but buying expensive boards feels sketch to me. Alibaba and Aliexpress just feel like ebays shady brothers to me; cheap stuff is ok, but read ALL the fineprint. Then theres mullet.se, orke bjorke moderkort ummm, is that Scandinavian? I dont know what most of that says. :p I heard some things about the popularity of a particular HP micro, but ive hunted those before, neigh impossible to find in my neck of the woods.

 

[gpsguy]

Yes, that's a nice CPU. I bought a budget server about 18 months ago, that included one. I'm not using it for FreeNAS, but could in the future.

this second hand 9 months used i3-6100 for 50 euros peeked my interest

 

[gpsguy]

Let me preface this reply to say that *I* am s member of the "use a system that supports ECC" camp.

But, having been a forum member for 6+ years, I also understand that getting ECC capable equipment at an affordable cost can be challenging for our users outside the USA.

Given that, you might me interested in an opinion by @Kris Moore - VP of Engineering at iXsystems.

I can't link to the video from my phone. Do a Google search for "Kris Moore video Talking FreeNAS" (July 3, 2018) and listen from about the 16:00 minute mark to 16:45.

Edit - here's the link to the video: https://www.youtube.com/watch?v=brfncQ5jPoA

 

[spotcatbug]

@spotcatbug Why do you need it so much though? Normally you just manage the thing through the FreeNAS front end or SSH or something. So if you need to get down and dirty outside of that, there's probably some crap hitting the fan and that NAS box needs to come out anyway, right?

My box is going to sit on a shelf above my workstation, so if I need to, its a 2 minute cable swap.

I'll just quote my own post: "believe me, you will have to put a screen and keyboard on the thing more than you think." I thought exactly the same as you: When I have a problem, I'll just SSH in and fix it. Or hopefully I can do it from the GUI. Now I think exactly as... me: I wish I had IPMI. What sorts of problems are you anticipating? Some subset of those problems will be network related. Can't SSH or use the GUI when the NAS has no network. Even if you're planning to keep the thing "keyboard and screen is easy" close to you, you may want to move it later. Servers can be kinda annoying sound-wise.

But, honestly, when you ask for experienced feedback, you shouldn't be thinking, "Well, that person needs to justify why they're telling me to do that!" You should be thinking, "I've never done this and they have, they probably know what they're talking about." I mean, that's why you're posting, right? I guess people are always surprised when what they think is right... isn't. I'm, of course, guilty of this as well. I would probably have IPMI right now if I had asked for feedback before I built.

Also, I forget where you landed on the topic of ECC RAM, but it was the first thing I thought about with your original post. See this thread.

I've noticed you are sporting the setup I was currently looking at, hows it working out for you, aside from the apparent need to have IPMI?

It's working out really well. I even have the Node 304 (great case). I went mini ITX specifically so I could get the Node 304. I wanted the Node 304 because I have 6 disks and the shelf in my closet won't fit a taller case. As I said, I just wish I had gotten the MB with IPMI.

 

[Sokonomi]

WD purple drives are a bit cheaper than the reds, while they seemingly do just about the same job.
Any thoughts on these? Could save me 40 bucks if its all the same.

Are there any other drives on this list that could do well for less money than the WD Reds?
Ive read the Baracuda from seagate is pretty good.

 

[Kevin Horton]

WD purple drives are a bit cheaper than the reds, while they seemingly do just about the same job.
Any thoughts on these? Could save me 40 bucks if its all the same.

See this forum post.

The firmware in WD Purple drives is apparently optimized to prioritize sending data on schedule (so the video doesn't freeze or stutter) at the expense of possibly sending bad data. The firmware in WD Red drives is optimzied to ensure the integrity of the data that is sent, even if the drive must pause to reread the data to ensure it is correct.

If the data is of no importance to you, go with the cheapest option. If corrupt data would be a problem, avoid the WD Purple. It is your data, and your money, so it is your decision to make.

 

[Sokonomi]

Ive looked into it, the purples are more geared towards constant writing whereas a red is better at bursts and prolonged periods of nothing. So purples arent really viable unless I intend to also do surveillance. But are reds the only viable NAS drives, or do other (cheaper?) brands have them too?

 

[IQless]

Both Seagate Ironwolf and HGST Deskstar NAS are geared towards NAS. I have two Ironwolf 4TB in my pool and they do run a little bit hotter than my other Red's but not by much. I have never tried the HGST drives tho.

The Seagate Ironwolf has been a bit cheaper to buy than the Red's, a least in Norway.

 

[Sokonomi]

ok, heres my hail mary on this whole NAS on a dime (more like on a dollar now) project. Please remember; im just looking for a passable grade, not a straight A+ cum laude. God knows that wont happen unless I put atleast a grand on the table ;)

Take 2 on the config;

CPU : Intel i3-6100 *
RAM : Corsair ValueSelect CMV16GX4M1A2133C15 1x16Gb *
MOBO : ASRock Rack C236 WSI
HDD :
Pool1 : 2 x WD Green 3Tb ** + 2x WD Red 3Tb (Raid1Z)
Pool2 : 2 x WD Green 1Tb ** (Mirrored)
SSD : Team Group L3 EVO 120GB (Boot pool)
PSU : Corsair CX450M
Case : Fractal Design Node 304
Cooler : Shuriken Rev. B
* Bought/buying secondhand.
** Already in stock.

Then, some deliberation on these parts;

CPU : This one is a good step up from the G4400 I originally slated, because it bumps up the clockspeed, adds more cache, and provides hyperthreading (though apparently freeBSD doesnt care?). Anyway, It was 45 bucks, 9 months used, still in store warranty, so I figured it was a good deal.

RAM : I'm gonna get an earfull, I already know it, but hear me out first. :p Yes, its basic birch DDR4 non ECC, but, BUT... its temporary. I deliberately chose a single stick of 16Gb for easier reselling, until I can either afford the ghastly expensive ECC stuff, or the prices of DDR4 come down a little again (They are UP there currently! ) Theres also some confusion going on with a friend of mine; do I, or do I NOT want registered ECC? Because if I do, there's this to consider; Kingston 16Gb Registered ECC. Otherwise there just aren't any affordable options right now and ill have to coast on regular peasant RAM for the time being.

MOBO : Its an asrock rack. Not the most ideal one, I know, but atleast its not a discoball all-flash-no-bang +/-100 euro one. It looks like its got the goods where it matters; Supports ECC (there it is), has a whopping 8 SATA ports allowing for full 6xHDD cage saturation (id have to sacrifice one for the boot drive otherwise), ample USB3.0 ports, and its got a pair of Intel NICs so freeBSD will play nice. I'll just have to pretend the audio bit isnt there.

HDD : Bit of a hodgepodge, for now. I already have two WD Green 3Tb and two WD green 1Tb loafing about. I intend to flash tweak them so they spinup quicker and can somewhat keep up with the pair of WD red 3Tb I intend to buy. I hope to ride out this config until the 6Tb drives have come down in price enough to swap out all HDDs for a fresh 6x6Tb Raidz2 down the road. Also, I doubt ill ever consider running a NAS with more drives than this.

SSD : I had to throw in a little SSD because Flash drives are rather unreliable. This one was cheap and the reviews I could find were alright enough for what it has to do, so there you have it.

PSU : Originally I had a cheap Intertec one planned, but I decided I should atleast go with a decent brand if I go cheap. Corsair has been a reliable horse in my stable for a while now, so im betting on it again. Is 450w enough?

Case : Its compact and well designed, as im used to from Fractal. Holds six normal 3.5" drives. The Node 804 has been mentioned, but its too bulky for the space I have reserved for it. It holds all the stuff I want, and has a good front to back airflow across the drives, which sounds like gravy to me.

Cooler : In all honesty I just grabbed a cheap one with good reviews. Its flat so it wont get in the way of the HDD cables, big fan so its quiet. Cant do worse than stock, and is probably way overkill for a stock speed 51W CPU. :D

So there you have it. I tried. If you have any suggestions or ideas, please share them. But if at all possible, please avoid more expensive suggestions, im already well beyond what I wanted to spend as it is now.

 

[KrisBee]

Check the memory qvl at

http://www.asrockrack.com/general/productdetail.asp?Model=C236 WSI#Memory

Board does not support registered ECC, its ECC UDIMM only. Don't make a costly error.

 

[Sokonomi]

Ive seen that remarkably short memory list; I doubt that is all the memory the board will accept.
Though you are correct in pointing out that the list only has unbuffered ECC on it.
Is unbuffered ECC no good in general, or is that just something to keep in mind when shopping?

I don't fully grasp the difference of registered vs. unbuffered, but isn't registered memory only useful in multi CPU builds?

 

[spotcatbug]

Board does not support registered ECC, its ECC UDIMM only. Don't make a costly error.

Er. Uh oh. Is there some reason you need registered ECC? I have that MB with unbuffered ECC.

 

[anmnz]

Er. Uh oh. Is there some reason you need registered ECC? I have that MB with unbuffered ECC.

No, you're fine. @KrisBee's point was that registered ECC memory does not work with that board, so it would be a "costly error" to buy the registered ECC memory that @Sokonomi mentioned.