First build with questions!

[Zefy]

So I am in the process of collecting parts for my FreeNAS build. I had a few questions after reading some other build threads along with the slideshow. Excuse me if these question have blatantly obvious answers. I am, after all, a noob...

So my goal for this is to have a small NAS box for a reasonable budget that is expandable down the road and is quick. All of my ultra critical data is backed up on other drives or in cloud storage so for that reason I've decided to use raidz1.

The build components so far is:

Fractal Design Node 304 mITX Compact Case
http://ncix.com/products/?sku=75056&vpn=FD-CA-NODE-304-BL&manufacture=Fractal Design

ASUS P8H77-I
http://ncix.com/products/?sku=71117&vpn=P8H77-I&manufacture=ASUS

Intel Pentium G2020
http://ncix.com/products/?sku=80243&vpn=BX80637G2020&manufacture=Intel

Intel PRO/1000 CT
http://ncix.com/products/?sku=37926&vpn=EXPI9301CTBLK&manufacture=Intel

Kingston HyperX Blu Black Series 16GB
http://ncix.com/products/?sku=78420&vpn=KHX16C10B1BK2/16X&manufacture=Kingston&promoid=1288

Silverstone SFX Series 450W
http://ncix.com/products/?sku=75786&vpn=ST45SF-G&manufacture=Silverstone Technology

Western Digital Red 3TB (x3)
http://ncix.com/products/?sku=74269&vpn=WD30EFRX&manufacture=Western Digital WD&promoid=1322

Kingston DT111/8GB
http://ncix.com/products/?sku=76090&vpn=DT111/8GB&manufacture=Kingston&promoid=1288

APC BACK-UPS RS 1500VA 865W
http://ncix.com/products/?sku=55354&vpn=BR1500G&manufacture=AMERICAN POWER CONVERSION&promoid=1245

So a few reasons I picked what I did.

The motherboard was chosen because it is one of the few mitx boards that has 6 sata ports on it along with 16gb of ram. I'm aware that it is not ECC along with the memory that I picked but the price difference to move up to ECC memory and mobo was quite a bit. I've read a few other builds and I know that the supermicro boards were recommended over the asus but the options are very limited for 16gb of ram and only 2-4 sata ports so a raid card would be required. Budget wise this isn't ideal. It also sounds like the build in ethernet on this board is crap so that's the reason behind the NIC.

The G2020 seems to be more than sufficient for a NAS box but I had a question about it. I was reading in the slideshow that if I want encryption i need an AES-NI processor. Now after looking around these are actually quite uncommon... Xeon and some i5/i7 processors have it at a cost of at least $200 for a processor. This is overextending my budget so I was wondering if I need/want encryption. I haven't seem much talk of it on the board here.

The power supply was chosen because it is rated for 24/7 use and is fairly cheap. I was also looking at the corsair cx430m.

My plan was to use 3 of the WD 3tb reds in raidz1. If I want to expand later I can make another vdev of 3 more HD's.

One other thing I was looking at is using an LGA1150 mobo. The ASUS H87I-PLUS has 6 sata 6gb/sec ports on it. I don't know if this is worthwhile or beneficial but the board is only 10 bucks more however the matching processor is substantially more but has AES-NI...

http://ncix.com/products/?sku=84516&vpn=H87I-PLUS&manufacture=ASUS&promoid=1342

Thanks for the help and I hope this is all reasonable!

 

[jgreco]

So instead of going for the $150 Supermicro board that has two decent Intel ethernets on it, you opt for a board with an extra Intel card that will end up costing you $130, simultaneously burning your only expansion on that limited mITX board. The real problem with all the mITX stuff is that you're cramming a lot of heat in a small space. You'd be better off with one of the micro-ATX Fractal cases and the Super board.

I wrote all of this for n00bs, I strongly suggest you read it and note that the words "irrational choices" apply to this little thread...

 

[gpsguy]

Another limitation with the mini-ITX boards is that most only have 2 memory slots with a system capacity of 16Gb max. Down the road, if you upgrade to 5Tb drives, you might need more RAM.

I know about this firsthand, since I have a beautiful Lian Li PC-Q25B system sitting unused in my closet. At the end of the day, the consumer mobo/RAM decided not to play nice with FreeNAS and I ended up buying another system and put ECC RAM in it.

 

[jgreco]

I know about this firsthand, since I have a beautiful Lian Li PC-Q25B system sitting unused in my closet. At the end of the day, the consumer mobo/RAM decided not to play nice with FreeNAS and I ended up buying another system and put ECC RAM in it.

Ouch. :( Listen to the voice of experience there.

 

[cyberjock]

When I saw this thread I knew jgreco would reply, and I knew what his response would be. And if you were wanting to spend your money smartly, you'd listen to his advice. Quite a few people have learned the hard way that jgreco has it right with how to plan a build without potentially screwing yourself later. I bookmarked that thread he made and its darn good advice. I refer to that thread for links when I build FreeNAS servers for friends, and I never go wrong.

The hardware you're looking to get I'd never ever recommend unless it was free...

Also, RAIDZ1 isn't a success path. A forum user that I'm working with to recover data from a failed array has lost 1 disk in 2 RAIDZ1s he has and guess what.. he's got corruption because he has no more redundancy from bad sectors. Google "RAID5 died in 2008" and you'll understand the math.

 

[Zefy]

The issue I have with moving up to a mATX is purely because of form... I don't have space for another desktop size computer nor do I have room for a server rack. The quest is for small footprint.

gpsguy, that's a good point about future proofing my system for larger HD's. One of the reasons I picked that board is because it is capable of 16gb of ram which a large portion of mITX boards are not capable of (intel and supermicro). But even if I was going to upgrade to 5tb drives later on (being that I'm only running 3 of them and for some reason would not opt for adding another vdev) I would still be fine with 16gb (6gb base + 10gb/per tb of HD).

That is very unfortunate that you were unable to get your original system to function properly. Did you try getting it to run on a different OS? NAS4free or (gasp) windows home server? I would personally rather switch OS than build an entirely different system.

As far as your advice jgreco, I've read your "irrational choices" thread but you're saying conflicting things. You recommend, in that same thread, the node 304 and with that, you must run a mITX board. One goes with the other, so I don't see how you can have that in your "read this NOOBS" thread without some conflict.

There is actually lots of conflicting info with this... cyberjock, you replied multiple times to this person's build thread with a VERY similar component list to my own but didn't mention anything about it all being poor hardware choices...

http://forums.freenas.org/threads/powerful-beautiful-mitx-build-looking-for-comments.11387/

Looking at other hardware builds it seems like half the people are building big servers with racks and full size cases while the other half is using mITX boxes. Can it really be that useless to use mITX?

I'm not looking for tons of storage and ultimate redundancy. I take lots of photos and videos. Over a weekend of I end up with 15gb of stuff. I end up using MAYBE 1gb of that. I usually end up deleting the rest. I would prefer not to, but if it gets lots, then fine. I could get by just fine with a single 3tb drive just plopped into my gaming computer but I like the other benefits of the NAS box and I don't like leaving that computer on. I've been getting by just fine for 7 years with a MAX of less than 400gb of SSD's between my 3 computers. :p

6tb should be grand I thinks...

Also, is there a reason to not use my one slot for the nic? What else would I put in there? I only have 6 HD bays so I won't need a raid card... What else might I need?

 

[cyberjock]

The reason why I didn't write out a long post is because I'm so tired of answering the same questions day in and day out when most of the mistakes people make are easily found on the first page of posts in the given section.

Then, to make it even more fun I respond to people(or other people respond) and tell them what pitfalls they may run into and they ignore the feedback anyway. So why do people bother posting if they're going to dismiss what some of the knowledgeable people have to say anyway?

 

[Zefy]

The reason why I didn't write out a long post is because I'm so tired of answering the same questions day in and day out when most of the mistakes people make are easily found on the first page of posts in the given section.

Then, to make it even more fun I respond to people(or other people respond) and tell them what pitfalls they may run into and they ignore the feedback anyway. So why do people bother posting if they're going to dismiss what some of the knowledgeable people have to say anyway?

If you're really that chafed about noobs posting in the noob forum and you feel obligated to reply then I would honestly recommend a copy and paste routine. It is EXTREMELY confusing for someone like myself to go read a number of build threads, sticky's, and discussions and wind up with a different opinion from you based on your mood.

I think I did a decent job coming up with my build plan and for you to come in here and say "The hardware you're looking to get I'd never ever recommend unless it was free..." when this is obviously a BS response. I picked ALL of my components based on other people's builds and sticky's.

My case, CPU, NIC, and HD's were all listed in the "So you want some hardware suggestions" thread. My PSU selection was from other build threads and it also meets the criteria listed in the same suggestions thread. Kingston flashdrives as far as I know are decent and at only 9 bucks a pop for 8gb versions I can get 3 very cheaply for the OS, updates, and backup... That covers all of my components except 3 (one of which is an APC which was listed in the slideshow so I put it on there) so that leaves only two components.

I'm assuming the biggest gripe is with my mobo (and thereby case, with the mITX design) and my memory choice. Once again this confuses me as you cannot recommend the node 304 case in the "chassis" section of the suggestions thread without implying the use of a mITX board. If that's really the case (pun teehee), then you should amend the suggestions thread to only include mATX cases as a minimum to avoid confusing us logical thinking noobs.

Enough of that now. Onto the learning bit...

I'm going to be honest now and say that I've actually owned the node 304 for a bit now... So I'm not going to stuff it in a closet and get a new case... live and learn I guess...

So being stuck with the mITX format I cannot get away from the heat issue involved with it like jgreco said. I can improve the cooling of the case but that is about it. I can monitor the CPU temps and add cooling as necessary.

The new mobo I'm thinking of using is the Intel DBS1200KPR mITX Server mobo. This little guy supports ECC deliciousness along with Intel 2 Gigabit LAN and 16gb of memory all for $165 ($50 more than original selection but I'm not having to buy a NIC so it's really only ~$15 more).

This mobo is untested with the g2020 according the ark website (http://ark.intel.com/products/67346/) so I changed my CPU to the G860 3GHz dual core processor (78 bucks, only $8 more than the original g2020 selection and it's slightly faster). Lastly is the memory selection. Going with Kingston KVR1333D3E9SK2/16gb (2x8gb kit). ECC ready and all that goodness. This is about $40 more than my original selection.

To add redundancy I will be moving up to 4 WD red 3tb drives and running in RAIDZ2. Although according to the article cyberjock told me to google (RAID5 died in 2008) also said that raid 6 (ala raidz2) should also be dead by now or very close to it... Curious about your take on it? RaidZ3 anyone? Raid Z4!?!(http://www.zdnet.com/blog/storage/why-raid-5-stops-working-in-2009/162)

So including my extra HD the price increase is about 240 bucks. Totally reasonable for an ECC beasting server grade machine that fits in my pretty NASbox. The downside to this board is it only has 4 sata ports on it. The implications of this is that I would have to either rebuild my zpool with 6 HD's to expand (and add a raid card, but no room for more memory) or more realistically build an entirely new system that can support 8-10+ HD's for growing room.

Hopefully this seems more reasonable to everyone and I hope I didn't piss off everyone to much. I truly am here to learn and I think my above configuration shows that.

 

[jgreco]

My case, CPU, NIC, and HD's were all listed in the "So you want some hardware suggestions" thread.

I'm assuming the biggest gripe is with my mobo (and thereby case, with the mITX design) and my memory choice. Once again this confuses me as you cannot recommend the node 304 case in the "chassis" section of the suggestions thread without implying the use of a mITX board. If that's really the case (pun teehee), then you should amend the suggestions thread to only include mATX cases as a minimum to avoid confusing us logical thinking noobs.

No. The point of mentioning the Node 304 was because it has been recognized by numerous users to have a small footprint, high drive density, and most importantly reasonably decent cooling for the drives, which is an incredibly important consideration. I do not mention the Node 304 as a way of implying "you can pick any mITX board you want and that's a good suggestion." I mention it because almost all of the other mITX cases are subpar for NAS use, and people have complained about poor build quality, excessive plastic, bad thermal design, tight cabling, etc., etc. I still expect you to put a server-grade board in there such as Intel's S1200KP, or the newer variant that you mention, but I really don't like recommending a board without some sort of experience with it, or at least a history of others with repeated successful builds.

 

[cyberjock]

It's not that I'm chaffed about noobs posting, its that so many don't even bother to read the manual or even look at the presentation I made.. for the noobs(I was one 16 months ago so my guide was written for noobs by a noob)... before asking a completely idiotic question or spending money on a server that has less than 4GB of RAM.

I will never do copy/paste as I prefer to give personal advice and not give you 10 bullets and let you figure out if it applies to you. If you're a noob what you don't need is me to give you a bunch of bullets and let you figure it out for yourself(you're probably plenty confused already).

I somewhat like your CPU upgrade(if you think you might want encryption someday you might want to look for a CPU with AES-NI support), your RAM choice, and your choice of RAIDZ2. I'm not sure if your CPU supports ECC. I know there's a handful of CPUs that unofficially support ECC(yours doesn't say that it does support ECC). I'd probably go for a CPU with hyperthreading as I've learned from my own mistake that hyperthreading can really increase your system's responsiveness and performance. I bought a Xeon that didn't have hyperthreading(I wrongly assumed it wouldn't really matter enough to constitute $100 difference in price) but I regret that decision now.

 

[Zefy]

Glad we got that all sorted out.

So I wasn't sure either about the ECC support on the g860 so I double checked. According to this website, it does have ECC.
http://www.acmemicro.com/Product/97...+0GHz+3MB+5+0GT/s+32nm+HD-Graphics+65W+Retail

As far as moving up to a xeon the price more than triples for one that supports hyper-threading. Like I mentioned in my original post, I wasn't sure (still not) if I want encryption. Does it only provide security for people physically stealing my HD's or does it also protect against intrusion of the virtual kind? I have to draw the line budget wise somewhere. I could pick between running raidz1 and a xeon (E3-1230 or similar) or get my second redundancy drive. If you had to pick between one or the other...?

As far as using the node 304 goes, I agree that construction and layout wise it stands above most of the other options for small cases. It just doesn't really seem like there is a good mobo choice to be made based on the info available on the forum here. Hopefully my build will help bridge the gap for small NAS awesomeness on a budget.

 

[cyberjock]

Encryption prevents anyone from accessing the data on the drives without the key. They can steal your drives, they can steal your key+passphrase (or recovery key), but unless they can put both together they'll never get your data. Every time the system boots up you must log into the GUI and put in the key+passphrase (or recovery key) to get access to the data. If you lose your keys you lose the data.. forever. I'm not sure what you mean by "the virtual kind" but I think my explanation will be sufficient for you to understand how it protects your data.

Yeah, I'm not to hip on the Xeon prices. Quite high just to add AES-NI support. Without AES-NI support you can expect extremely poor performance (possible less than 20MB/sec) with encrypted zpools. Just something to think about. You could always upgrade to a Xeon someday if you felt the need. I only mentioned AES-NI because I like to know what my potential limitations are before I click "buy".

 

[SkyMonkey]

I have the S1200KPR, with a G555 and 16GB of RAM. It supports ECC. It's mounted in a PCQ25B with 6 2TB WD Reds. The drives idle (spun down) at around 5C over ambient. I've seen them hit 40C-42C during a scrub/heavy use a few times. This in a very warm room (usually between 25-32C).

Full system build is described here. http://forums.freenas.org/threads/hardware-suggestions-nas-for-home-storage.11803/#post-54229

The entire system works quite well, with one complaint: I get IRQ storms when a device is plugged or unplugged while the system is powered. This has happened with wireless USB keyboards and a monitor (yes, via the DVI port). Once the storm starts, it eats a full core until a reboot. As long as I don't plug or unplug something from USB or the monitor while powered, I don't seem to have an issue. Note that I've only tried the keyboard via USB, and not anything else (other than the boot image USB stick).

 

[jgreco]

As far as using the node 304 goes, I agree that construction and layout wise it stands above most of the other options for small cases. It just doesn't really seem like there is a good mobo choice to be made based on the info available on the forum here.

You have grasped the fundamental problem. All options suck. Some just suck a lot less than others. Arguably the only people targeting this market are the guys at HP with the MicroServer, which suffers from only having 4 bays. The 304 won't fit a reasonable mainboard and doesn't have drive trays. etc., but those may not be issues for many hobbyist/home users.

 

[SkyMonkey]

Tried to edit this in above, but I'm getting a message that I cannot because my account doesn't meet some requirements for posting links(?) though the message seems to indicate that it does indeed meet the requirements...probably new forum gremlins.

I should also note that hot-plugging seems to not work on my system (Intel S1200KPR). No idea if it's a driver, backplane (would be odd, since it's passive) or Intel chipset issue, but plugging a disk in tends to make the array drop one to several other disks. Bad juju.

 

[cyberjock]

I've seen very erratic behavior with hot swap. I just assume hot-swap is broken since a server shutdown isn't a big deal for me. What would suck is an unstable system that forces me to reboot anyway. As some people have experienced in the forum, one unsafe shutdown can render a zpool unmountable.

Pro-tip: An unmountable zpool is a BAD thing.

 

[jgreco]

I've seen interrupt storms under ESXi when varying the number of cores (which is quite possibly only teasing a problem of some other sort into visibility). I wonder if it is somehow related.

Plugging a disk in to a passive backplane - one with each channel separate back to the mainboard - causes problems on other channels? Is that what you're saying? Yikes if so. Haven't seen that.

 

[SkyMonkey]

jgreco: That's indeed what I mean by passive backplane. Independent channels back to the mainboard (though shared power...I think it has 2 or 3 inputs for power.).

I documented my somewhat limited testing here: http://forums.freenas.org/threads/problem-due-to-hot-plug-recovery-etc-raidz2-test-config.12945/

 

[jgreco]

I actually saw that thread but didn't dig into it and didn't have any immediately obvious ideas. Doing so, I do notice one thing, and I think you should look at it. Your hardware lists a HighPoint SATA add-on. Now I'm pretty cynical and I'll take either end as being likely problematic for most problems, but I noticed on several reviews there that there was some nasty stuff in the one star review section for that card...

So could you please map out your drives and see if maybe you yanked a drive off the HP and the other drive was what dropped off. Also what driver does that attach with? It is possible that despite being labeled "hot-swap" in mfr specs that the FreeBSD driver is only partially baked.

 

[gpsguy]

I could have installed Server 2008R2 on a new computer to begin with (my 7+ year old "server" was running Server 2003). But, I decided to try FreeNAS, thinking it might be something (free) that I might recommend to other IT professionals. What better way to learn something, than to eat your own dogfood.

My needs were similar to your's and I started picking up parts, before deciding on a mobo and cpu. I too, wanted a mini-itx case (my old "server" was a Shuttle pc).

The system ran fine for ~4 months and then it would kernel panic while loading FreeNAS. I replaced the flash drive, ran exhaustive tests on the memory and hard disks, but I couldn't pinpoint the problem. I had been doing backups along the way, so I had access to most of the data, but didn't know if I had it all.

To make a long story short, life came crashing down on us (my wife and I), on Father's day. FreeNAS, my server/data, wasn't that important. After 5 months with no access to the server (just my backups), I bought a new "server" (time more precious than money), popped in my old array, restored my backup configuration and haven't had a problem with FreeNAS since then.

That is very unfortunate that you were unable to get your original system to function properly. Did you try getting it to run on a different OS? NAS4free or (gasp) windows home server? I would personally rather switch OS than build an entirely different system.