BUILD Build Check On Two Servers For a FreeNAS First-Timer

[Shrike Alvaron]

First of all, as I can see now while proofreading this post I can get a bit wordy as I like to explain my hardware choices. Sorry about the length!

Anyway, hello! I've been aware of the advantages of using a NAS for a while but sort of ignored them since hey, storage in my primary gaming PC was serving me just fine. However, while I have yet to have any sort of catastrophic failure, ever since Cryptolocker came to prominence I've gotten more and more paranoid. Currently my storage consists of a 2TB drive inside my PC that's backed up to an external hard drive that is rather unwisely always connected to my PC as if I had to unplug it and plug it in every week to backup I wouldn't do it. I also have cloud backup with Backblaze, but I've never been super thrilled with cloud backup. It works, but if I could get remote backups going on my own terms I'd prefer that.

So I started looking into FreeNAS earlier this year and have been impressed by what I've seen. Along with just file serving (of which I really like the sound of ZFS snapshots!), other things I think I'd like to do with it would be to serve as a PLEX media server, OpenVPN server for secure connections with my phone anywhere I go, an FTP server to share some of my files with friends, a Minecraft server for those same friends, and even perhaps a small PhpBB forum for those same friends again (though yes, I am aware that internet-facing jails can be a vulnerability hence why I'm still unsure on that). Probably will come up with more uses for it later too.

But going back to what I said, backup. I am well aware of the idiom that RAID is not backup, and while Crashplan can work on FreeNAS, with the recent discontinuation of their restore to door service I don't see it as being feasible for me to depend on as my backup as it would take months to download everything from scratch. As such... Well, I'm all about overkill. So how about I build TWO servers instead of one? First of all is Nexus, which will be my main server at home and use for those above purposes.

Primary Home Server "Nexus"

Motherboard/CPU - ASrock C236 WSI Mini ITX w/ Intel Core i3-6100: ITX is basically a requirement for me these days as I just love small things. Originally I was looking at the Supermicro and ASrock Avoton motherboards, but when I looked into just using a socketed board and an i3 CPU I found that the cost was basically the same. So, why not go for more performance? The higher clock speeds on an i3 will also be better for a Minecraft server and Samba performance (not as if I really need the best performance out of it for a single user). I went with Skylake over Haswell now that FreeNAS 9.10 has updated compatibility, given the superior performance and power usage of Skylake along with 16GB DIMMs.

RAM - Crucial 16GB DDR4-2133 ECC UDIMM CT16G4WFD8213: This was tough to find, and at first I didn't think DDR4 16GB ECC UDIMMs existed. As far as I can see this is the ONLY such option available for now, but at around $100 it's reasonably priced if not hard to find. I'm only getting one stick for now, so that later I have the ability to upgrade to 32GB.

Case - Fractal Design Node 304: I had been looking at the Lian-Li Q25 at first but the design of the Node 304 really grew on me as I researched. I like that it's cheaper, for one. Shame I can't find it in white, I wonder if they discontinued that one? The Q25 is still open for debate as it has the same 6 HDD + 2 SSD capacity the Node has, but for now I'm settling on the Node.

Cooling - Noctua NH-L9x65 CPU Cooler, 1 NF-A14 FLX 140mm fan, 2 NF-A9 FLX 92mm fans: ...I really like Noctua. I'm one of those who actually like the beige color. Noise is a big concern for me and I appreciate how quiet Noctuas are, hence why I want to replace all the Node 304's stock fans. As for going with the x65 version of the cooler, well... Why not? The 304 isn't lacking for space, though with the mess SATA cables will cause I want to avoid any large towers.

Storage - 6x Western Digital Red 2TB in RaidZ2 for 8GB Usable: Not much to say, Reds are pretty much standard. After learning of how you can't really expand ZFS pools in the traditional sense, I wanted to fill out the case's storage. 8TB (or whatever that ends up being in actual usable non-manufacturer-speak space) should be good for me for hopefully years seeing as I'm not even using 2TB right now. I do want to get into ripping my Blurays though, so that'll take up space...

Power Supply - SeaSonic SSR-360GP ATX PSU: This is something I reconsidered recently after seeing the power supply guide pinned in this forum, but after running the numbers I can't see a need for anything larger. The Skylake i3 uses a fair bit less power than the Xeon E3 from the guide, plus it seems WD has improved the power usage of the Reds when looking at their spec sheet. Conservatively I estimate the peak usage of the system will be around 232 watts, maybe 250 at the most. That is still below 80% of the PSU's capacity. Of course it does only have 4 SATA connectors, so I'll be grabbing two of the 3 way splitters from Monoprice to provide 8 total SATA connections, allowing for the addition of mirrored SSDs as a future upgrade path.

UPS - CyberPower CP850PFCLCD: At first I was unsure about needing a UPS, then I decided to go with a cheaper ones, then I learned that modern power supplies either need or at least really should use a UPS with pure sign wave output... This seems to be the cheapest model you can get with a pure sine wave, and provides plenty of capacity for me

Boot Drives - Transcend 16GB JetFlash 710 USB 3.0 Flash Drive x2: Well, FreeNAS 9.10 now supports USB 3 so might as well get USB 3 boot drives. In a mirror, of course. Since the Sandisk Ultra Fit appears to have heat issues above what is normal for USB 3 sticks, I went with a different brand that is still rather low profile. Briefly I had considered using mirrored 120GB SSDs for boot along with the system dataset and jails, but even if it's possible now it doesn't seem to be recommended plus I found at least one person who had issues with it. I still like the idea of mirrored SSDs for system and jails, but... Future upgrade.

So that's the primary server. For backup, I'll build a far cheaper system I can keep at my parents' house. It will be connected to Nexus over a VPN tunnel and I'll use ZFS replication to backup to it. I'm calling this remote backup server "Valhalla".

Remote Backup Server "Valhalla"

Motherboard/CPU - ASRock Q1900 Mini ITX w/ Integrated Intel Celeron J1900: First of all, no ECC support! I decided that resiliency to failures really isn't important to this server. If something bad happens, I can replace the fault, haul it back home, and rebuild the backup from scratch. Price was all important here, and the Q1900 is only $75 for a motherboard and CPU. There's even cheaper, but the price difference is small.

Memory - Mushkin Essentials 8GB DDR3-1333: Nothing special, just the cheapest decent brand 8GB SODIMM I saw. Again I'm only using one stick though so that if needed I can upgrade it in the future. I doubt I would need to in this one, but you never know.

Case - In Win BP671 Mini ITX Case w/A PSU I won't use: Since this is going to my parents who have their U-Verse gateway sitting in their entertainment center... I really need a case as small as possible. This case is SO tiny that I should be able to wedge it behind there to keep it out of their sight. Of note the case only has one proper 3.5" bay, but I'll use an adapter to put the second drive in the 5.25" bay. Also, it does come with a PSU but... Never use the PSU cheap cases come with.
Cooling - Noctua NF-R8 redux-1200 80mm fan: According to reports the fan that case comes with is horrible, so... Another Noctua.

Storage - 2x Western Digital Red 4TB Striped for 8GB Usable: Again, no redundancy! I don't exactly care if my backup fails (unless I also lose three drives in the main server at the same time, though you have to draw a line somewhere). I do want to make sure it has the same usable space, though.
Power Supply - Seasonic SS-300TFX Bronze: I didn't realize there were actually any good PSUs in TFX form factor, but apparently Seasonic makes them. It was with discovery of this PSU that I chose to go with the ultra tiny In Win case, and while it's not Gold rated this box should be using little enough power it would hardly make a difference. Plus, it's my parent's power and not mine anyway!

UPS - ...None. Again, I don't really care if I lose the pool on this backup. I can rebuild it, I have the technology.

Boot Drives - SanDisk Cruzer Fit 8GB USB 2.0 Flash Drive x2: Okay fine, a little redundancy. Because flash drives are cheap enough, why not. Also I just realized that this mobo DOES have two USB 3 ports on it, so I might swap these for the same Transcends as from the Nexus build just... Because. They're the same price.

And that's that. I'd also need to get a new router as my current 2Wire Gateway for AT&T U-Verse only has 100Mbps ports and 802.11g which really bottlenecks my PLEX streaming to my Chromecast. While PFsense is tempting... This is already quite a lot. I've tentatively chosen the Asus RT-AC66U for a new router as I don't need anything super fancy considering me and my devices would be its own clients. I do want AC for future proofness, though. In all, I'm looking at around $2000 for all this. Not a cheap undertaking that's for sure. I can offset it some with stuff I need to sell, but still. Because of the cost I really want to make sure I've thought things through which is why I'm posting this here. I feel like there's nothing I could have missed at this point given how much research I've done, but if there's something about these builds that's off (other than the lack of resiliency to failure Valhalla has) do tell me!

 

[Kevin Horton]

Remote Backup Server "Valhalla"

Motherboard/CPU - ASRock Q1900 Mini ITX w/ Integrated Intel Celeron J1900: First of all, no ECC support! I decided that resiliency to failures really isn't important to this server. If something bad happens, I can replace the fault, haul it back home, and rebuild the backup from scratch. Price was all important here, and the Q1900 is only $75 for a motherboard and CPU. There's even cheaper, but the price difference is small.

Understand that if your non-EEC RAM in the backup server has an issue, it could lead to silently corrupted files - i.e. it won't necessarily cause an obvious error until you try to use the bad files. This is OK, as long as you understand that your backups may be bad. More info here.

What is the purpose of having a backup, if you can't be sure that it will be in good shape when you need it? If you care about your data, then you should build a system with EEC RAM. If you don't care about your data, why have a backup at all?

 

[Ericloewe]

ASrock C236 WSI Mini ITX w/ Intel Core i3-6100

Sure, but consider an i3-6300, as the extra MB of L3 cache is always nice to have.

As far as I can see this is the ONLY such option available for now,

Not at all. Both Samsung and Hynix have 16GB DDR4 UDIMMs.

Power Supply - SeaSonic SSR-360GP ATX PSU: This is something I reconsidered recently after seeing the power supply guide pinned in this forum, but after running the numbers I can't see a need for anything larger. The Skylake i3 uses a fair bit less power than the Xeon E3 from the guide, plus it seems WD has improved the power usage of the Reds when looking at their spec sheet. Conservatively I estimate the peak usage of the system will be around 232 watts, maybe 250 at the most. That is still below 80% of the PSU's capacity. Of course it does only have 4 SATA connectors, so I'll be grabbing two of the 3 way splitters from Monoprice to provide 8 total SATA connections, allowing for the addition of mirrored SSDs as a future upgrade path.

A few thing to take into account:

• Modular cabling, like in the G-450, is very helpful, especially in small chassis.
• Many SATA power splitters seem to be dodgy. Careful with what you acquire.

 

[Jailer]

Motherboard/CPU - ASrock C236 WSI Mini ITX w/ Intel Core i3-6100

If you have a little more room in your budget I would consider the ASRock E3C236D2I. It's $60 more but it doesn't have the things you don't need (on board audio/video) but it does have IPMI. While IPMI is not a necessity, it's a wonderful convenience that you'll be thankful you have once you use it.

 

[Shrike Alvaron]

Understand that if your non-EEC RAM in the backup server has an issue, it could lead to silently corrupted files - i.e. it won't necessarily cause an obvious error until you try to use the bad files. This is OK, as long as you understand that your backups may be bad. More info here.

What is the purpose of having a backup, if you can't be sure that it will be in good shape when you need it? If you care about your data, then you should build a system with EEC RAM. If you don't care about your data, why have a backup at all?

I need to research replication more but from my understanding the replication will fail if there's a checksum mismatch. Unfortunately, Cyberjock's thread only mentions replication in a scenario where the push machine lacks ECC, and finding examples of cases where pull lacks it is harder. Upgrading the backup to use ECC would cost around $150 though since the entire motherboard and CPU would need replaced, so it's something I'll have to think about. If replication would fail if there's a bit flip, then I could easily pick up the server (I love close enough to my parents so getting the server to bring home as needed isn't an issue) and rebuild the pool at home overnight so not sure if the extra expense would be worth it.

Sure, but consider an i3-6300, as the extra MB of L3 cache is always nice to have.


Not at all. Both Samsung and Hynix have 16GB DDR4 UDIMMs.


A few thing to take into account:

• Modular cabling, like in the G-450, is very helpful, especially in small chassis.
• Many SATA power splitters seem to be dodgy. Careful with what you acquire.

I'll consider the 6300! It's not too much more, it may depend on if I upgrade the backup to ECC or not. As for the DDR4... Huh, I actually checked Samsung's site after hearing mention of them before and didn't see anything. Must not have looked hard enough. And for modular cables, I think I'd be using all but the PCIe cables from the 360w (need a molex to power the fans) so not sure if modular would be that big for me. The splitter is a SATA to 3xSATA and looks to be good quality, I'd never use a molex to SATA since it's those I always hear horror stories about yeah.

If you have a little more room in your budget I would consider the ASRock E3C236D2I. It's $60 more but it doesn't have the things you don't need (on board audio/video) but it does have IPMI. While IPMI is not a necessity, it's a wonderful convenience that you'll be thankful you have once you use it.

I had looked at this one! Ultimately I decided for the cheaper model due to the extra 2 SATA ports, allowing for upgrade in the future without needing a new card. Since the server will be sitting in the corner of my living room, it'll be easy to get to it physically if need be. I even have an old monitor I can use with it as required.

 

[Kevin Horton]

I need to research replication more but from my understanding the replication will fail if there's a checksum mismatch. Unfortunately, Cyberjock's thread only mentions replication in a scenario where the push machine lacks ECC, and finding examples of cases where pull lacks it is harder. Upgrading the backup to use ECC would cost around $150 though since the entire motherboard and CPU would need replaced, so it's something I'll have to think about. If replication would fail if there's a bit flip, then I could easily pick up the server (I love close enough to my parents so getting the server to bring home as needed isn't an issue) and rebuild the pool at home overnight so not sure if the extra expense would be worth it.

What happens if the data is successfully replicated, but then there is a RAM error during the periodic scrub on the backup server? Answer - One or more files on the backup are now silently corrupted. Bye bye backup.

 

[Ericloewe]

Huh, I actually checked Samsung's site after hearing mention of them before and didn't see anything.

Their site is close to useless, unless you're $_big_company looking to acquire DIMMs by the thousands. But trust me, they do:

 

[Shrike Alvaron]

What happens if the data is successfully replicated, but then there is a RAM error during the periodic scrub on the backup server? Answer - One or more files on the backup are now silently corrupted. Bye bye backup.

If I go this route, I suppose I could drop one of the drives from the backup to help fund the ECC upgrade and then add the second drive in later once I need it. I suppose I don't need 8GB of backup space quite yet anyway, and the single 4GB might do fine for the next year or so.

Their site is close to useless, unless you're $_big_company looking to acquire DIMMs by the thousands. But trust me, they do:

Allright, I can look up that model number. Still, I do like Crucial and have seen others who've said they used that stick in the ASrock C236 ITX boards without issues.

 

[Ericloewe]

Allright, I can look up that model number. Still, I do like Crucial and have seen others who've said they used that stick in the ASrock C236 ITX boards without issues.

Any of the big three (Micron/Crucial, Samsung and Hynix) is generally fine, though it's best to get something from the QVL.

 

[Shrike Alvaron]

Any of the big three (Micron/Crucial, Samsung and Hynix) is generally fine, though it's best to get something from the QVL.

Strangely enough ASrock doesn't have a list of compatible memory for the C236 WSI, unless it's hidden away somewhere. There's only a CPU and hard drive list on their site.

But okay, I've looked around and come up with a possible new build for remote backup with ECC RAM. It sacrifices the second 4TB drive leaving it at just 4TB capacity starting out, but I could add the second drive later on.

Memory: Kingston 4GB (1 x 4GB) DDR3-1333 Memory ($22.99 @ SuperBiiz)
Memory: Kingston 4GB (1 x 4GB) DDR3-1333 Memory ($22.99 @ SuperBiiz)
Storage: Western Digital Red 4TB 3.5" 5900RPM Internal Hard Drive ($149.99 @ B&H)
Case: Inwin BP671.FH200B Mini ITX Tower Case w/200W Power Supply ($49.99 @ Directron)
Power Supply: SeaSonic 300W 80+ Bronze Certified TFX Power Supply ($30.87 @ Amazon)
Case Fan: Noctua NF-R8 redux-1200 21.1 CFM 80mm Fan ($10.57 @ Amazon)
Other: ASRock C2550D4I Mini ITX Server Motherboard FCBGA1283 DDR3 1600 / 1333 ($280.00)
Other: Intel Avoton C2550 Quad-Core 2.4Ghz (Integrated) ($0.00)
Other: SanDisk Cruzer Fit 8GB USB 2.0 Low-Profile Flash Drive ($12.00)
Total: $579.40

An alternative would be an ASrock server board and a Haswell or Skylake Celeron with the stock heatsink, but I'd be concerned about my parents getting annoyed by the noise. Adding a Noctua heatsink to such a setup would bring it to the same cost as this passively-cooled Avoton. This is about $65 more than what I had previously been going with along with the additional cost for the second drive later on.

I do like this Avoton board, it has support for all of ECC, IPMI, and AES-NI. I forgot to mention it but I plan for both servers to be encrypted, not an issue for the main server with the Skylake i3 which all support AES-NI, but the J1900 doesn't. I figured that it wouldn't be an issue though since all the server would be doing would be receiving replication backups (though it would be cool to be able to use it for other tasks if I think of any I suppose). Neither do the Haswell Celeron's support AES-NI it seems, though Skylake Celerons do. But then, the only real ECC ITX Skylake board is the same C236 WSI which doesn't support IPMI without spending $50 or so extra. The Avoton seems to be a nice middle ground, and the quad-core version will be plenty for how little stressed the remote server will be.