Please, save me from myself.

[Svengal]

I’ve been putting together a parts list for my FreeNAS build and a few components have had me chasing my tail. Hopefully this isn’t too long a post. I’ve summarized the inline questions at the end on the remote chance someone wants to cut to the chase.

Intended use
Home and small office use. File serving to 3-4 Linux and Windows clients via Samba share. Usually not all simultaneously, but sometimes 2 clients at the same time.

Data

• Critical work related files (CAD, 2D graphic files, spreadsheets...)
• Personal (aka irreplaceable) photos, videos
• Commercially available music and videos (not irreplaceable)

Jails and plugins (not sure about this as I’ve done very little research here)

• Kodi or Plex to serve the occasional movie or more frequently music
• Resilio to keep select files synchronized

Here is the setup I am thinking of
Headless
Motherboard: Supermicro X11SSM-F
Processor: (see thoughts/questions below)
Kaby Lake i3-7100,
or Skylake Xeon 1220-v5
or Skylake Pentium G4400
Memory: (see questions below) 16 GB in four x 4GB sticks from QVL
Storage: (see questions below) two x 8TB WD Red drives arranged as two way mirror
Boot media: two SanDisk Ultra Fit 32GB USB 3.0 arranged as two way mirror
PSU: SeaSonic - PRIME Titanium 650W
CPU Heatsink: Noctua - NH-C14S
Case: Fractal Design R5
UPS: Cyberpower 1000PFCLCD
Backup Process/Resources: (see questions below) TBD


Processor: Kaby Lake i3-7100, Skylake Xeon 1220-v5 or Skylake Pentium G4400
Below are a few of the processors I’ve considered and their pricing and feature sets.

Skylake Processor Candidates
All provide a minimum of 2 cores, ECC support, AES-NI, DDR4-2133, 3MB of smart cache
$54, Pentium G4400 @ 3.3GHz
$78, Pentium G4500 @ 3.5 GHz
$120 i3-6100 @ 3.7GHz adds Hyperthreading (2 cores / 4 threads)
$200 Xeon 1220-v5 @ 3.0 GHz, adds 4 cores with 3.5GHz turbo freq and 8MB cache

Kaby Lake Processor Candidates
All improve over Skylake processors with at least 2 cores, 4 threads and faster memory (DDR4-2400). Oddy Intel ARK does not list the G4600 and 1220-v6 L3 cache as being “Smart Cache”.
$87, Pentium G4600 @ 3.6 GHz
$102, Core i3-7100 @ 3.9 GHz
$215, Xeon E3-1220 V6 @3.0 GHz adds 4 cores with 3.5GHz turbo freq and 8MB cache

Processor Selection
I’m guessing an i3 would be more than powerful enough for my use case and the Kaby Lake i3-7100 seems like it is right in the sweet spot at $100.

BUT the X11SSM-F requires BIOS ver 2.x to support Kaby Lake processors. Can the BIOS be upgraded through the IPMI (or maybe using BIOS recovery via USB) without a processor? I see a lot of discussion on this in the forum, but I can’t find any definitive answer. If not I should just buy a Skylake processor and avoid the whole issue.

The price/feature trade off of the Skylake processors is basically the same, but it irks me that I’d pay $20 more for the Skylake i3-6100 (a slightly slower processor) than I would for the Kaby Lake i3-7100. For $80 more I could get a xeon 1220-v5 and my build would never break a sweat. Would a Xeon require require a separate video/graphics card to do the install, etc? On the other hand, maybe I’ve totally over spec’d this and a Skylake Pentium such as the G4440 would be fine?

After awhile of thinking about this my head has started to spin. A few words of advice would go a long way here. Please, save me from myself.

Memory
I’m planning to use 16 GB in four 4GB sticks to take advantage of DDR4’s quad channel capability. This precludes adding more memory without discarding the 4GB modules, but I think it is unlikely I will need to upgrade anytime soon. When I do upgrade it’ll probably be years from now and memory will be so inexpensive I can toss the old UDIMMs without it breaking my cheap-ass heart. Sanity check please? Do you think I’ll regret filling all four slots?

Whatever I choose it’ll be from from the Supermicro qualified vendor list or the Crucial equivalent of the Micron parts on the QVL.

Initially I was thinking I’d buy one stick of 16GB and discussed equivalent part numbers for the DDR4-2400 16GB Micron UDIMMs on the QVL with Crucial support. In case it is useful to anyone else here’s what I learned.

Micron MTA18ASF2G72AZ-2G3A1 = Crucial CT16G4WFD824A.18FA1
Micron MTA18ASF2G72AZ-2G3B1 = Crucial CT16G4WFD824A.18FB1
Micron MTA18ADF2G72AZ-2G3A1 = Crucial CT16G4XFD824A.18FA1
Micron MTA18ADF2G72AZ-2G3B1 = Crucial CT16G4XFD824A.18FB1

Perhaps this could be included in the “So, you’ve decided to buy a Supermicro X11 board…” resource.

Storage
I have two 8TB WD Red drives which I purchased on sale when I first started thinking about building a FreeNAS box. My plan was to arrange them in a 2 disk mirror for my vdev. This would give me 50% storage efficiency, an easy upgrade path and quick resilvering.

Then I read "ZFS Storage Design and Other Information", in which Cyberjock says RAIDZ1 does not provide sufficient redundancy and is therefore dangerous. That got me wondering if a 2 disk mirror vdev, (which like RAIDZ1 can survive just one disk failure) might also not provided enough redundancy to be “safe”.

But, I've also read a blog post by Constantin Gonzalez. Granted the post is old and even has a disclaimer stating so at the beginning, but the concepts still seem sound. It explains why a mirror vdev will resilver much faster than a RAIDZ vdev and thus have a smaller time period during which its vulnerability to additional disk failures is increased. To be honest I don't completely understand why. I suppose because rebuilding one harddrive requires reading from all the other drives in the vdev.

Is a 2 disk mirror somehow safer than RAIDZ1 or should I buy a 3rd drive for a 3 way mirror?

Components that are probably overkill (just in case you were wondering)
I could probably save some money by choosing different a PSU, case and CPU heatsink. All these components total $342, or 23% of my total $1462. I could choose stuff which is still good, but not quite as nice as the above three components and save about $130 or maybe more. But the savings isn’t worth it to me.

The UPS is probably bigger than I need but, I got it for a price equal that of more entry-level models. Otherwise I would be thinking of a Cyberpower 685AVR.

Backup Solution
Ideally for backups I would build a 2nd FreeNAS server, stash it someplace outside of the house, and replicate on an automated schedule. But building another server would completely blow my budget. I guess the I could shuck some external drives I have lying around (two 3TB WD My Book Lives and two 1TB WD Passports) and cold-swap them into and out of my main FreeNAS box. This idea doesn’t thrill me. It sounds like a hassle and like it would quickly wear the connectors and ancillary equipment in my box. I was thinking of using Amazon’s Glacier data storage service. Maybe it’s just the inveterate do-it-yourselfer in me, but I just don’t like the idea of storing data data in the cloud. Maybe rsyncing to a Raspberry Pi and some external drives. Obviously I need to look into backup solutions a bit. Any suggestions on particularly good threads or resources to read or what tools/technologies to look into would be appreciated.

Thanks
Thanks to all those who have posted information on the FreeNAS forums. It really is quite and amazing resource. Thanks especially to Jgreco, Ericloewe, Cyberjock and DrKK whose knowledgable posts and resources I have come across again and again. Thanks to UncleFester for his nearly-all-in-one, gentle-introduction-for-the-noob and his magnificent profile photo. Finally, of course, thanks to the FreeNAS developers and iX Systems. I couldn’t have done it without you. Queue tears… Oh wait, I haven’t done it yet. Never mind.

Recap of Questions:

• Can the X11SSM-F BIOS be upgraded through the IPMI (or maybe using BIOS recovery via USB) without a processor?
• Need advice on processor for my use case. I3-7100, Xeon 1220-v5 or Pentium G4400?
• Would a Xeon require require a separate video/graphics card when the box is not headless (to do the install, etc)?
• Please comment on the wisdom of filling all four memory slots with 4GB UDIMMs, making use of quad channel capability but limiting future memory upgrades?
• Is a 2 disk mirror somehow safer than RAIDZ1 or should I buy a 3rd drive for a 3 way mirror?
• Any suggestions on particularly good threads or resources to read or what tools/technologies to look into for putting togeather a backup solution would be appreciated.
  

 

[danb35]

Can the BIOS be upgraded through the IPMI (or maybe using BIOS recovery via USB) without a processor?

Yes, but it requires a key code that usually costs about $20 to do so. BS in my opinion, but Supermicro doesn't ask me.

I’m planning to use 16 GB in four 4GB sticks to take advantage of DDR4’s quad channel capability.

I'd go for a single 16 GB stick, or at least two 8 GB sticks. The performance penalty would be minimal (if even noticeable), but the room for expansion would be useful.

Is a 2 disk mirror somehow safer than RAIDZ1

Yes, but (IMO) not as safe as four-disk RAIDZ2. With RAIDZ1, when one disk fails, you have two others that could throw errors before the rebuild is finished (though with ZFS, the odds are very slim that such an error would result in loss of the whole pool). With a two-disk mirror, there's only one other disk to error. Mirrors are easier to expand and resilver faster than RAIDZ. RAIDZ2 or RAIDZ3 can offer both better storage efficiency and better redundancy than mirrors.

Ideally for backups I would build a 2nd FreeNAS server

The Proliant ML10 seems to be out of stock at TigerDirect--last I saw, they were selling it for $169. A stick of RAM and drives, and you'd have your backup solution. Other than that, FN11.1 looks like it's going to ship with cloud sync capabilities, so you could use your favorite cloud storage provider. Or there are threads here describing setups of swapping disks in and out and doing scripted backups on your box directly.

Would a Xeon require require a separate video/graphics card when the box is not headless (to do the install, etc)?

No, the IPMI controller includes a basic video adapter for such purposes.

 

[Benc]

Just a note. I recently bought X11SSM-F and it came with 2.0a bios so there was no need for upgrade to work with kaby lake cpu.

 

[Ericloewe]

I’m planning to use 16 GB in four 4GB sticks to take advantage of DDR4’s quad channel capability.

No.

For starters, the platform is not quad-channel, it's dual-channel. Second, memory bandwidth is not really a big concern for NAS. If you really, really want to operate in Dual-Channel mode, at the very least go with 2x 8GB. A single 16GB DIMM is still the better option.

Is a 2 disk mirror somehow safer than RAIDZ1 or should I buy a 3rd drive for a 3 way mirror?

It's safer because the rebuild is more straightforward and faster. It's not a huge difference. As for three-way mirrors, it's not a typical solution due the absurd "wasted" space - it's far more typical to go for RAIDZ2.

Components that are probably overkill (just in case you were wondering)
I could probably save some money by choosing different a PSU, case and CPU heatsink. All these components total $342, or 23% of my total $1462. I could choose stuff which is still good, but not quite as nice as the above three components and save about $130 or maybe more. But the savings isn’t worth it to me.

What you have now is outright wasteful. A 550W PSU would still be good for some 14 drives. But, more than that, the cooler is absurdly oversized and only serves to increase the risk of physical damage to the motherboard. A Noctua 90mm cooler is still much more than capable and quiet, but is cheaper and significantly reduces the risk of damaging your system.

$102, Core i3-7100 @ 3.9 GHz

This one is something of a risk. Besides needing the 2.0 BIOS, it was originally listed as not supporting ECC. That has since been changed, but the sour taste stayed behind.

On the other hand, maybe I’ve totally over spec’d this and a Skylake Pentium such as the G4440 would be fine?

If you're not going to transcode, a Pentium will do fine.

 

[Evertb1]

Backup Solution
Ideally for backups I would build a 2nd FreeNAS server, stash it someplace outside of the house, and replicate on an automated schedule. But building another server would completely blow my budget. I guess the I could shuck some external drives I have lying around (two 3TB WD My Book Lives and two 1TB WD Passports) and cold-swap them into and out of my main FreeNAS box. This idea doesn’t thrill me. It sounds like a hassle and like it would quickly wear the connectors and ancillary equipment in my box. I was thinking of using Amazon’s Glacier data storage service. Maybe it’s just the inveterate do-it-yourselfer in me, but I just don’t like the idea of storing data data in the cloud. Maybe rsyncing to a Raspberry Pi and some external drives. Obviously I need to look into backup solutions a bit. Any suggestions on particularly good threads or resources to read or what tools/technologies to look into would be appreciated.

There are several strategies/methods/tools to choose from for your backups. My experience is that a collection of external drives is indeed a hassle. And any backup strategy with a lot of hassle will fail at some point. You start with postponing needed backup actions and at a certain point you are not even sure anymore when you did it the last time.

I use 2 methods of backup. I have an OMV based lightweight server. Its only purpose is acting as backup medium. It's not really off-site but it has a safe place in my garage, that is separated from the house. I use rsync with scheduled push jobs from the FreeNAS server to backup up all my datasets. I to would prefer a second FreeNAS server for replication but that is not in the budget right now.

I also run backups to the cloud. I have a total of 5 TB cloud storage and run backup jobs on the FreeNAS SMB shares from a Windows client. I use a tool from StableBit called CloudDrive so the system can access my cloud storage as if it was a local drive. I have a small SSD as cache for this backups (CloudDrive support this) but that is hardly needed. It runs smoothly as it is. The initial backup was a lengthy process of course, but maintaining the backups has been no trouble at all. Everything that is written to the cloud is encrypted. And yes, If needed I can access the data from another computer and decrypt it again.

 

[Svengal]

Yes, but it requires a key code that usually costs about $20 to do so. BS in my opinion, but Supermicro doesn't ask me.

Just a note. I recently bought X11SSM-F and it came with 2.0a bios so there was no need for upgrade to work with kaby lake cpu.

Good to know vendors might be shipping the X11SSM-F with BIOS 2.x and that there's a cheap (if irksome) method to upgrade. Thanks.

I'd go for a single 16 GB stick, or at least two 8 GB sticks. The performance penalty would be minimal (if even noticeable), but the room for expansion would be useful.

A single 16GB DIMM is still the better option.

OK. A single stick of 16GB it'll be.

Yes, but (IMO) not as safe as four-disk RAIDZ2. With RAIDZ1, when one disk fails, you have two others that could throw errors before the rebuild is finished (though with ZFS, the odds are very slim that such an error would result in loss of the whole pool). With a two-disk mirror, there's only one other disk to error. Mirrors are easier to expand and resilver faster than RAIDZ. RAIDZ2 or RAIDZ3 can offer both better storage efficiency and better redundancy than mirrors.

It's safer because the rebuild is more straightforward and faster. It's not a huge difference. As for three-way mirrors, it's not a typical solution due the absurd "wasted" space - it's far more typical to go for RAIDZ2.

Hmm. OK. I'll need to think about this a bit. I can stretch to buy a 3rd disk and do a three-way mirror, but buying a fourth and using RAIDZ2 would be too much at the moment. 8TB is more storage space than I need. I purchased the 8TB drives because they were on sale for about the price of a smaller (4TB, 6TB? I can't recall) drive. My point is given the hardware I have a three way mirror might be my best path forward.

What you have now is outright wasteful. A 550W PSU would still be good for some 14 drives.

I know and that's what pushed me to the Ti certified (the only 80+ certification that specifies efficiency at 10% load) . I estimated idle power consumption for my configuration at about 60w and max power at 192w to 221w depending on the processor. Applying a safety factor of 125% gives a target supply size of about 240-275 watts. For the very first PC I built I chose just about the cheapest PSU and case I could find. The thing sounded ... well, loud. In the latest PC (workstation) I built I used a a Seasonic SS-660XP2 (Platinum Series) which was a joy and I swore I'd use nothing but Seasonic from then on.

So having estimated my power requirements I looked at all the Seasonic Gold, Platinum and Ti rated supplies. I discarded all the fanless options and the smallest supply was 550w. 110w (20%) is far larger than my idle. Even if I considered all brands and fanless supplies (which I am dubious of), the smallest supply that is more than a few dollars cheaper than the SeaSonic - Prime 650W Ti is a 450w gold unit. 90w is still a long way from 60w. So I decided to throw money at the problem. Even 10% of 660w is above my estimated idle, but at least its close. Actually its the closest I can get. I know this isn't an elegant solution, but given supplies available??? Is my assumption that idle should land within a range 20%-100% of the supplies rating bad?

But, more than that, the cooler is absurdly oversized and only serves to increase the risk of physical damage to the motherboard. A Noctua 90mm cooler is still much more than capable and quiet, but is cheaper and significantly reduces the risk of damaging your system.

Good point. Maybe something like the Noctua NH-U9S or NH-L9x65.

Thanks for coming to my aid. All the feedback has untangled me a bit and made it easier to see how to move forward.

 

[Svengal]

You start with postponing needed backup actions and at a certain point you are not even sure anymore when you did it the last time.

I can totally see this happening.

I use 2 methods of backup. I have an OMV based lightweight server.

OMV = OpenMediaVault? I'll have to look at this more. The lower hardware requirements might make it something I can spend for. Thanks.

I...run backup jobs on the FreeNAS SMB shares from a Windows client

This might bridge the gap, until I can spend for backup hardware and/or untill FreeNAS 11.1 comes out with the cloud sync feature danb35 mentioned. Nice.

I'm off for a few hours to rip soggy drywall off my ceiling. What fun.

 

[Evertb1]

OMV = OpenMediaVault?

Yes it is. Using abbreviations is a bad habit, I know.

OpenMediaVault is a good RSync target. The heart of my server is an Asrock E350M1 Series motherboard that I had lying around from a (failed) htpc project. It contains an AMD APU and together with 4 GB DDR3 memory it is more then sufficient for running OMV for my use case. It also is very low in the power consumption.

Succes with the drywall.

 

[Stux]

If you have a good backup then mirrors should be fine.
Seems like you should go for the kaby lake cpu.

If you need to spend $20 to update the bios, then you still break even and have a better cpu.