BUILD check my SuperMicro ZFS build?

Status
Not open for further replies.

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
Ready for another obligatory "look over my hardware" post?

EDIT: per Cyberjock's pointers, RAM and PSU replaced.

I've done a bit of research lurking here, but there's nothing like an individual double-check to be sure I haven't made a dumb mistake. Feedback appreciated!

Components breakdown:
Motherboard/CPU: SuperMicro MBD-A1SRi-2758F-O
RAM: Crucial 16 GB (2x8) 1600 ECC unbuffered (wrong form factor)
Kingston 16 GB (2x8) ECC unbuffered SO-DIMM
PSU: Rosewill CAPSTONE-450-M Gold
better brand: Corsair CSM450M Gold

Case (already owned): Bitfenix Prodigy
HDDs (temporary, already owned): 2x3TB WD Green
HDDs (eventual plan): 6x4TB Seagate STBV4000100 (cracked open for ST4000DM000)

Components explanation:
Motherboard/CPU: SuperMicro gets recommended non-stop here, and I'm already tied to mini-ITX because of my case. This model was on Newegg and Amazon with 64GB max RAM (1600 ECC), 6 SATA ports, an IPMI ethernet port (not to mention 4 others), and an integrated Atom octa-core 20W TDP CPU.

RAM: ECC, unregistered, enough(?) capacity for my needs (I'll explain below), max speed my motherboard supports, reputable brand.

PSU: wattage I've heard is sufficient to run efficiently with a server board and 6 HDDs, 80+ gold rating, reputable brand.

HDDs: I know the 3TB drives are less than ideal, but it was a good sale. Those will be a non-redundant pool temporarily just so I have *a* backup solution for my most critical data until I can afford my full loadout of 6 drives, which will be RAIDZ2 - 16TB usable space.

Usage plan:
This is meant to be a backup NAS/media hub, a daily rsync'ed clone of all my drives. As my PC's internal drives die (currently 17TB unique space, but I can downsize), I will leave them unreplaced and let the NAS copy of everything be the master version. The most strenuous thing I am currently planning on doing with it is using ZFS compression and RAIDZ2, but I want to be able to expand its duties if I feel like it.

Any thoughts? I've decided against some of the fancier things like a ZIL, but I'm open to the idea of a 3rd party NIC if people think it would help. The way I store data, deduplication isn't terribly attractive, but scrubbing would be a possibility, especially with the kind of drives I'm buying. I'm also open to the idea of NAS drives - my current plan is to let the redundancy of RAIDZ2 paired with my low-intensity use patterns be safe enough with consumer drives. Oh, and I'll be adding a UPS to this configuration, probably when I get those 6 drives.
 
Last edited:

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I'm sorry, but I don't consider Rosewill to be a "reputable PSU manufacturer". I just don't.

That RAM isn't compatible with the motherboard. That motherboard requires SO-DIMMs. What you have are DIMMs. ;)

I have no idea what you mean when you say "scrubbing would be a possibility". Scrubbing isn't optional. It's necessary!
 

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
See? All that research and still I pick the wrong form factor of RAM. Thanks for the quick reply, Cyberjock! Replaced with 2x8GB 1600 Kingston ECC 204-pin SO-DIMM. I would rather not get Kingston because of your concerns about their musical part numbers with DIMMs (though I guess I don't know that that extends to SO-DIMMs specifically), but I have not found a single ECC SO-DIMM that wasn't Kingston. If anyone knows better, feel free to embarrass me with it.

I hear you about Rosewill - "reputable" may have been an oversell. Replaced with Corsair CSM450M Gold modular.

I suppose it would be more accurate to say my system should be able to handle regular scrubs well. It's optional the way putting data on the NAS is optional.
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Morning Choco,

You've got the right choice in that you don't need a ZIL or L2ARC for your usage patterns as it won't benefit you at all for general media storage.

If you intended on taking multiple full system backups (ie: imaging your drive or storing VM snapshots) dedup might be of use, but you've not got anywhere near the RAM to tackle that safely. Much better to just use a proper incremental backup solution and only keep the files you need.

You don't need a 3rd party NIC as the onboard one for that board is an Intel.
 

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
Hi HoneyBadger,

Glad you agree on the ZIL/L2ARC front.

That would be a different story, yeah. Maybe down the line a more hardcore backup approach like that would be useful, at which point I would add a bunch more RAM. But this is just a daily updated clone, with a queue of rsync tasks to sync 8 HDDs, no imaging, no snapshots. I know it's a highly debated point, but would you consider that a "proper" backup solution? If by "incremental" you just mean "updated in small bursts," that's what I'm going for, but if you mean versioned, like ZFS snapshots or Time Machine, that's not really what I'm after.

Phew, no discrete NIC required!
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Hi Choco,

By the "incremental" one I meant moreso if you were storing complete images or backing up the whole system, so that it would store a full backup once a week/month, and then only the changed file(s) every night. This is valid because it lets you roll back to any point along that chain. I like rolling snapshots for that reason. "Whoops, I shouldn't have deleted file.txt two days ago. Better open the .snapshot folder, go back in, and get it."
 

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
Hi Honeybadger,

Ah, I gotcha! Yeah, I have never really had much need for that functionality, and it's a bit too RAM-intensive to jump on it for the gee-whiz factor. But I might look into it later if needs must. My fiance currently uses Time Machine, so if she's used to that, I might want to introduce snapshots to the setup. If I just fill the other two SO-DIMMs on my motherboard with 8GB sticks, I'll have 32GB for my 16TB unique space, which I understand might be enough for deduplicating snapshots.

How do you handle your changed-file backups between full ones?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Hi Honeybadger,

Ah, I gotcha! Yeah, I have never really had much need for that functionality, and it's a bit too RAM-intensive to jump on it for the gee-whiz factor. But I might look into it later if needs must. My fiance currently uses Time Machine, so if she's used to that, I might want to introduce snapshots to the setup. If I just fill the other two SO-DIMMs on my motherboard with 8GB sticks, I'll have 32GB for my 16TB unique space, which I understand might be enough for deduplicating snapshots.

How do you handle your changed-file backups between full ones?

First of all, avoid dedup like the plague unless you know exactly what you're doing. That also means something on the order of 64GB of RAM to start with, probably more, for 16TBish.

Second, snapshots perform a sort of "temporal dedup" , due to ZFS being Copy-on-Write. They can't be deduped, nor would such a thing make sense.
 

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
Hi Ericloewe,

Thanks for the tip! Really, part of my avoidance of dedup is that I know you have to be careful to do it right. I remembered a bit from reading introductions to ZFS that because of copy-on-write, snapsnots and dedup don't mix, but as snapshots aren't yet something I want to use, I didn't really pursue the research much further.

If I go down that road, I understand I have a lot of reading ahead of me to get it right. Also, that's a shocking amount of RAM. All I really kept in mind is that you have a base amount of RAM plus an additional quantity per TB of storage, and after a little bit of math, the amount of storage I wanted to have made my RAM requirement really difficult to hit, both in cost and in board capacity, so that helped rule out dedup as well.
 

Choco

Cadet
Joined
Oct 23, 2014
Messages
6
I'm considering a change to my build in terms of hard drives. I'm going to be hobbling along on my 2x3TB setup for a good while, and it looks like 6TB drives are already more prominent on the market than I'd realized, so I'd like to use 6 of those instead of my original plan of 6x4TB drives. I figure by the time I get around to putting in a full array of drives, the 6TB ones will be better established, so I might as well make the jump. That raises a couple questions:

Are there any concerns with drives of that size having stability issues? I understand that brand and product line makes a big difference, but I often hear miscellaneous warnings about using the largest drives being sold, whatever the brand. (In case anyone wants to scare me off, the ones that caught my eye are WD Reds.)

I've heard that a rule of thumb for RAIDZ2 RAM requirements is 1GB per TB of usable space, so would 24GB of RAM likely be enough for this build? (I have definitely ruled out dedup, especially with an increase in capacity like this.) The usage of this is still going to be media server/ rsync backup, nothing especially resource-intensive.

I'm realizing more and more how little I know about FreeNAS, so forgive any faux pas I may be making here...
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
There are no known specific issues with WD Reds of any size, but they are newish.
I wouldn't worry about it.

As for RAM, in home settings that rule is rather flexible. 16GB may be enough. 24GB will most likely be enough.
More RAM is always better, of course.
 
Status
Not open for further replies.
Top