Final build - comments/suggestions

[Allan_M]

Hi everybody

This is supposed to be the final thread/sanity check before hitting 'buy'. You can check the previous wall of text, describing thoughts behind and use case here; 20 bay build suggestion/opinions

• Case: Norco RPC-4220 (~$45)
• PSU: Seasonic X Series Gold - 850W (~$160)
• Mobo: Supermicro X10SL7-F (~$260)
• RAM: 4x 8 GB Samsung memory D3 1600 8GB ECC (~$250)
• CPU: Intel Core i3-4170HR (~$105)
• HBA: LSI SAS 9211-8i (~$60)

Total cost (without drives), based on local availability and prices: ~$880
Not included: cables, USB drives and other smaller parts.

I have two spare SSDs on the shelf; Don't know if they could be put to good use in some way (SLOG/L2ARC)?

 

[Yatti420]

I haven't been around in awhile but looks decent.. Mirrored boot/system drives maybe for SSDs instead of USBs.... Looks fine i'd say.. The case is ok? It is a server chassis.. $45? Seems like a steal.. Why was this not advised? You going to do a z2 pool? I run 6x2tb z2pool.. You don't have to run datasets fyi.. Just create the pool and dump data in.. For single users probably wont matter..

 

[Allan_M]

I haven't been around in awhile but looks decent.. Mirrored boot/system drives maybe for SSDs instead of USBs.... Looks fine i'd say.. The case is ok? It is a server chassis.. $45? Seems like a steal.. Why was this not advised? You going to do a z2 pool? I run 6x2tb z2pool.. You don't have to run datasets fyi.. Just create the pool and dump data in.. For single users probably wont matter..

The case I found used. It's perfectly fine disregarding the cosmetics.

I would, initially, buy five 2- or 3 TB drives and set up a Z2 for a total of 6-9 TB of usable storage. The immediate requirements is around 6-8 TB of storage for backups and media storage - if everything currently residing on external USB3.0 and FW800 drives was put into one 'pot' (or, if you want, 'pool').

Then buy a disk every other month, and after a year or so - have two mirrored pools for extra redundancy or two striped pools for capacity. The final setup is meant to last for the next 8-10 years - so added cost isn't a concern when it is spread out over such a long period. With 20 bays, the logic was that the primary focus was on performance and redundancy. Should the need for more capacity arise, I would then start swapping out the 'smaller' 2/3 TB drives with larger drives and grow the pools that way.

The terminology; pools, volumes, shares, datasets and so on, are still a little confusing to me - since I've grown used to the terminology from regular consumer NAS'es.

Thank you for your contribution - it's very much appreciated :)

 

[Yatti420]

Yea thats fine.. Start simple.. Get your first z2 pool setup with 6x2tb .. Is what I run should leave you around 6tb.. If you need all that might aswell buy 4tb drives though.. Make it easy.. Spend the cash now etc.. Get that extra space off the start and wait to buy the 6tb or 8tb when cheapest over long run..

 

[Klontje]

Hi Allan, adding a single disk every now and then isn't gonna work with ZFS (at least it's not gonna give you something extra until you expanded every disk in your vdev to the new size). This has to do with how ZFS treats vdevs (bundled disksets). Maybe take a look at the intro cyberjock has written: https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/ This will explain how ZFS works and what is and what isn't possible.

 

[Allan_M]

Yea thats fine.. Start simple.. Get your first z2 pool setup with 6x2tb .. Is what I run should leave you around 6tb.. If you need all that might aswell buy 4tb drives though.. Make it easy.. Spend the cash now etc.. Get that extra space off the start and wait to buy the 6tb or 8tb when cheapest over long run..

6x 2 TB@Z2; wouldn't that equate to 8 TB usable? I meant 5x 2 TB@Z2; which should equate to 6 TB with 4 TB of parity - please correct me if I'm wrong.

One of the points, with buying a 20 bay enclosure in the first place, was that I didn't have to fork out for the (where I live) 'much' more expensive +4 TB drives. I know, it you calculate $/TB, the larger drives are cheaper. But the expense up front is larger.

Hi Allan, adding a single disk every now and then isn't gonna work with ZFS (at least it's not gonna give you something extra until you expanded every disk in your vdev to the new size). This has to do with how ZFS treats vdevs (bundled disksets). Maybe take a look at the intro cyberjock has written: https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/ This will explain how ZFS works and what is and what isn't possible.

Not to come of harsh; but I know. What I meant was - something along the lines of: Imagine that I buy a drive every other month. Then, after 10 months, I would have bought (or saved up enough money to buy) 5 new drives, that I could add to a RAIDZ2 and stripe or mirror with my existing pool*. With 20 bays, and the first five drives up and running initially - that would mean around 30 months before all 20 bays would/could be populated. I've read the primers, guides and manuals (both the FreeNAS and the Oracle one), so I'm quite confident in my strategy. But English is not my native language - so some things are bound to get lost between the lines and the finer nuances when writing on an online forum :)

* in the long run: One of the two following possibilities:

1: Imagine two sets, of two sets with five drives

• Pool1: 6 TB (two mirrored RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIZz2 (5x 2 TB), 6 TB
• Pool2 6 TB (two mirrored RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIDZ2 (5x 2TB), 6 TB

Total capacity: 12 TB

2: Imagine two sets, of two sets with five drives

• Pool1: 12 TB (two striped RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIZz2 (5x 2 TB), 6 TB
• Pool2 12 TB (two striped RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIDZ2 (5x 2TB), 6 TB

Total capacity: 24 TB

 

[Klontje]

True, I was triggered by your comment "The terminology; pools, volumes, shares, datasets and so on, are still a little confusing to me - since I've grown used to the terminology from regular consumer NAS'es." This came across like you wanted to extend your Hybrid RAID ;)

 

[Allan_M]

True, I was triggered by your comment "The terminology; pools, volumes, shares, datasets and so on, are still a little confusing to me - since I've grown used to the terminology from regular consumer NAS'es." This came across like you wanted to extend your Hybrid RAID ;)

... mocking will not be tolerated :p

 

[Bidule0hm]

1: Imagine two sets, of two sets with five drives

• Pool1: 6 TB (two mirrored RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIZz2 (5x 2 TB), 6 TB
• Pool2 6 TB (two mirrored RAIDZ2's)
  • RAIDZ2 (5x 2 TB), 6 TB
  • RAIDZ2 (5x 2TB), 6 TB

Total capacity: 12 TB

You can't mirror vdevs.

 

[Allan_M]

You can't mirror vdevs.

To my understanding, that would be possible via a 'netsted vdev', but I might have grotten it wrong

 

[Bidule0hm]

Nested vdevs? Do you have any doc on that?

 

[Allan_M]

Nested vdevs? Do you have any doc on that?

Link: https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/

Perhaps I got it wrong, and it's only possible with mirrors, which in itself is a vdev?

Stupid syntax, I know, but perhaps it can explain what I had in mind:

zpool create pool1 mirror raidz2 disk1 ... disk5 raidz2 disk6 ... disk10
​

EDIT: Ah! I think I understand now. I can't work, because both mirror and raidz2 are vdevs - right?

 

[cyberjock]

EDIT: Ah! I think I understand now. I can't work, because both mirror and raidz2 are vdevs - right?

Yep.

You could do a very bastardized RAIDZ2 where every disk has a mirror. But I'd never recommend you go that route because it's very, very non-standard to do that. Especially in FreeNAS. You might be the only one on the planet that goes with that configuration, which means you'll likely be that one sucker that lost all of his data because of some edge-case scenario.

If you really want lots of redundancy, go with RAIDZ3 on 7 disks instead of 6 disks. :)

 

[Bidule0hm]

Yeah this is not nesting. Nesting would be to have RAID-Z1 vdevs inside RAID-Z2 vdevs mirrored together for example (yes, that's two levels of nesting).

You can only stripe vdevs ;)

 

[Allan_M]

... You might be the only one on the planet that goes with that configuration, which means you'll likely be that one sucker that lost all of his data because of some edge-case scenario.

If you really want lots of redundancy, go with RAIDZ3 on 7 disks instead of 6 disks. :)

Well. My mum keeps telling me I'm special

The reason for doing RAIDZ2 with 5 drives, was because it worked out 'beautifully' with 20 bays. Two pools, each with 10 drives. Aggregated, with 2 TB drives; 24 TB (80 % ~ 19 TB). I'm not sure there is any other reason behind this logic, than it just worked out?

Yeah this is not nesting. Nesting would be to have RAID-Z1 vdevs inside RAID-Z2 vdevs mirrored together for example (yes, that's two levels of nesting).

You can only stripe vdevs ;)

'RAIDception' much? I think I get it now, and I humbly apologize :)

 

[Bidule0hm]

No need to apologize, I was just curious about vdev nesting because I never heard of them before :)

 

[Allan_M]

No need to apologize, I was just curious about vdev nesting because I never heard of them before :)

I imagine. As my info states, I'm a teacher - So I'm myself used to hearing a lot of weird stuff, almost on a daily basis, that doesn't always seems to make sense. But once the students starts explaining, it usually stems from some sort of misunderstanding :)

 

[Yatti420]

You can run four drive z2pools if you wanted.. Around 7.3TB usable but you have to take a percentage off that for overhead I believe.. I'm pretty much maxed out here... http://www.servethehome.com/raid-calculator/