Drive layout question

[dTardis]

I guess I am, however I still feel everyone should make up their own minds on how they want to run their pool. I just want to ensure they know what the options are, not what I personally prefer. Personally I can't imagine having 16 2TB drives for a home system, it seems a bit excessive and I'd opt for fewer larger drives. 16 drives is a lot to maintain as when the drive start to fail, replacing them will be expensive. 23TB of storage for a home is a lot of data and if the data ever needs to be moved off, well like I said, that is a lot of storage.

@dTardis do you really need that large of a pool? What are your real storage needs? I'm asking because of my comments above, that is a lot of drives to manage. In my situation I have six 2TB drives in a RAIDZ2 configuration. Even with all my movies and backups (I have a few backups for certain) I'm only using almost half my capacity. Now lets say I need to expand my capacity well I can do it one of a few ways... My personal favorite way is the most economical and that is with good planning and I forecast my data growth and I replace my 2TB drives with say 4TB drives to double my capacity, however I replace them as they either fail or as I can afford them. Of course you will not see an increase in your pool size until all the drives have been upgraded but it's better for your wallet. Other ways to grow your pool size it to add vdevs which means more hard drives to manage and is good for a large office, not the home office.

So if you can survive using a smaller handful of the 2TB drives for now, then replace them with larger drives in the future, that would be what I'd recommend you do. But it's up to you to figure out your storage requirements. You could always use 8 drives for an off-site backup or some other project, or keep a few for replacements.

Sorry, I get long winded.

Those are some great thoughts. So currently I am using a Synology 8bay box. The last time I checked I am north of 9TB of data. The Freenas box that I am currently working on, I am 0(zero) Dollars in. It came with 16 drives, so I figured why not use them, but your comments are appreciated and I will have to think on pairing down the install. It may be advantageous to not use all the drives just because I have them.

 

[joeschmuck]

Well you hopefully have read about where ZFS begins to choke from a capacity perspective and that in general you shouldn't exceed 80% capacity, 90% will suck the life out of your system. I plan for 3 years of growth (the warranty period of my drives). I too have a second NAS, but not fancy or anything but I can retain a good copy of my FreeNAS data there without issue, well about 80% of my data, but more importantly the important stuff. I wouldn't dream of getting rid of that second old NAS, it works great and it's paid for.

 

[dTardis]

Well you hopefully have read about where ZFS begins to choke from a capacity perspective and that in general you shouldn't exceed 80% capacity, 90% will suck the life out of your system. I plan for 3 years of growth (the warranty period of my drives). I too have a second NAS, but not fancy or anything but I can retain a good copy of my FreeNAS data there without issue, well about 80% of my data, but more importantly the important stuff. I wouldn't dream of getting rid of that second old NAS, it works great and it's paid for.

I think I read that, but I'm very glad you reminded me. So currently I am using just short of 11TB of storage which is 84% of my current available. I'm no longer growing very quickly, but consistently.

 

[Montel Bahn]

I guess I am, however I still feel everyone should make up their own minds on how they want to run their pool....

Or pools plural right?
Am I correct in that there are no hidden drawbacks, bugs or gotchas if someone wants to run 2 pools?
Seems this option always gets short shrift, though I think it has many subtle advantages. (i.e. you'd be able to replace 4 or 6 drives, say from 2TB to 6TB, (or whatever the size sweetspot is value-wise in a few years) without having to replace all 16 to grow your storage space!
Also HDD spindown doesn't really seem to function and 8 drives use about 100$ of power, at 10c per KW/h, per year...

 

[Mirfster]

Or pools plural right?

You can run as many pools as you want. But administration may become a bit of a PITA.

Seems this option always gets short shrift, though I think it has many subtle advantages. (i.e. you'd be able to replace 4 or 6 drives, say from 2TB to 6TB, (or whatever the size sweetspot is value-wise in a few years) without having to replace all 16 to grow your storage space!

Doesn't make sense to me. A Pool consists of one or more vdevs; it is the vdevs that consists of one or more drives.
So, I can have a Pool consisting of a single vdev that is of all 16 drives; in which case I would then have to replace all 16 to "autoexpand"
But, I can also have a Pool consisting of 2 vdevs where each has 8 drives; now I only have to replace 8 at a time to "autoexpand"
Same scenario scales to a single Pool with say 4 vdevs consisting of 4 drives, so only need to replace 4 drives at a time

 

[Montel Bahn]

You can run as many pools as you want. But administration may become a bit of a PITA.

Doesn't make sense to me. ....

You're right obviously about the autoexpand. (Sorry if I was misleading).
The 16 wide Z2 or Z3 is unrealistic; probably off the table?
3 vdevs, of say 5 disks, is a sane option. Also, a pool with 4 vdevs might be considered.
I guess performance would be good? Is it acceptable practice to do Z1 and Z2 with 4 disks now? I thought I read that somewhere.
The pool would be unbalanced as it expanded, but I guess that's not a big deal right?

I just hate the idea of having to keep 10-16 drives spinning to get at my data.
I find it unwieldy...and, counter-intuitively, limiting future expansion.
So you can never go back to ATX towers, let alone a mini-atx (like the FreeNASmini, which has what, 4 sata ports?) or repurpose a useless old pc...
say if your big shiny's power supply or mobo dies.
Simply re: transfering TBs of data... make sure you can get your Synology networking properly with FreeNAS, before you decide.

I'm all ears re: how 2 pools could become a PITA. I hate complexity too, but we're not talking an extra layer of abstraction.

 

[Mirfster]

The 16 wide Z2 or Z3 is unrealistic; probably off the table?

I wouldn't do it, but meh to each their own.

I guess performance would be good? Is it acceptable practice to do Z1 and Z2 with 4 disks now? I thought I read that somewhere.

I would think more vdevs = more iops. As to acceptable practice, that is up for debate I suppose.

The pool would be unbalanced as it expanded, but I guess that's not a big deal right?

Not necessarily unbalanced. Would be at first since stagnant data would not be re-allocated across; but as new data and files get modified I would think that they would balance somewhat.

I just hate the idea of having to keep 10-16 drives spinning to get at my data.
I find it unwieldy...and, counter-intuitively, limiting future expansion.

True, but it would depend on where you started. If a person started with 4 x 4TB drives instead of 16 x 1TB drives then increasing size would be a much simpler process. Design considerations :)

I'm all ears re: how 2 pools could become a PITA. I hate complexity too, but we're not talking an extra layer of abstraction.

Just from the aspect of permissions and datasets. Having 2 many not be that bad, but as that number increases it would only make things more cumbersome. I personally keep track of the hard drive serial numbers and what vdev it is part of. Don't think I want to further have to keep track of what pool it is part of as well.

 

[depasseg]

I'm all ears re: how 2 pools could become a PITA. I hate complexity too, but we're not talking an extra layer of abstraction.

To me, having multiple pools for releated data is the same as having multiple external USB drives for the same type of data. If you have a dataset on one pool with all your media, for instance, and that pool is full, yet your other pool with user shares has TB's of free space, what are you going to do? The only real option is to buy additional drives for your full poll even though you have free space. Or I guess you could start a second media dataset on your pool, and then try to remember which media pool that file resides on.

 

[joeschmuck]

Multiple pools might be fine if you were trying to keep data separated, lets say Company Internet Website and on a separate pool Employee Payroll & Privacy Act Data. Actually I think for something like that it would be two different servers on two different networks/subnets. Eh, it was a shot.

 

[Montel Bahn]

I think maybe my personal SOHO and family use cases bias me quite a bit. Plus I'm a cheap hoarder with grab bags of drives that are orders of magnitude apart in size. So even when I get around to remote replication with snapshots, it'll probably involve starting with mundane pool-to-pool shifting of data, maybe sans-network even, to say a 5 disk zpool. I think off-site, cold storage of a zpool, (also a viable option I think for some people) might be less complex if the pool has less disks. Now that I think about it, not having an HBA card with extra connectivity is really what is making me a little wonky in my conceptions. Sorry for the sidetrack, and thanks for the musings since they've brought some clarity my way.

... It may be advantageous to not use all the drives just because I have them.

... Let us know if you run into any SNAFUS.

 

[dTardis]

I think maybe my personal SOHO and family use cases bias me quite a bit. Plus I'm a cheap hoarder with grab bags of drives that are orders of magnitude apart in size. So even when I get around to remote replication with snapshots, it'll probably involve starting with mundane pool-to-pool shifting of data, maybe sans-network even, to say a 5 disk zpool. I think off-site, cold storage of a zpool, (also a viable option I think for some people) might be less complex if the pool has less disks. Now that I think about it, not having an HBA card with extra connectivity is really what is making me a little wonky in my conceptions. Sorry for the sidetrack, and thanks for the musings since they've brought some clarity my way.


... Let us know if you run into any SNAFUS.

I will. I really enjoyed the back and forth. I'm not sure if having all the flexibility available is always a good thing as important decisions need to be made and the outcome can be hard to predict. This is one of the reasons I like what we are using for production equipment where I work.