What pool type extendable and fast write?

[katit]

In general I understand differences between stripes/mirrors/etc. In practice - I have 4tb mirror (2x WD reds)
Now I am at a point where I need more space and I am ready to get more storage. At the same time I am OK with leaving existing 4TB pool in place.

New pool will need to be:
a. Write-efficient (not much reading) - it's for backups mostly
b. Failure resilient
c. At least 10Tb
d. Easily expandable if I need more storage later
e. Minimum count of physical discs

What kind of pool would your recommend?

Thank you!

 

[jgreco]

That's mirrors all the way. Go with a two-way mirror of 14-22TB disks (depending on what is "affordable") and that's just two drives. When you need more space someday, add another two disks.

 

[katit]

That's mirrors all the way. Go with a two-way mirror of 14-22TB disks (depending on what is "affordable") and that's just two drives. When you need more space someday, add another two disks.

Well, yes, but "easily expandable" part? Ideally I want "DatasetA" to be expanded when needed so it's clients don't need to know. With mirror (like now) - it's 4Tb and thats it.

I want (if possible and feasible of course) to do 10Tb(or so) now and be able to increase it later as needed.

 

[danb35]

With mirror (like now) - it's 4Tb and thats it.

No, it isn't--add another pair of disks and it's expanded.

 

[jgreco]

Well, yes, but "easily expandable" part? Ideally I want "DatasetA" to be expanded when needed so it's clients don't need to know. With mirror (like now) - it's 4Tb and thats it.

When you add a second vdev to a ZFS pool, that space serves everything on the pool. It's transparent to the pool users, except that they may see a bigger number reported for free space. It's almost magic.

 

[katit]

Oh, I see. So, what you saying I can just add 2 disks on top of my existing mirror (and I can add different size pair?) and that would just increase space on my already existing dataset?? If so it will make it even easier

 

[danb35]

and I can add different size pair?

Yes, you can.

that would just increase space on my already existing dataset?

It would increase space on the pool, which would in turn increase space (unless there was a quota in place) on any datasets contained in that pool.

 

[jgreco]

Yup. Your pool ends up looking like this:

Code:

        NAME                                            STATE     READ WRITE CKSUM
        storage4                                        ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/88c7c222-11f9-11ea-ae4c-0050569fe971  ONLINE       0     0     0
            gptid/60b9f1cc-1a05-11ea-bed9-0050569fe971  ONLINE       0     0     0
          mirror-1                                      ONLINE       0     0     0
            gptid/ebf66ef2-1a26-11ea-8355-0050569fe971  ONLINE       0     0     0
            gptid/ed4436e7-1a26-11ea-8355-0050569fe971  ONLINE       0     0     0
          mirror-2                                      ONLINE       0     0     0
            gptid/cd018df5-751c-11eb-941b-0050569fb18f  ONLINE       0     0     0
            gptid/d21b2288-b67e-11ec-8057-0050569fb18f  ONLINE       0     0     0

and when that gets too full, a mirror-3 will join it because this chassis has 8 bays.

 

[katit]

Yes, you can.

It would increase space on the pool, which would in turn increase space (unless there was a quota in place) on any datasets contained in that pool.

Thats great!
On a side note :) Why do you have this complex setup? Any certain reasons or "because you can"?
Pool: 6 x 6 TB RAIDZ2, 6 x 4 TB RAIDZ2, 6 x 8 TB RAIDZ2, 6 x 12 TB RAIDZ2

 

[katit]

Yup. Your pool ends up looking like this:

Code:

        NAME                                            STATE     READ WRITE CKSUM
        storage4                                        ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/88c7c222-11f9-11ea-ae4c-0050569fe971  ONLINE       0     0     0
            gptid/60b9f1cc-1a05-11ea-bed9-0050569fe971  ONLINE       0     0     0
          mirror-1                                      ONLINE       0     0     0
            gptid/ebf66ef2-1a26-11ea-8355-0050569fe971  ONLINE       0     0     0
            gptid/ed4436e7-1a26-11ea-8355-0050569fe971  ONLINE       0     0     0
          mirror-2                                      ONLINE       0     0     0
            gptid/cd018df5-751c-11eb-941b-0050569fb18f  ONLINE       0     0     0
            gptid/d21b2288-b67e-11ec-8057-0050569fb18f  ONLINE       0     0     0

and when that gets too full, a mirror-3 will join it because this chassis has 8 bays.

Thank you! Bonus question: If space permits, can I remove mirror out of storage? Or adding mirror is a non-reversible operation?

Why I ask. I have 2x 1tb drives I can add right now to go from 4tb to 5tb. But at the same time I may want to get rid of them later. They will use power just like bigger once, they will use slots, etc..

 

[jgreco]

Well, because it's really not complex. I anticipate that I will add another vdev. But because the existing disks are all 12TB, I may end up adding a more aggressively large pair for the last vdev, such as 22TB, and then I still have the option to upgrade the remaining three vdevs with 22TB drives as well. This means that I currently have 36TB of pool space or about 28TB usable on the pool currently, but I can expand it up to 88TB of pool space or about 70TB usable without much fuss. I'm not one of the people who buys space "just cuz".

 

[jgreco]

Thank you! Bonus question: If space permits, can I remove mirror out of storage? Or adding mirror is a non-reversible operation?

Historically it's been a non-reversible operation. There's active work on making it possible to remove vdevs under certain conditions but personally I just wouldn't consider trying it until it had a lot of people who have successfully done it. That's just conservative, cynical ol' me though.

 

[danb35]

Why do you have this complex setup?

It mainly has to do with increasing capacity needs over time and what disk capacity was available at attractive $/TB levels. What I've done is similar to what @jgreco and I are suggesting to you, just with RAIDZ2 vdevs rather than mirror vdevs.

If space permits, can I remove mirror out of storage?

Yes, you can. That's been available at the CLI at least since CORE 12.0, and it's in the GUI in CORE 13.0 and (I believe) all releases of SCALE. Note, though, that this option is only available with the pool consists only of single disks or mirrors; you cannot do this if your pool contains any RAIDZn vdevs.

 

[Etorix]

Well, yes, but "easily expandable" part? Ideally I want "DatasetA" to be expanded when needed so it's clients don't need to know. With mirror (like now) - it's 4Tb and thats it.

All wrong. With any layout, you always have to options to increase space: 1) add more vdevs; 2) replace drives in a vdev by bigger drives—upon replacing the last drive, the vdev expands to use the new capacity.
This goes equally for mirrors or raidz, but raidz# necessarily means more disks, adding more disks or replacing more disks.

 

[katit]

Well, because it's really not complex. I anticipate that I will add another vdev. But because the existing disks are all 12TB, I may end up adding a more aggressively large pair for the last vdev, such as 22TB, and then I still have the option to upgrade the remaining three vdevs with 22TB drives as well. This means that I currently have 36TB of pool space or about 28TB usable on the pool currently, but I can expand it up to 88TB of pool space or about 70TB usable without much fuss. I'm not one of the people who buys space "just cuz".

adding more disks or replacing more disks.

Ok, I guess we are onto something.
So, if I understand correctly, it can be like this:
2x4Tb - Current

I can add 2x1Tb vdev right now and have pool with 5Tb

I don't really need to "delete" vdevs, my point was more about replacing I guess, I just didn't realize replacing is possible.

So, now let's talk about replacing 2x1Tb vdev. How is that done?
Can I just remove one disk (while shut down server), replace with 10Tb and turn it on? It will re-silver it with data from remaining 1Tb, correct?
And then I can replace second 1Tb driver with 10Tb and then system will recognize that this vdev now 10Tb?

If it works something like that - then it's all I really need as I can replace/upgrade old pairs as needed.

 

[danb35]

Can I just remove one disk (while shut down server), replace with 10Tb and turn it on?

Yes, and then tell the GUI to replace the old disk with the new. Then this will happen:

It will re-silver it with data from remaining 1Tb, correct?

...and then you can do this:

And then I can replace second 1Tb driver with 10Tb and then system will recognize that this vdev now 10Tb?

 

[katit]

Ok. Makes sense and I don't need to buy anything then just yet as it seems to be easy deal when needed. I just though I will have to deal with shares and so on config on a new pool, but I really going to just live with one pool and expand on demand.

 

[Etorix]

Can I just remove one disk (while shut down server), replace with 10Tb and turn it on? It will re-silver it with data from remaining 1Tb, correct?

Yes. Even better, if you have a free slot and a free SATA port, you can (and preferably should) add the new drive without removing the old one, tell the GUI to replace the drive. No loss of redundancy while replacing.

 

[katit]

Well, something didn't go well. I deleted 1Tb dataset, erased all the data. All good.

Then I added those 2 disks as vdev to my existing 4tb dataset. It didn't go well, as one disk didn't properly add. Now I have this citutation:

And I have 2 disk hanging. How do I attach it to make it mirror with sdb?

 

[danb35]

How do I attach it to make it mirror with sdb?

You should be able to click on sdb, and then Extend on the right.