Are we any closer to vdev expansion?

[NickF]

You cold have just added 2 more disks instead of 4 more disks.

 

[Whattteva]

How so?

With mirrors, you only have to add/replace 2 disks at a time instead of 4 at a time in your current setup.

I did have a plan before reaching this point: vdev expansion had an ETA of 2023, so I created a vdev I believed was large enough to last me through at least 2024. I just...got it very badly wrong. My Synology is 70TB and took six years to fill, my TrueNAS is 35TB and filled in about a year.

Terrible plan. You should never make a plan based on vdev expansion. You have to realize that we are dealing with a file system here, not just some word processor software that when it crashes, you can just be "meh" and restart it. It has be tested and works near perfectly before it gets pushed into the wild. Would you be happy if you got a Christmas present that you thought expanded your pool, but instead thrashes and corrupts that 35 TB of data you have amassed over the years? I don't know about you, but I would rather have a vdev that never expands vs a vdev that thrashes my files.

 

[koberulz]

Mirroring would mean one redundant disk per data disk, though. Were I able to expand the vdev, I would be adding zero redundant disks going forward.

Again, I was expecting this storage to last me until at least the end of 2024, probably longer. It was a very long-term plan.

 

[Whattteva]

Mirroring would mean one redundant disk per data disk, though. Were I able to expand the vdev, I would be adding zero redundant disks going forward.

Not really. This is theoretically true, but doesn't mean it's a good idea to do in practice. I run striped mirrors, but if I were to run RAIDZ, I would never go beyond 10 disks at most. There are posts on these forums of people running into issues with vdev sizes of 20 and they usually have performance and other issues. Even ZFS core developers discourage using very wide vdev sizes though they don't specifically state a hard limit.

ZFS is very robust and quite frankly, the best file system I have ever used since I started using computers 20+ years ago, but it is not magic. You can't have your cake and eat it too unfortunately. Some tradeoffs have to be made. You can't have performance, upgrade flexibility, storage efficiency, cheap all at once.

 

[Davvo]

Mirroring would mean one redundant disk per data disk, though. Were I able to expand the vdev, I would be adding zero redundant disks going forward.

You phrased it wrong, but I understand what you mean (not adding redundancy to the entire pool).

Each time you add a (2-way mirror) vdev, you add a data disk and a parity disk. Each VDEV actually has a redundant disk; your pool is more exposed to data loss, but it's the price to pay for ease of expansion (and performance).

Also do note that mirrors can be N-wide, and not only 2-wide, where each disk adds parity.

 

[koberulz]

Okay so there's 16TB or so that's unrelated to the media server and can be put elsewhere. But if I just get two more 20TB drives, that simply shifts the problem...

 

[Davvo]

Okay so there's 16TB or so that's unrelated to the media server and can be put elsewhere. But if I just get two more 20TB drives, that simply shifts the problem...

I mean, there isn't much else you can do: RAIDZ expansion isn't there, buying enough disks to add another VDEV to your existing pool is not an option, recreating the pool is not an option... I'm out of ideas.

Right now your best strategy appears be to create a new pool with two new disks in order to carry on, then put those drives in your original pool once RAIDZ expansion is a thing. Doing so will also give you the option to continue expand the new pool as needed until you can merge everything. You will have to figure out where to temporarely put some of your data while you do the merging, but that's an issue for the future you.

 

[koberulz]

I also need to figure out how to physically get the disks into the case. It's been so long since I set it up, and I'm not even sure if the case will take them without added bits and pieces...

I assume you would recommend mirroring the two new drives?

 

[Davvo]

I assume you would recommend mirroring the two new drives?

Yup, it guarantees you redundancy while keeping the number of drives you have to buy down.

 

[koberulz]

Well it looks like this is gonna take a while. After four hours, TeraCopy has added a little over two million files totalling just 838 GB, and transferred nothing.