Adding some Metadata SSD. Dumb it down for me.

[CarlB]

Hey guys, you help dumb down the metadata vdev? I read some other threads and it isn't really clicking for me.

I've got a pool of 8 disks in RAIDZ1. There is an old intel 400GB PCIe SSD in my server. This thing is basically 4x100GB. So I used 1x for cache and 1x for log, which these added without issue to the existing pool. (If curious this did massive performance increases for iSCSI performance. Reads and writes were bogged way down for me over 10gig.)

If I try to add a metadata vdev, I get the following warning. "Metadata vdev must be the same type as the data vdevs." This sounds like it wants me to have a raidz set of SSD's for the metadata, but is that actually necessary? Just want to make sure that these 2x100GB SSD's are not usable. I don't mind buying enough SSDs to have raidz for metadata, but trying to understand the distinction and use the SSDs I have if possible.

Thanks!

 

[sretalla]

The first thing to understand is that a metadata VDEV is not at all like L2ARC nor SLOG in that the data that's going to live on the special VDEV will ONLY live there (and not on the other VDEVs in your pool at all)...

Following on from that, if you were to lose your metadata/special VDEV, you lose your pool.

Hence the warning that you need to have the same consideration for redundancy with that VDEV as for the others in the pool.

Clearly you can use your grey matter and decide that a mirror would be good enough to go along with your RAIDZ1... but be aware that if one of the drives in that mirror is lost, your pool is at risk of loss with no redundancy in that VDEV.

 

[CarlB]

Thanks, wasn't aware of the distinction that they become part of the data pool. I'll plan accordingly.

 

[sretalla]

Also worth a note that adding one to an already existing pool doesn't put the metadata on the newly added metadata VDEV, only the metadata for new and updated files...

You can do a ZFS send | zfs recv to a new dataset on the same pool (and do some renaming, etc) to effectively force it to happen... or just delete and copy back all of your pool contents.

 

[Astronaut]

I wonder if it would be a better investment to add additional RAM for caching (metadata) than to add two metadata SSD's. That would also keep complexity down.

 

[CarlB]

I wonder if it would be a better investment to add additional RAM for caching (metadata) than to add two metadata SSD's. That would also keep complexity down.

I'm maxed out on RAM, but appreciate the idea.