Advice on how to split vdevs across backplanes in enclosure

allegiance

Explorer
Joined
Jan 4, 2013
Messages
53
I'm creating a new pool in a new enclosure and was wondering if there is any benefit in the way I spread the vdev across the backplane/HBA channels. 24 bay enclosure with 6 x SFF8087 connectors, each one connecting to 4 drives on the front end and a single SFF8087 port on my HBA / storage controller. I plan on creating 4 x 6-disk raidz2 vdevs in this enclosure.

I was going to just start with bay 1 in the upper-left corner and keep moving left-to-right then down to the next row until it's done. Let's call it Option B. That would put the first 4 disks of the first vdev on one "channel", then the last two disks of that vdev on the next "channel" with the first two disks of the next vdev, etc.

Is this fine, or should I have each of the 6 disks from each of the 4 vdevs on a separate "channel", or does it not matter? Let's call that Option A.


I've attached a picture that hopefully explains it better than I did with many words. Thank you.
 

Attachments

  • Screen Shot 2023-01-10 at 7.29.55 PM.png
    Screen Shot 2023-01-10 at 7.29.55 PM.png
    75.5 KB · Views: 177
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,700
Bandwidth won't be an issue if they are HDDs. No difference in the options for performance reasons (IMHO).

Lose any of the backplanes with Option B due to some electronic fault or cable issue and your VDEV is gone (or at least unavailable)... meaning also your pool.

With Option A you can lose a backplane (or even 2) with degradation, but no pool unavailability.
 

allegiance

Explorer
Joined
Jan 4, 2013
Messages
53
Bandwidth won't be an issue if they are HDDs. No difference in the options for performance reasons (IMHO).

Lose any of the backplanes with Option B due to some electronic fault or cable issue and your VDEV is gone (or at least unavailable)... meaning also your pool.

With Option A you can lose a backplane (or even 2) with degradation, but no pool unavailability.
Thank you. They are HDDs, so does sound like bandwidth is not an issue then. As I thought about it more, I was leaning towards A because of the redundancy, but was wondering if I was missing something.

I just realized that I mis-labeled my image. So much for making it clearer. I'll edit the post's text to match the image.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,079
You wouldn't be bandwidth constrained in that hardware even with SSDs. Rough numbers, but even the most optimistic performance from mechanical disks and you can run 24 on a SAS expander backplane with no loss of performance.
Also, if you loose connectivity to a section of the pool, the pool goes down, but when you restore connectivity to the pool, the pool should survie. I found that out by personally having a SAS controller fail taking out the data path to half my drives. Once the SAS controller was replaced the pool was able to be accessed again.
 
Top