Some thoughts on data safety and redundancy reg. hardware config

[Steve M Cline]

A while ago I read somewhere that it would be advisable
to attach disks to different controllers in a ZFS mirror.
This makes total sense to me.
A disk controller is a device that can potentially go bad
any time just as a harddisk or any other device.
When it goes bang, it could potentially send bad data to
all disks attached to it, blasting your zpool.

That would mean that one actually needs a multitude of disk
controllers for enhanced redundancy. In a Raid-Z2, only 2 disks may be
connected to any controller, in a Raid-Z3 a max. of 3 disks,
to prevent the zpool from getting destroyed in case a controller goes berzerk.
More than 3 disks attached to one controller is a single
point-of failure, putting your zpool at stake.

I know this is not really practical for large arrays, but is my theory right?

Actually the same problem exists (even worse) for power supplies.
A failing power supply sending a high voltage blast
could burn all harddisks in one go.
Having separate power supplies for small sets of 2 to 3 harddisks
each is probably even more impractical.

Hence, to have one's data even remotely safe, one needs
one's data mirrored to at least two zpools on different physical servers each.

What do you think?

 

[Madd Martigan]

I know that this post is pure blasphemy in this forum but one would question why you would use such marginal equipment in the first place if your requirements are so strict. In an enterprise environment you would buy the physical hardware that offers the necessary reliability and, in my current professional role, I haven't seen the need for that level of reliability. Even in enterprise class SAN storage systems the RAID sets aren't distributed in a manner that you describe. Disk meta groups are carved out across multiple RAID groups in these high end solutions but even those RAID groups are still connected to a pair of redundant disk controllers.

The same thing goes for power supplies. The point of redundant power supplies is just that; redundancy. I've never seen (nor do I expect to ever see) a configuration where you have power supplies dedicated to specific devices within a system. The only servers that I have seen that have more than two power supplies do so because they need more power than a single, redundant pair can deliver from the connected circuit. An example of this would be that you have a server that has four 1200W power supplies of which it requires greater than 1200W of power for the system. Currently I deal with HP ProLiant hardware and the only examples I have seen from that line of hardware are the four or eight socket servers that support greater than 1TB of RAM and have a myriad of disk and peripheral expansion options as well as support for four 130W processors or more.

At the end of the day you may be trying to polish the proverbial "turd".