So, 4 disks leave an 8-disk ZFS array, and one sez...

[RedBear]

... he sez, "Hey, did we just irretrievably crash that ZFS array by leaving at the same time?"

I kind of asked this question a couple months back before I did my initial FreeNAS install. This is just a theoretical follow-up question to make sure I understand exactly what will happen in a certain scenario.

Given a machine with several built-in SATA ports and also an installed M1015 card (in IT mode), with 4 SATA disks connected to the built-in SATA ports on the motherboard and 4 other SATA disks connected to one of the mini-SAS ports on the M1015, and all 8 disks then put together into a single 8-disk RAID-Z2 (or Z3) array, what happens when one of the two different SATA controllers suddenly dies while the pool is in active use?

A) The ZFS array is instantly and irretrievably unrecoverable because it was Z2/Z3 and it "lost" 4 devices, which neither of those RAID schemes can handle?

B) ZFS just immediately takes the pool offline and waits for me to find some way to reconnect those 4 devices (replace HBA and/or motherboard or move the drives to the SATA controller that didn't die), and then the pool imports just fine and goes on its merry way?

I know it's fine and quite common to use multiple SATA controllers with FreeNAS arrays, I just want to make sure I fully understand what sort of Murphy's Law probabilities I'm going to be playing footsie with if I go from 8 disks directly connected to an M1015 to 11 disks with 3 disks connected to the motherboard.

Am I pondering excessively negative possible outcomes needlessly again?

 

[SweetAndLow]

B

 

[titan_rw]

Yes, B.

The pool will be in a faulted state and inaccessible. Generally reconnecting the drives and rebooting will bring the pool back. Any data buffered in memory that hasn't been committed in a transaction group will of course be lost.

I had this happen on more than one occasion when I was running a nas on an old Core2Quad that used the nvidia nforce chip. A bit flakey for freenas. It would randomly drop drives. I had a z2 pool, and it definitely happened where I'd lose 3 or more drives, sometimes all at once, sometimes one at a time until the pool faults. Once the pool faulted while my desktop was pushing backups to it. Of course the backup that was in progress was corrupted, but on reboot everything else was ok.

I was extremely impressed at how FreeNAS handled running on such crappy hardware. More than once I felt that if it was typical hardware raid I would have been done. I've since moved on to better hardware, but it was fun watching it cope with it at the time. It was only ever holding backups, so losing the pool wouldn't have been the end of the world anyway.

 

[RedBear]

These responses are reassuring. But now I'm wondering exactly what does finally cause a pool to be unrecoverable.

 

[Ericloewe]

These responses are reassuring. But now I'm wondering exactly what does finally cause a pool to be unrecoverable.

Corruption (unlikely or impossible with ECC) with working drives or, more realistically, enough unrecoverable drive failures (total, soft, whatever).