I have 3 Sans Digital TowerRAID TR5M+BNC - 5 Bay eSATA with a raid controller card. 14 ttl drives. Freenas running off ada14 (USB thumb drive).
Your TowerRAID TR5M+BNC is a port multiplier. Port multipliers are not expected to work. See
This resource was originally created by user: jgreco on the TrueNAS Community Forums Archive. Please DM this account or comment in this thread to claim it. In the last year or two, we’ve had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use...
www.truenas.com
RAID controllers are extremely dangerous for use with ZFS. See
1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...
www.truenas.com
Basically your system isn't expected to work at all, though occasionally by pure luck things can work out for awhile. I would not trust this system with any valuable or important data that you wanted kept safe.
Your experience with the resilvering indicates that the system is not completely stable under stress, and ZFS likes to stress systems. You might have gotten away with it this time, but since the vast majority of resilvering involves reads, and you still managed to rack up some checksum errors (possibly as a controller hang, given that it seems to have considered a file damaged).
My best theory, understanding how port multipliers work, would be that one of your disks indeed faulted, had a read error that consumed excessive time and incurred a timeout/bus reset/something-like-that, and as a result the other four drives sharing that port also lost I/O transactions as part of the reset. I bet if you identify the four "degraded" drives, they're all in the same chassis as the "faulted" drive.
You need to get rid of the port multipliers and the RAID card.
