Pool unhealty...how can this happen?

[Mark St.]

Its a pool of 2 SSD as a mirror.
Content is a encrypted virtual machine (running fine)

This happended this morning:

root@truenas[~]# zpool status -v
pool: M1-2x500SSD
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
scan: scrub repaired 0B in 00:18:04 with 0 errors on Sun Aug 8 00:18:04 2021
config:

NAME STATE READ WRITE CKSUM
M1-2x500SSD ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/42d94d06-6d35-11eb-9dd7-a0369f3aecec ONLINE 0 0 0
gptid/42e0b27e-6d35-11eb-9dd7-a0369f3aecec ONLINE 0 0 0

errors: Permanent errors have been detected in the following file
M1-2x500SSD/LAOTSE100sec:<0x1>

So what happened here?
Drives are in Mirror-Configuration, so errors should be corrected....both drives are online...

 

[sretalla]

So what happened here?

Perhaps a controller error? (same data written to both disks via the controller) Check the firmware on your HBA/controller. (since it's potentially connected to SSDs here and there's this: https://www.truenas.com/community/resources/lsi-9300-xx-firmware-update.145/)

Also would be interesting to investigate the type of the SSDs... are they using a controller that has known problems with FreeBSD and TRIM?

 

[QonoS]

Interesting! That is one reason why backup is essential and mirrors do not count as "backup".

Some Intel SSDs have a feature called "end-to-end data protection" that would probably have prevented that error from happening.
https://www.intel.com/content/dam/w...s/guides/ssd-technology-terminology-guide.pdf : "5. End-to-End Data Protection"

 

[Mark St.]

The Disks are Crucial MX500 (SM2258) connected to intel-onboard.
Because of the mirror-configuration, the propability of the disks being responsible should be low...
Something one level above the disk should be responsible

The virtual machine has Trueimage backups....mainly because i couldnt make the zfs-replication work... (
Source 'M1-2x500SSD/LAOTSE100sec' is a volume, but target 'Z1-3x10' already exists and is a filesystem.)

 

[sretalla]

Crucial MX500 (SM2258)

These use a silicon motion controller and I have seen many cases where file corruption occurs on these when they are used as boot drives, so I guess these are in fact the problem.

Take a backup of the pool and replace them with SSDs with a different controller (or disable trim and recreate the pool).

how to enable trim but not on the boot device?

my usb key was faulty so i found an old vertex ssd (like the first gen, 30 gig) that i had and it is working perfectly but it doesnt support trim i keep getting ata error status related to trim in the console, after a little of googling i found out that it was because of the trim on that ssd...

www.truenas.com

My bizzare trip to SSD boot drives (and I gave up)

My FreeNAS complains about boot pool errors every time it scrubs itself. The checksum of one of the USB thumb drives would always become 1 and I'd have to go to the CLI to clean it up manually every week. So with WD Green 120GB SSD becoming cheap enough (50 Canadian Rupees each tax incl.), I...

www.truenas.com

Those 2 links contain one (of the many) discussion on the topic and the needed setting (which you would do via tunable) to disable trim if that's how you decide to go. You could search the forum for silicon motion, 3D NAND, WD Green or SSD TRIM and see all the others.