Pool status unhealthy

[dias]

Hi,​

My current system is n5105 celeron 2x12 tb seagate exos (mirror) + 2x8 tb wd red pro 32 gb ram and i am using TrueNAS-SCALE-22.12.0
I've made a scrub for my drives lately and it gave an error during operation but continued now i am seeing this error in my dashboard:

Pool seagate state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.​

I've checked with zpool status and saw these messages:

pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:00:26 with 0 errors on Tue Aug 29 03:45:28 2023
config:

NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sdc3 ONLINE 0 0 0

errors: No known data errors

pool: seagate
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
scan: scrub repaired 0B in 14:09:08 with 1 errors on Mon Sep 4 08:37:10 2023
config:

NAME STATE READ WRITE CKSUM
seagate ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
4ced2a17-6b88-4bec-a301-ae54cb41d627 ONLINE 0 0 2
fbf0eef0-74af-4c71-9702-6371eaf3fbb1 ONLINE 0 0 2

errors: 1 data errors, use '-v' for a list

pool: wd
state: ONLINE
scan: scrub repaired 0B in 01:00:40 with 0 errors on Sun Aug 13 01:00:41 2023
config:

NAME STATE READ WRITE CKSUM
wd ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
b9e99684-cab8-4adf-add3-dacadae436ef ONLINE 0 0 0
d16360d8-1281-4b39-bfc0-585bb9f5f37a ONLINE 0 0 0

errors: No known data errors

it is the seagate one as you can see, should i worry about my drive? i dont wanna do another scrub as it took 14 hours to complete, when i run zpool status -v i see a movie file permanently corrupted which i dont care but my dashboard seems like this and i am worried, what should i check? and how can i remove this error message?

Thanks in advance

 

[sfatula]

You can clear the errors (though no reall urgency to do so) via zpool clear seagate. That you got a checksum error on both drives in the mirror for the same file seems pretty unlikely. So, I wonder about a memory error, you using ecc ram?

You should check or run smart tests on the drives anyway.

 

[dias]

You can clear the errors (though no reall urgency to do so) via zpool clear seagate. That you got a checksum error on both drives in the mirror for the same file seems pretty unlikely. So, I wonder about a memory error, you using ecc ram?

You should check or run smart tests on the drives anyway.

Thanks for the help i ran short smart test and it didnt give any errors, my ram is not ecc btw as you can understand from my cpu ^^ it is not capable of ecc ram. Btw i can access my files without any problem just worried. I bought the rams a few days ago.

 

[NugentS]

I suggest:
1. delete the corrupted and replace from backups (if you have any)
2. Take the NAS down, and run memtest for 24 hours (minimum) to thoroughly test the RAM. If you get issues deal with them. If not the restart the NAS, run zpool clear and off you go

 

[sfatula]

Yeah, I'd agree, run memtest, and what are you using as a disk controller? I'm pretty skeptical it's the disks. Ah, didn't look at your machine in your signature, lol. I am not sure I'd be so worried myself.

Ideally, you always run memtest with new ram before using the memory, at least I do.

 

[dias]

Yeah, I'd agree, run memtest, and what are you using as a disk controller? I'm pretty skeptical it's the disks. Ah, didn't look at your machine in your signature, lol. I am not sure I'd be so worried myself.

Ideally, you always run memtest with new ram before using the memory, at least I do.

well i changed that board recently and bought a topton motherboard with onboard n5105 celeron cpu for less power usage, so had to change the rams as that motherboard supports 260 pin so-dimm memory not desktop rams ^^

 

[NugentS]

So what is your hardware - is your signature correct? It says Intel i5-6500 not N5105

 

[dias]

So what is your hardware - is your signature correct? It says Intel i5-6500 not N5105

Changed the signature sorry, i recently changed my system from i5-6500 to celeron for power saving reasons.

 

[NugentS]

That makes more sense now. Still I would memtest for 24 hours just in case

 

[dias]

That makes more sense now. Still I would memtest for 24 hours just in case

i made memtest but it took 6 hours 15 minutes to complete, choosed option 13 (hammer test) and it passed all 4 test with 0 errors like below;

I then deleted the file with permanent error and then restarted but when i check it again with zpool status -v i see some weird 0x0227c0 there with error like below

and it still gives

CRITICAL​

Pool seagate state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.​

error in alerts and pool status is still unhealthy

 

[Etorix]

This is an issue with metadata. Short of diving deep into raw ZFS structures (BIG ANGRY DRAGONS AHEAD!!!), the only way to repair is to destroy the pool and restore from backup.

 

[dias]

This is an issue with metadata. Short of diving deep into raw ZFS structures (BIG ANGRY DRAGONS AHEAD!!!), the only way to repair is to destroy the pool and restore from backup.

Thanks, does it generally occur because of temporary ram problem (non-ecc ram) or hdd?

 

[intercom]

Thanks, does it generally occur because of temporary ram problem (non-ecc ram) or hdd?

Did you find an answer to this? Im having the same issue.

 

[Redcoat]

Im having the same issue.

@intercom Better you start a new thread with all the information requested in the Forum Rules (on the masthead in RED) and describe your problem precisely, including the actual error message you receive.

 

[Etorix]

His thread is already open.

ZFS Health: Pool is not healthy

Disks are just a year old, the message "Pool is not healthy" is not exactly a great description of what is wrong. Anyone have an idea to what I should do? root@truenas[~]# zpool status -v pool: boot-pool state: ONLINE status: Some supported and requested features are not enabled on the...

www.truenas.com

 

[Redcoat]

His thread is already open.

Thx. He could have ref'd it...

EDIT And I had read it - and anjoyed the battery holder work description!

 

[intercom]

Thx. He could have ref'd it...

EDIT And I had read it - and anjoyed the battery holder work description!

Glad you enjoyed it
Is there a Discord/Slack server still? I tried finding it on Discord, but it seems like its not available anymore.

My other thread was originally about something else, so I think its kind of dead.

The output Im getting is this (and endless more lines of the same):

Code:

root@truenas[/mnt/tank/bigdata/Alex]# cp -a Bildesamling /mnt/pool_8tb_disks/media
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/Untitled.mp4’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/P1010874.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems/folders.501/TemporaryItems’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems/folders.501’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/P1000113.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/DSCF3659.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/DSC_0147.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/20160603_211801.jpg’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/Copy (3) of img-518171326-0001.jpg’: Operation not permitted

 

[anodos]

Glad you enjoyed it
Is there a Discord/Slack server still? I tried finding it on Discord, but it seems like its not available anymore.

My other thread was originally about something else, so I think its kind of dead.

The output Im getting is this (and endless more lines of the same):

Code:

root@truenas[/mnt/tank/bigdata/Alex]# cp -a Bildesamling /mnt/pool_8tb_disks/media
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/Untitled.mp4’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/P1010874.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems/folders.501/TemporaryItems’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems/folders.501’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/.TemporaryItems’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/P1000113.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/DSCF3659.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/DSC_0147.JPG’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/20160603_211801.jpg’: Operation not permitted
cp: preserving permissions for ‘/mnt/pool_8tb_disks/media/Bildesamling/000 Asortert/Pappa/Copy (3) of img-518171326-0001.jpg’: Operation not permitted

That's because your target for cp probably has dataset aclmode set to restricted.

 

[sfatula]

You beat me to it, was just posting that! It's a permissions issue one way or another.

 

[intercom]

That's because your target for cp probably has dataset aclmode set to restricted.

I figured it was some permissions issue, but I dont understand anything out the ACL setup. I made a completely new dataset on the target drive that looks like screenshot below. How does that look to you?