All disks to degraded on RaidZ3

[zhe]

sys is rueNAS-SCALE-22.02.4
disk is 16TB 22 disks build RaidZ3

Gui display all disks on degraded

ssh cmd show
----------------------
root@truenas[~]# zpool status
pool: HomePool
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Nov 28 23:54:38 2022
1.70T scanned at 85.7M/s, 1.58T issued at 79.6M/s, 4.19T total
71.0G resilvered, 37.70% done, 09:32:36 to go
config:

NAME STATE READ WRITE CKSUM
HomePool DEGRADED 0 0 0
raidz3-0 DEGRADED 0 0 0
a2a4471f-c7ef-4e31-a39c-71345f922ce8 DEGRADED 0 0 5.89K too many errors
e2c94101-3f15-435f-95b4-53f26b1f5e72 DEGRADED 0 0 5.89K too many errors
e96c2b49-98cc-4b72-b059-515f268ff346 DEGRADED 0 0 5.95K too many errors
e8382bbd-d369-49d7-b1e9-709fea3e6de6 DEGRADED 73.9K 2 5.97K too many errors (resilvering)
e9ddfd9a-997c-4806-859d-868aa8a8c2cb DEGRADED 0 0 5.95K too many errors
1ec87af4-7b0b-4e4a-a461-b48c537f8e44 DEGRADED 9.44K 29 5.95K too many errors (resilvering)
9e0c8a31-01eb-4766-95f3-f26e682380e3 DEGRADED 0 0 5.95K too many errors
b8035223-6b9c-4954-801a-07d461ee037b DEGRADED 332 0 5.95K too many errors (resilvering)
dba5b5d2-1b59-44bf-8144-d2b5cf9e4ac3 DEGRADED 0 0 5.95K too many errors
53d01a6b-cced-48db-8a1c-918c84652846 DEGRADED 39.9K 255 5.95K too many errors (resilvering)
0e2d892d-bfb7-47a6-8210-47054905a378 DEGRADED 37.0K 4 5.95K too many errors (resilvering)
2d44d0ab-02c5-4c72-8eac-bf607ca7c672 DEGRADED 0 0 5.95K too many errors
a2e69f62-a7a0-4a92-8c92-c509fca4b9fc DEGRADED 0 0 5.95K too many errors
615105ba-558f-4828-923d-eb8ba790fb22 DEGRADED 0 0 5.95K too many errors
replacing-14 DEGRADED 0 0 5.95K
16752858698199900939 UNAVAIL 0 0 0 was /dev/disk/by-partuuid/a64222d9-5a9f-4e5a-b7a4-3f295a866654
da148efe-31e2-4e0a-927d-1925378a89ed DEGRADED 0 0 0 too many errors (resilvering)
2d251bba-1faf-43c5-98c9-04c13df6f4d1 DEGRADED 0 0 5.94K too many errors
61d6cf52-c171-49ff-ae69-c45bf13c7059 DEGRADED 0 0 5.88K too many errors
eee02552-500a-4143-8776-36158ab2385f DEGRADED 0 0 5.88K too many errors
a726ac5f-4c98-4cea-9a4c-625e337d3c08 DEGRADED 0 0 5.88K too many errors
cf9010f9-05a7-4d5f-9491-f1363d22e590 DEGRADED 0 0 5.88K too many errors
12172b1d-8078-491f-8377-ca4c057c5811 DEGRADED 0 0 5.88K too many errors
ff8f4b49-4e61-4a73-b814-f96e94b925d2 DEGRADED 0 0 5.88K too many errors
cache
b0f87678-043d-4b8d-b33c-3c71c3f58989 ONLINE 0 0 0

errors: 255956 data errors, use '-v' for a list

pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:00:06 with 0 errors on Mon Nov 28 03:45:08 2022
config:

NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sda3 ONLINE 0 0 0

errors: No known data errors

--------------------------------------
#smartctl -H <all disks> all pass

i can read and write by SMB, data not lose
-----------
how can i do to fix it to online?

 

[zhe]

i want to try
1, wait for time of resilvered finish
2, cmd run #zpool clear sb{b,c,d,e....w}

who have a better good idear? thanks a lot

 

[jgreco]

Welcome to the forums.

Sorry to hear you're having trouble. Please take a few moments to review the Forum Rules, conveniently linked at the top of every page in red, and pay particular attention to the section on how to formulate a useful problem report, especially including a detailed description of your hardware, HBA, and specific hard drives being used.

You've basically given no one anything to work with, so the responses will tend to be random guesses rather than anything useful.

 

[zhe]

yes, i show more describe

hadware
i5 7400t , ram is 16GB,
one 216G ssd for sys-boot
one 512G ssd memory catch for homepool my named datapool , 22 disks per 16TB

sofware
tureNAS-SCALE-22.02.4
make RaidZ3
pool name is HomePool
and Dataset connect by SMB

operate
one disk is adm degraded , i offline the bad one but error , so i shutdown then out badone insert new one power on
and replace it by GUI function button , all night time running
second day morning all disks degrauded FL the picture

check disk by SMART
ssh cmd
#smartctl -H <> one by one disk ,all disks is check passed is ok

now i am running
#zpool scrub [-s|-p] [-w] pool…
for a long time
have one time unknow be shutdown by nothing....
reboot then go on resilvering for a long time

to be continued ......

 

[jgreco]

Conspicuously missing are two of three important things I asked for, which included the HBA used to attach the disks, and the specific type of the disks. You are not going to get meaningful guidance until you provide details. Additionally, you say you have 22 16TB disks, if I understand what you wrote correctly, but only 16GB of RAM? That's crazy. Normal guidance is to have 1GB of RAM per TB of disk. You can skimp on this a bit once you get out past, maybe 64GB of RAM, but I'm thinking you should have a minimum of 64-128GB RAM. Failing to do so will make your scrubs, resilvers, and pool imports suck.

 

[joeschmuck]

Additionally, you say you have 22 16TB disks, if I understand what you wrote correctly, but only 16GB of RAM?

I agree, that is crazy. 352TB of RAW disk space and 16GB RAM. Does not compute.

one 512G ssd memory catch for homepool

This does not make up for the lack of RAM, it actually makes things worse.

 

[zhe]

yes update my operate.
stop here and go here link

All disk change to de DEGRADED , all new disk setup

i replace one disk running all night. today morning but ..... what is happen? my all data not lose yet , can read and write now how can i do it to online? #zpool clear ? after scanned finished? root@truenas[~]# zpool status pool: HomePool state: DEGRADED status: One or more devices is...

www.truenas.com

 

[joeschmuck]

Closing this thread since the OP started another one for this problem.