SOLVED Failed perc damaged multiple disks, how to fix?

[Daisuke]

I had a H710 perc that failed in my Dell R720xd. I replaced it with a new one and when I started Scale, the polls were ruined with several platter disks showing as faulted or degraded. Since this is a new setup, there was no data into pools, so I decided to destroy all pools and start over. Re-creating the pool was difficult, I had to wipe each of 12 disks. Once all disks wiped, the pool was created but the disks show the same status, faulty or degraded.

What should I do next, to fix this issue? Is there a command I can run in a Linux livecd to re-format the disks properly? Everything is a mess. I also have 2 SSDs installed that it takes forever to create an applications pool in mirror.

Code:

# zpool status default
  pool: default
 state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
    Sufficient replicas exist for the pool to continue functioning in a
    degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
    repaired.
  scan: scrub repaired 180K in 00:01:49 with 0 errors on Tue Jul  5 20:53:30 2022
config:

    NAME                                      STATE     READ WRITE CKSUM
    default                                   DEGRADED     0     0     0
      raidz2-0                                DEGRADED     6   615     0
        a3d8d2ab-ce18-49ba-a3ba-d507ea2815e0  FAULTED      0    16     0  too many errors
        cf639408-92c3-4a06-8613-d93f9cc8e4a2  DEGRADED    21   533     0  too many errors
        cbc5a2eb-4040-40dc-91c2-a74ee19f41c5  FAULTED      0    16     0  too many errors
        9f44f3b4-a512-464d-96a5-70610e763c09  DEGRADED     0   399     1  too many errors
        5b3f653f-2305-4ead-ba3d-637fc733329a  DEGRADED    10   911     4  too many errors
        4aa941bb-b220-4247-9868-213e1d67f8f5  DEGRADED     3   540     4  too many errors
        9a6db0a4-194c-43ee-aa39-551b2f5c8c29  DEGRADED    27   728     6  too many errors
        ff9e7cd4-092e-4618-9640-fe0bc8d5df01  DEGRADED     0   356     2  too many errors
        95d1f7bf-ab78-445e-a90b-36daf692a568  DEGRADED     4   504     6  too many errors
        54b647a2-f0c0-4884-87e8-dcb770b67cb9  DEGRADED     4   578     6  too many errors
        9e04f193-8a1e-42dd-b39b-d308075ba6eb  DEGRADED    15   822     4  too many errors
        0be664b2-2ad4-482b-a402-979e63e8b847  DEGRADED     0   520     1  too many errors

errors: No known data errors

I get tons of errors like:

 

[Samuel Tai]

Are you sure the replacement PERC is in IT mode? It's possible your replacement PERC is also bad.

 

[Daisuke]

Yes, I flashed the perc to IT mode successfully. If the perc would be defective, I would not see the disks, prior making the pool. When the first perc failed, a big red X was showing next to each pool. Before destroying the pool with new perc installed, it was showing several disks online:

Code:

# zpool status -v default
  pool: default
 state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
    Sufficient replicas exist for the pool to continue functioning in a
    degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
    repaired.
  scan: scrub in progress since Tue Jul  5 16:02:18 2022
    166G scanned at 201M/s, 3.49M issued at 4.24K/s, 17.2T total
    8.93M repaired, 0.00% done, no estimated completion time
config:

    NAME                                      STATE     READ WRITE CKSUM
    default                                   DEGRADED     0     0     0
      raidz2-0                                DEGRADED   273   140     0
        629cefc5-7187-41e3-bd66-d06e7c3707f7  DEGRADED   211   126     0  too many errors  (repairing)
        2c08fe13-bf6b-4af4-8557-d276ec9fb39a  DEGRADED   459   118     0  too many errors  (repairing)
        2a15af88-b984-442b-8c15-4400587c5edb  ONLINE     161    93     0  (repairing)
        9aa9d5f9-40a9-4868-a0ff-c5de94cf6f5d  ONLINE     524   153     0  (repairing)
        419c3bec-fd1a-46b1-96c4-688ca14d9393  DEGRADED   134   132     0  too many errors  (repairing)
        9f176f88-76a2-4617-a008-4ec668c92ee8  ONLINE     195   117     0  (repairing)
        3393954b-c917-49d2-8e1a-ce523005939f  ONLINE     367   122     0  (repairing)
        7beb091a-2b52-4a47-ac40-2f554026d6d7  ONLINE     325   111     0  (repairing)
        44b86a8d-b12b-4e14-8a35-63d62c28cb99  FAULTED      6    10     0  too many errors
        a1320232-66ce-43de-a96f-252284e0933a  FAULTED     10    73     0  too many errors
        993f9da2-b761-4492-8232-4c6584da9d38  DEGRADED   169   129     0  too many errors  (repairing)
        99efdf13-ce7e-4b6e-bf9b-d87704ffaa79  ONLINE     175   119     0  (repairing)

errors: No known data errors

It has be a way to destroy the pool and wipe the disks properly, I don't think the UI is the right choice in my situation.

 

[Samuel Tai]

Try reseating the replacement PERC, and cleaning the contacts with alcohol.

 

[Daisuke]

The alcohol did something, no more issues! The perc was seated properly, I'm familiar with how delicate they are.

 

[anodos]

The alcohol did something, no more issues! The perc was seated properly, I'm familiar with how delicate they are.

Next time try blowing hard on it from an angle... oh wait, PERC isn't a Nintendo game cartridge. :)

 

[Samuel Tai]

Next time try blowing hard on it from an angle... oh wait, PERC isn't a Nintendo game cartridge. :)

@anodos, I take it you've seen Nick Robinson's epic Blastoise resurrection video?

Can we rescue a deleted Pokémon - 15 years after its death?

NEW 2-HOUR DOCUMENTARY: https://youtu.be/oy3wEeeW8mk --~--Get an exclusive NordVPN deal here ➡ https://nordvpn.com/nickrobinson risk-free with Nord's 30-day ...

youtu.be