zpool degraded, disk unavailable, but SMART ok

[daniels7]

Thanks for your answer, I wish you a happy new year :)

That's an interesting theory, though my IPMI Event Log is clean and all sensors are green. The cables are exactly the same as before.
It's of course possible that it's the PSU, since the new Seagate Ironwolf 10TB drives consume about double as much power as the old WD Red 4TB drives.

I'll check that :)

If anyone else has any recommendations what could cause the problems feel free to answer as well.

 

[joeschmuck]

Could anyone please respond?
The error is happening again and again each time on different drives and I'm seriously worried about it

First of all I'd like to start all over in troubleshooting this problem.

since the new Seagate Ironwolf 10TB drives consume about double as much power as the old WD RED 4TB drives.

You apparently never looked at the spec sheets becasue that is not true. The power consumption difference is minor.
WD Red Spec Sheet
Seagate Ironwolf Spec Sheet

Have you used the Hard Drive Troubleshooting Guide? (link in my signature)

Your problem appears to be a communications issue. Replace, swap out, or reseat the SATA cables for the two drives da3 and da6. Ensure you use the serial numbers are for da3 and da6, they are not always in order.

da3: Serial Number ZA20HV0L
da6: Serial Number ZA20HYFX

Also are you using VMWare for this machine? And while you did post the degub file, it would be nice to see the system specs in the posting.

 

[daniels7]

Thank you.

Regarding the power consumption I just trusted the numbers on Amazon. But it's still about 50% more Power Consumption on Load in the Spec Sheets you posted.

I didn't see the Hard Drive Troubleshooting Guide before, but I already tested most of the parts listed there and can rule out some others. I don't think that the Sata-Cables are faulty because this morning I had errors on da1 and da5 and on other days da2 and da4 had errors as well, it's totally random and very unlikely that 6 Sata-Cables are wrong seated, no?

Yes, I use VMWare, this machine is an ESXi-Host and FreeNAS is inside a VM.

The Specs are:
ASRock Rack EPC612D4I - Motherboard
4 x Kingston ValueRAM - DDR4 - 16 GB - SO DIMM 260-PIN - 2133 MHz / PC4-17000 - CL15 ECC RAM
1 x Intel Xeon E5-2620V4 - 2.1 GHz - 8-Core - 16 Threads - 20 MB Cache
6 x Seagate IronWolf 10TB
BE QUIET PURE POWER L8 300W
IBM M1015 flashed to newest LSI IT Firmware

On this Host I assigned 16GB of RAM to FreeNAS and it has exclusive access to the LSI Controller through VT-d/DirectPath so it has direct access to harddrives and Smart Data.

The Smart-Data shows 0 as value for all critical fields.

 

[SweetAndLow]

Thank you.

Regarding the power consumption I just trusted the numbers on Amazon. But it's still about 50% more Power Consumption on Load in the Spec Sheets you posted.

I didn't see the Hard Drive Troubleshooting Guide before, but I already tested most of the parts listed there and can rule out some others. I don't think that the Sata-Cables are faulty because this morning I had errors on da1 and da5 and on other days da2 and da4 had errors as well, it's totally random and very unlikely that 6 Sata-Cables are wrong seated, no?

Yes, I use VMWare, this machine is an ESXi-Host and FreeNAS is inside a VM.

The Specs are:
ASRock Rack EPC612D4I - Motherboard
4 x Kingston ValueRAM - DDR4 - 16 GB - SO DIMM 260-PIN - 2133 MHz / PC4-17000 - CL15 ECC RAM
1 x Intel Xeon E5-2620V4 - 2.1 GHz - 8-Core - 16 Threads - 20 MB Cache
6 x Seagate IronWolf 10TB
BE QUIET PURE POWER L8 300W
IBM M1015 flashed to newest LSI IT Firmware

On this Host I assigned 16GB of RAM to FreeNAS and it has exclusive access to the LSI Controller through VT-d/DirectPath so it has direct access to harddrives and Smart Data.

The Smart-Data shows 0 as value for all critical fields.

Only 300w power supply with 6 drives, e5 and hba. This system probably needs more power.

Sent from my Nexus 5X using Tapatalk

 

[daniels7]

Are you sure?
According to my UPS the system is never consuming more than 150W

 

[joeschmuck]

Are you sure?
According to my UPS the system is never consuming more than 150W

The 115VAC power consumption is not the same as the power consumption on the +12VDC, +5 VDC, and +3.3VDC outputs (as well as others) which is how a computer power supply is actually rated.

I also agree that you are likely pushing your power supply to it's limits. The only way to know for certain is to measure each load or find out what the maximum power draw is for each load. But even then, as a power supply is pushed harder typically there will be power fluctuations (un-clean voltage). Now I'm not saying this is your issue here but as some friendly advice, I would recommend you re-evaluate it.

don't think that the Sata-Cables are faulty because this morning I had errors on da1 and da5 and on other days da2 and da4 had errors as well, it's totally random and very unlikely that 6 Sata-Cables are wrong seated, no?

Yes, it is possible for it to be the cable. Let me ask you this... Are the drives all connected using one SFF-8087 cable? If yes then relocate the cable to the other port and see if that works. If you have the drives split across two of these cables then are the affected drives on a single cable? The common denominators here are: 1) The SFF-8087 Cable, 2) The M1015 Controller, 3) The Drives. Odds are, and we have seen this before more than once, the connections were disturbed and need to be remated or the cable actually went bad. Also, what is the length of the SFF-8087 cable? If it's really long, you migth try a shorter cable.

Regarding the power consumption I just trusted the numbers on Amazon. But it's still about 50% more Power Consumption on Load in the Spec Sheets you posted.

[xtable]
{tbody}
{tr}
{td} {/td}
{td}WD Red {/td}
{td} Seagate {/td}
{/tr}
{tr}
{td}Start Current {/td}
{td} 1.75A{/td}
{td} 1.8A{/td}
{/tr}
{tr}
{td}Ave. Current {/td}
{td} 4.5W{/td}
{td} 6.8W{/td}
{/tr}
{tr}
{td}Idle Current{/td}
{td} 3.3W{/td}
{td} 4.4W{/td}
{/tr}
{/tbody}
[/xtable]

I can see that you were looking at the average current draw which while it is pulling ~50% more power, it's not a major consern. I was looking at the startup current which is ~ 21 Watts (Amps x Volts) per drive and this the factor you were really pointing to when you originally stated that your PSU could be the problem becasue the drives were pulling 50% more power. They pull a combined total of ~1 Amp more power while they are operating and that is insignificant in my opinion. So while I don't think your PSU is at fault here, if you find out it is, the fact that it failed due to replacing the hard drives would mean that you were really pushing your PSU for a while and I don't believe that at this time.

Okay, getting back to the problem at hand, check your cable(s) and you might need to replace it.