UNAVAIL cannot open

[phyx]

Evening Everyone,

Preamble: I'm new at this, bare with me.

Issue: I'm running freenas on...

Chenbro NR12000 1U 1x 8m QUAD CORE E3-1220 3.1GHz 8GB RAM NO HDD | eBay

Server Type Chenbro NR12000 1U SERVER. Chenbro NR12000 1U 1x 8m QUAD CORE E3-1240V2 3.4GHz 32GB RISER RAIL NO HDD. Chenbro NR12000 1U 1x 8m QUAD CORE E3-1240 3.3GHz 32GB RAM 6x 500GB SATA. Chenbro NR12000 1U 1x 8m QUAD CORE E3-1230v2 3.3GHz 32GB RAM 8x 1TB SATA.

www.ebay.com

It's sporting 10 6TB HDDs, 4 on SATA and 6 on the onboard RAID Controller, the TrueNAS OS is on a 250g SSD

I had a drive fail on me while I was away for about a month. When I returned to resolve the issue, the ZPOOL was completely unavailable. When I attempt a zpool mount -m, I get insufficient replicas and ALL the drives on the raid controller are unavailable. I scoured the almighty google and this forum and cannot find anything to resolve the problem or what the problem even really is. Admittedly, the failure is mine. During that month, it was erroring due to the bad drive on the SATA port and my family continued to use it until it completely died out.

There's a LOT of data on there and recovery is obviously the ideal. Can anyone advise? Is there more information I can include?

 

[phyx]

Also, I upgraded the RAM to 32g, I doubt that's relevant, but I forgot to include it.

 

[jgreco]

You can tell us more about the RAID, which according to a little research might be LSI, and if it happens to be running under LSI mfi driver, is probably a bad sign.

The following link should be considered nonoptional reading but probably doesn't help too much:

https://www.truenas.com/community/r...bas-and-why-cant-i-use-a-raid-controller.139/

 

[NugentS]

As @jgreco says - we need a lot more info before serious suggestions can be made.

 

[phyx]

It is indeed LSI SAS2008
LSI 9211-8i (PCIe 2.0 based on LSI 6Gbps SAS2008) HBA

The drives aren't set up for RAID at BIOS level, I was using FREENAS for that.

zpool import -m
pool: Storage
id: 16525799538964911811
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-3C
config:

Storage UNAVAIL insufficient replicas
raidz3-0 UNAVAIL insufficient replicas
gptid/9e330b74-6352-11eb-967d-00e081ebe5a8 ONLINE
gptid/a03a4c95-6352-11eb-967d-00e081ebe5a8 ONLINE
gptid/a261a248-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/a4f18def-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/a6f7fa24-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/a93e15d3-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/a9e560e4-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/abb6ff34-6352-11eb-967d-00e081ebe5a8 ONLINE
gptid/aadbbeb1-6352-11eb-967d-00e081ebe5a8 UNAVAIL cannot open
gptid/ac5c3184-6352-11eb-967d-00e081ebe5a8 ONLINE

The six unavailable above are plugged into the SAAS controller.
The four Online are plugged into SATA.

What other information would help. I'm somewhat desperate.
Thanks in advance.

 

[NugentS]

Bad Cable(s)?
Try reseating all the SAS cables. Making sure they aren't tight or anything like that
If you boot the system is the SAS seen?
Whats the firmware on the SAS. Is it running in IT mode or RAID with JBOD?

 

[phyx]

Cables appear to be fine. It would seem somewhat unlikely that all the cables failed at the same time, but I did reseat them.
The drives appear successfully on boot and they're running in IT mode.

 

[phyx]

Another note. They are SATA drives on a SAS interface, if that makes a difference.

 

[praetorian46]

@phyx Did you solve this problem? I'm seeing the same thing. I just introduced a Intel SAS Expander Card (RES2SV240). And that is connected to one port of my LSI 9211-8i. Every disk connected to the Intel SAS Expander seems to be in the sate of "UNAVAIL cannot open" This only happened some time after I setup the ZPool and started replicating data there. Maybe a bad cable between the Expander and the HBA? Any clues would be great. I'm running TrueNAS Core 12.0-U8