Maybe I lost a disk...Maybe I didn't lose a disk. Maybe I lost a port on the backplane?

[notjoe]

So,

I am not sure if a HDD port on my Dell 710's backplane went bad or not. This is the situation. I woke up this morning to a lost storage pool. I don't really care about the data as it is merely a storage box for content that i consume on my HTPC. When I boot the server I am seeing 6 drives in on my sas controller's firmware. Drive 0 though 5, 6 drives.
When FreeNAS boots I am only seeing 5. I've found the drive that isn't reporting in for duty. I'll start with showing you the result of zpool import.

root@freenas:~ # zpool import
pool: storage
id: 9872509597826203642
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-6X
config:

storage UNAVAIL missing device
raidz1-0 ONLINE
gptid/d0fcbda4-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d231b178-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d36724ab-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d491ef41-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/de2cd97a-1fae-11e7-8c34-d4ae52bc5f31 ONLINE

Additional devices are known to be part of this pool, though their
exact configuration cannot be determined.

Now if I swap those two drives I see this on the import.

root@freenas:~ # zpool import
pool: storage
id: 9872509597826203642
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-6X
config:

storage UNAVAIL missing device
raidz1-0 ONLINE
gptid/d0fcbda4-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d231b178-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d36724ab-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/d491ef41-978c-11e5-ab9c-3ca82aa050dc ONLINE
gptid/4e318685-35a6-11e7-af1d-d4ae52bc5f31 ONLINE

Additional devices are known to be part of this pool, though their
exact configuration cannot be determined.

Notice that gptid/de2cd97a-1fae-11e7-8c34-d4ae52bc5f31 got replace with gptid/4e318685-35a6-11e7-af1d-d4ae52bc5f31 but gptid/de2cd97a-1fae-11e7-8c34-d4ae52bc5f31 never showed up on da5 which is where gptid/de2cd97a-1fae-11e7-8c34-d4ae52bc5f31 used to be.

dmesg output
https://pastebin.ca/3835139

So, it looks like da5 is gone bad but like I said I see it everywhere else other than inside the OS.

Here is the dmesg output from me swapping drives.

mps0: mpssas_prepare_remove: Sending reset for target ID 7
da4 at mps0 bus 0 scbus0 target 7 lun 0
mps0: da4: <ATA WDC WD50EZRX-00M 0A80> s/n WD-WX31D3400005 detached
Unfreezing devq for target ID 7
GEOM_MIRROR: Device swap0: provider da4p1 disconnected.
(da4:mps0:0:7:0): Periph destroyed
mps0: SAS Address for SATA device = 48643e5bedd9a18d
mps0: SAS Address from SATA device = 48643e5bedd9a18d
da4 at mps0 bus 0 scbus0 target 7 lun 0
da4: <ATA WDC WD50EZRX-00M 0A80> Fixed Direct Access SPC-3 SCSI device
da4: Serial Number WD-WX21D84PD67R
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 4769307MB (9767541168 512 byte sectors)
da4: quirks=0x8<4K>

 

[notjoe]

So this is getting interesting. iDrac is only showing 5 disks

 

[notjoe]

Further proof that the port seems to be ok on the backplane.

 

[Johnnie Black]

Noob question, how come a raidz1 pool doesn't work with a single missing device?

 

[notjoe]

Noob question, how come a raidz1 pool doesn't work with a single missing device?

It's a legitimate question. I had originally set up a raidz1 with 4 drives. I added two more drives to grow the capacity of the pool but not of the capacity of the raidz1 vdev.

You can tell by the formatting that those two 5tb drives are part of the pool but not part of the raidz1

 

[notjoe]

And the plot thickens. I booted up on a FreeNAS Installer USB stick...and guess what.... I see all my disks and the pool looks fine!
I was on FreeNAS 11-Nightlies train (working fine) when this happened. I'm reinstalling 9.3.1 now which sees all the ports on the backplane. If all goes well and I can import my pool I'll try switching to the FreeNAS 11-Stable train and see what happens.

 

[Johnnie Black]

You can tell by the formatting that those two 5tb drives are part of the pool but not park of the raidz1

Got it, thanks.

 

[notjoe]

A Full reinstall starting with FreeNAS 9.3.1 and then an upgrade to FreeNAS 11-Stable worked like a champ. No hardware issue. Looks like it was software related.
Problem solved.

 

[danb35]

Noob question, how come a raidz1 pool doesn't work with a single missing device?

Because it isn't simply a RAIDZ1 pool; it's a RAIDZ1 pool with one or more disks striped in, though it wasn't obvious since you didn't post your zpool status output in code tags. Try that, and I think it will become clear.

 

[SweetAndLow]

Further proof that the port seems to be ok on the backplane.

In this photo why does one say boot? Are you sure that's a disk from the pool or is it a boot device.

Sent from my Nexus 5X using Tapatalk

 

[Ericloewe]

Oh dear, RAIDZ1 plus two single-drive vdevs. Ouch.

 

[notjoe]

In this photo why does one say boot? Are you sure that's a disk from the pool or is it a boot device.

Sent from my Nexus 5X using Tapatalk

I'm positive. Even though it says boot the OS isn't booting from a SSD and those 6 drives are all part of the pool. Without changing anything in any of the various Dell firmwares I was able to get things fixed by reinstalling FreeNAS.

 

[notjoe]

Oh dear, RAIDZ1 plus two single-drive vdevs. Ouch.

Agreed. However, I wasn't able to add those drives to the existing RAIDZ1. Seems counter productive that one cannot "grow" their RAIDZ* once it has been created.

 

[danb35]

Seems counter productive that one cannot "grow" their RAIDZ* once it has been created.

It is what it is, and it's always been that way with ZFS. It most likely always will be that way. But somehow, despite all the warnings in the manual and in the UI, people persist in doing exactly what you did.

 

[notjoe]

It is what it is, and it's always been that way with ZFS. It most likely always will be that way. But somehow, despite all the warnings in the manual and in the UI, people persist in doing exactly what you did.

I think you're confusing me with someone who cares about their data. I don't. The FreeNAS I Build for my office consists of mirrored vdevs that form the pool. None the less this wasn't about my pure need for capacity rather than data integrity. I'm aware of best practices and the implications that having two disks added to the pool that are not redundant implies. If I cared about the data I would have mirrored those disks then added them to the pool. As I said in the OP "I don't really care about the data as it is merely a storage box for content that i consume on my HTPC"

I wasn't looking for help to restore my pool as much as I was thinking that there might have been an issue in FreeNAS 11-Nightly. I found it extremely odd that one of my disks would magically disappear over night never to be seen from again until I had to reinstall FreeNAS. Then again...Maybe you are saying that it IS possible to add more disks to an existing RAIDZ1 setup?

 

[danb35]

I think you're confusing me with someone who cares about their data.

True, we do tend to assume that here. If you're aware of what you did and its implications, and that doesn't bother you, carry on. I'd confused @Johnnie Black's question (to which I responded) with being yours.