SOLVED 5 x HDD's reporting as "unknown" out of 8.

[british]

Hey guys, I hope everyone is good and well.

TL;DR: 5 of 8 drives are reporting as being "unknown" , and not "HDD" in TrueNAS, they contain a RAIDz2 pool. Were shipped via DHL to a friend in another country. Their rotation rate is also unknown. Yet they pass SMART tests, can do a dd if=/dev/[problem_device] of=/dev/null and can be left for ages, reads and writes match when CTRL-C'd. Can see the serial number, capacity, model number etc all perfectly. Can also do an zdb -l /dev/[problem_device]p2 and see the ZFS pool info.

and a sample UNKNOWN disk:

Long story:

I had 8 x WD RED 3TB drives in a RAIDz2 pool, which i ended swapping for 8 x 10TB's by replacing each drive in the TN web UI. Once I'd replaced all drives I decided to offer them to a friend. I (in my mind) package the drives well in bubble wrap and antistatic bags and shipped them in a sturdy box with DHL.

My mate purchased a Dell PowerEdge T320 with PERC card, which reported to Ubuntu Live USB as having the following controller.

Code:

08:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2208 [Thunderbolt] (rev 05)
        Subsystem: Dell PERC H710 Adapter
        Kernel driver in use: megaraid_sas
        Kernel modules: megaraid_sas

The 2208 was flashed to IT by my friend using the following procedure.

Introduction - Fohdeesha Docs

fohdeesha.com

The result was

Code:

    Adapter Selected is a LSI SAS: SAS2308_2(D1)
    Controller Number              : 0
    Controller                     : SAS2308_2(D1)
    PCI Address                    : 00:08:00:00
    SAS Address                    : 5141877-0-3078-9800
    NVDATA Version (Default)       : 14.01.00.06
    NVDATA Version (Persistent)    : 14.01.00.06
    Firmware Product ID            : 0x2214 (IT)
    Firmware Version               : 20.00.07.00
    NVDATA Vendor                  : LSI
    NVDATA Product ID              : SAS9207-8i
    BIOS Version                   : N/A
    UEFI BSD Version               : N/A
    FCODE Version                  : N/A
    Board Name                     : SAS9207-8i
    Board Assembly                 : N/A
    Board Tracer Number            : N/A

He's also shuffled the drives around, and still the same drives report the same problem. He has tried one of the UNKOWN drives directly attached to the motherboard, and that still shows as UKNOWN.

Sort of a bit baffled about whats up here. We're toying with the idea of trying another box (and put TN on it) and see what the drives report as.

Has anyone encountered anything like this before as we're totally baffled. I'd assume that if the drives had been damaged in transit that they wouldn't be able to be read via dd for example, nor see all the info on the drive via geom disk list.

Hope you guys can help, TIA and best wishes

 

[sretalla]

According to the latest header (/usr/local/share/smartmontools/drivedb.h) Those drives should probably be matched to this type:

{ "Western Digital Red", // tested with WDC WD10EFRX-68JCSN0/01.01A01,
// WDC WD10JFCX-68N6GN0/01.01A01, WDC WD30EFRX-68EUZN0/82.00A82,
// WDC WD40EFRX-68WT0N0/80.00A80, WDC WD60EFRX-68MYMN1/82.00A82,
// WDC WD80EFAX-68LHPN0/83.H0A83, WDC WD80EFZX-68UW8N0/83.H0A83,
// WDC WD80EZZX-11CSGA0/83.H0A03 (My Book 0x1058:0x25ee),
// WDC WD100EFAX-68LHPN0/83.H0A83,
// WDC WD120EMFZ-11A6JA0/81.00A81 (Easystore 0x1058:0x25fb)
"WDC WD(7500BFCX|10JFCX|[1-6]0EFRX|[68]0E[FZ]ZX|(8|10)0EFAX|120EMFZ)-.*",
"", "",
"-v 22,raw48,Helium_Level" // WD80EFAX, WD80EFZX, WD100EFAX, WD120EMFZ
},
{ "Western Digital Red (SMR)", // ticket #1313, tested with WDC WD60EFAX-68SHWN0/82.00A82
"WDC WD[2346]0EFAX-.*",
"", "", ""

The drives are WDC WD30EFRX-68A

This is the line which should match it:
"WDC WD(7500BFCX|10JFCX|[1-6]0EFRX|[68]0E[FZ]ZX|(8|10)0EFAX|120EMFZ)-.*",

It seems to me that it would match according to the green text above. no idea why it doesn't.

 

[british]

According to the latest header (/usr/local/share/smartmontools/drivedb.h) Those drives should probably be matched to this type:

{ "Western Digital Red", // tested with WDC WD10EFRX-68JCSN0/01.01A01,
// WDC WD10JFCX-68N6GN0/01.01A01, WDC WD30EFRX-68EUZN0/82.00A82,
// WDC WD40EFRX-68WT0N0/80.00A80, WDC WD60EFRX-68MYMN1/82.00A82,
// WDC WD80EFAX-68LHPN0/83.H0A83, WDC WD80EFZX-68UW8N0/83.H0A83,
// WDC WD80EZZX-11CSGA0/83.H0A03 (My Book 0x1058:0x25ee),
// WDC WD100EFAX-68LHPN0/83.H0A83,
// WDC WD120EMFZ-11A6JA0/81.00A81 (Easystore 0x1058:0x25fb)
"WDC WD(7500BFCX|10JFCX|[1-6]0EFRX|[68]0E[FZ]ZX|(8|10)0EFAX|120EMFZ)-.*",
"", "",
"-v 22,raw48,Helium_Level" // WD80EFAX, WD80EFZX, WD100EFAX, WD120EMFZ
},
{ "Western Digital Red (SMR)", // ticket #1313, tested with WDC WD60EFAX-68SHWN0/82.00A82
"WDC WD[2346]0EFAX-.*",
"", "", ""

The drives are WDC WD30EFRX-68A

This is the line which should match it:
"WDC WD(7500BFCX|10JFCX|[1-6]0EFRX|[68]0E[FZ]ZX|(8|10)0EFAX|120EMFZ)-.*",

It seems to me that it would match according to the green text above. no idea why it doesn't.

Hi, thanks for the fast reply. I will have to get the full model numbers off my friend. What's odd is that I had the drives in my FN/TN for years, all without issue. Then when I send them to my friend and he puts them in, bang... this happens. Very odd

 

[sretalla]

It could be that the Megaraid adapter (even though flashed into IT mode) is somehow still acting as RAID and passing the drives as individual JBOD containers, hence unknown to ZFS/TrueNAS.

Really a stretch, but I don't have much more to go on.

 

[british]

It could be that the Megaraid adapter (even though flashed into IT mode) is somehow still acting as RAID and passing the drives as individual JBOD containers, hence unknown to ZFS/TrueNAS.

Really a stretch, but I don't have much more to go on.

Hi, well he tried placing one of the "unknown" drives directly to the motherboard, and it still showed exactly the same. So I'm going to make the assumption that its localised to the drive(s). He's planning on testing the remaining "unknown" drives in a similar manner. Just to point out he tried to import the pool, but its not seen. We did an zdb -l on each of the drives, and all showed info. All can be read with dd, and no error / log to dmesg etc.... sort of stumped.

 

[british]

Hi, OK he's come back and heres the results!

Code:

da0 = WD-WCC1T0585706 UNKNOWN  ATA WDC WD30EFRX-68A
da1 = WD-WCC1T0604775 UNKNOWN  ATA WDC WD30EFRX-68A
da2 = WD-WCC1T0607948 UNKNOWN  ATA WDC WD30EFRX-68A
da3 = WD-WCC1T0577656 UNKNOWN  ATA WDC WD30EFRX-68A
da4 = WD-WCC4N1ZY6C4X HDD      ATA WDC WD30EFRX-68E
da5 = WD-WCC1T0565370 UNKNOWN  ATA WDC WD30EFRX-68A
da6 = WD-WCC4N6YD0DP6 HDD      ATA WDC WD30EFRX-68E
da7 = WD-WCC4N5JVXN0N HDD      ATA WDC WD30EFRX-68E

So all the 68A's are reporting as UKNOWN. and all 68E's as HDD.

He's running the latest version of TrueNAS, TrueNAS-12.0-U4.1

Hope you guys can help. Best wishes.

 

[british]

Hi we've tried installing FreeBSD 13.0 , and it also shows the same.

Code:

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68A
   lunid: 50014ee208405a45
   ident: WD-WCC1T0585706
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da1
Providers:
1. Name: da1
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68A
   lunid: 50014ee208402d16
   ident: WD-WCC1T0604775
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da2
Providers:
1. Name: da2
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68A
   lunid: 50014ee2b2e1c426
   ident: WD-WCC1T0607948
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da3
Providers:
1. Name: da3
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68A
   lunid: 50014ee2b2e8c107
   ident: WD-WCC1T0577656
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da4
Providers:
1. Name: da4
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68E
   lunid: 50014ee20eb9c83d
   ident: WD-WCC4N1ZY6C4X
   rotationrate: 5400
   fwsectors: 63
   fwheads: 255

Geom name: da5
Providers:
1. Name: da5
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68A
   lunid: 50014ee2083e4fd4
   ident: WD-WCC1T0565370
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

Geom name: da6
Providers:
1. Name: da6
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68E
   lunid: 50014ee2b9739a07
   ident: WD-WCC4N6YD0DP6
   rotationrate: 5400
   fwsectors: 63
   fwheads: 255

Geom name: da7
Providers:
1. Name: da7
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA WDC WD30EFRX-68E
   lunid: 50014ee20e331862
   ident: WD-WCC4N5JVXN0N
   rotationrate: 5400
   fwsectors: 63
   fwheads: 255

Complex :-/

FWIW, he also tried connecting drives directly to the motherboard SATA ports, and got same result so I'm going to rule out bad flashing of HBA.

Best wishes.

 

[british]

More info. So in Ubuntu we did a hdparm and extracted info , again no speed, but here's also the firmware version

Code:

sdb Firmware Revision: 80.00A80 Model Number: WDC WD30EFRX-68AX9N0 Serial Number: WD-WCC1T0577656
sdc Firmware Revision: 82.00A82 Model Number: WDC WD30EFRX-68EUZN0 Serial Number: WD-WCC4N6YD0DP6 Nominal Media Rotation Rate: 5400
sdd Firmware Revision: 82.00A82 Model Number: WDC WD30EFRX-68EUZN0 Serial Number: WD-WCC4N1ZY6C4X Nominal Media Rotation Rate: 5400
sde Firmware Revision: 82.00A82 Model Number: WDC WD30EFRX-68EUZN0 Serial Number: WD-WCC4N5JVXN0N Nominal Media Rotation Rate: 5400
sdf Firmware Revision: 80.00A80 Model Number: WDC WD30EFRX-68AX9N0 Serial Number: WD-WCC1T0585706
sdg Firmware Revision: 80.00A80 Model Number: WDC WD30EFRX-68AX9N0 Serial Number: WD-WCC1T0604775
sdh Firmware Revision: 80.00A80 Model Number: WDC WD30EFRX-68AX9N0 Serial Number: WD-WCC1T0607948
sdi Firmware Revision: 80.00A80 Model Number: WDC WD30EFRX-68AX9N0 Serial Number: WD-WCC1T0565370

 

[british]

FYI thanks to Allan Jude for help solve this mystery. FWIW it's down to the fact I actually replaced each of the drives in the pool one by one. Essentially the pool is gone on the 8 x 3TB's, and only alive on my box with the 8 x 10TB's. Oh well, live and learn. Thank you to all on IRC for their assistance, and to anyone who's read this.
All the 8 x 3TB's are functional and work. And a new RAIDz2 pool has been created using them and all 100%.

Best wishes.