unable to use smartctl

[michal_f]

Hello
After logging to my trueNAS TrueNAS-12.0-U1 I noticed an unhealthy pool. The status page reported some disks with checksum errors.
The first thing I tried is to run smartctl on those disks. The command failed so:

Code:

# smartctl -a /dev/da11
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               DELL
Product:              PERC H710P
Revision:             3.13
Compliance:           SPC-3
User Capacity:        4,000,225,165,312 bytes [4.00 TB]
Logical block size:   512 bytes
>> Terminate command early due to bad response to IEC mode page
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

Quick google search suggests this is due to disks going offline due to loose cable or something, but all of my 12 disks show this error !
Unfortunatelly I am not a sys-admin, just an 3d artists running a server :) - so please give some hints how to debug this.

My hardware is a dell poweredge r720xd with controller perc h710p mini

Btw. - why my disks are listed as /dev/da* instead of typical /dev/ada* ?

 

[WN1X]

Drives brand and model?

 

[HoneyBadger]

dell poweredge r720xd with controller perc h710p mini

Uh oh. That's a RAID controller. While it can be flashed into an HBA (following very specific instructions) in its stock format it's bad news.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

Have you set up each disk as an individual RAID0 virtual disk?

Btw. - why my disks are listed as /dev/da* instead of typical /dev/ada* ?

daX is the naming format for SCSI devices (including those behind a SAS controller) where adaX is used for SATA.

 

[michal_f]

Drives brand and model?

dell mg03sca400

 

[michal_f]

Uh oh. That's a RAID controller. While it can be flashed into an HBA (following very specific instructions) in its stock format it's bad news.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

Have you set up each disk as an individual RAID0 virtual disk?



daX is the naming format for SCSI devices (including those behind a SAS controller) where adaX is used for SATA.

I am aware that ZFS does not want RAID controllers but a BOD (bunch of disks?).
Unfortunatelly I was not the one who setup the server.
How can I check now if the controller is flashed with correct firmware and if disks are raid0 virtual disks ?
Sorry for all that lame questions, I have to somehow get out of this mess...

 

[HoneyBadger]

I am aware that ZFS does not want RAID controllers but a BOD (bunch of disks?).
Unfortunatelly I was not the one who setup the server.
How can I check now if the controller is flashed with correct firmware and if disks are raid0 virtual disks ?
Sorry for all that lame questions, I have to somehow get out of this mess...

sas2flash -list and lspci | grep LSI or lspci | grep SCSI output will confirm it, but looking at the smartctl results where it shows your disk make/model as "DELL PERC H710P" it's pretty clear there.

Since you see multiple disks within TrueNAS (da0, da1, up through da11) that means each disk was set up individually. Can you also post the output of zpool status -v?

When posting output please use the CODE tags so that the output is formatted properly:

Code:

it should look like this
   with leading spaces
and multiple rows

Unfortunately I don't believe that the H710P RAID0 virtual disks will be readable or importable by a proper HBA (an H310 mini can be purchased if you aren't able/comfortable with flashing the only controller you have) so the migration plan may involve "backup, destroy, restore"

 

[michal_f]

Code:

#
# sas2flash -list
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved

        No LSI SAS adapters found! Limited Command Set Available!
        ERROR: Command Not allowed without an adapter!
        ERROR: Couldn't Create Command -list
        Exiting Program.

So yeah... I need to flash my controller.

Also zpool status -v :

Code:

# zpool status -v
  pool: FramesPool
 state: ONLINE
  scan: scrub repaired 100K in 04:01:02 with 0 errors on Sun Aug  7 04:01:04 2022
config:

        NAME                                            STATE     READ WRITE CKSUM
        FramesPool                                      ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/247e2946-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/25c6b7d9-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/25b5df11-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/266a7fd1-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/2682bbfa-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/2707bc02-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/279222fa-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/2847842f-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/28117a44-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/285e1646-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
            gptid/287977e2-3fb1-11eb-93be-c81f66f2061b  ONLINE       0     0     0
        cache
          gptid/27f96034-3fb1-11eb-93be-c81f66f2061b    ONLINE       0     0     0
        spares
          gptid/28b0769a-3fb1-11eb-93be-c81f66f2061b    AVAIL

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: scrub repaired 0B in 00:00:07 with 0 errors on Tue Aug 30 03:45:09 2022
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2       ONLINE       0     0     0

errors: No known data errors

Thanks for all the help! I will now investigate how to obtaint correct firmware or buy new hardware...

 

[sretalla]

Maybe try sas3flash instead... it may be a PCIe v3 adapter.

 

[michal_f]

sas3flash also does not show anything, nor lspci

Do you know any good How-To for my controller model so I can flash with correct firmware ?

 

[sretalla]

You could see what shows up in dmesg

I guess it will be close to the beginning of the output and should be after the CPU and first sections of PCI bridge, etc.

 

[Redcoat]

Do you know any good How-To for my controller model so I can flash with correct firmware ?

https://fohdeesha.com/docs/perc.html