Device not available

[João Fernandes]

I've recently acquired a DL380p Gen8 server and I've managed to run TrueNAS Scale on the 2 disks provided that are connected to a P420i controller in HBA mode.
After validating that most of the necessary apps would properly run either with a VM or as within the K3S cluster, I've decided to buy 6 extra hard drives (Seagate Savvio ST900MM0006).

After adding those, no disks were made available in the disk section. Looking at the server, I've noticed a amber light.

Using this thread to troubleshoot, I wasn't able to even list those new drives.
After a server reboot, those were made available under the disk section inside TrueNAS but with 0 bytes available

My initial suspicion was that those might be with a 520 byte sector but when I tried to format with 512, I got the message that the device was not ready

currently I'm really not sure if this is related to TrueNAS but any hint would be welcome.

 

[Daisuke]

I'm really not sure if this is related to TrueNAS

Is not. Seagate ST900MM0 are mostly formatted with proprietary firmware, try using Seagate's SeaChest and see if you can read the disk. openSeaChest commands are already included into Scale, but we might need the proprietary SCSI commands.

Are you sure /dev/sg3 is the right disk? Run sg_map to see where /dev/sdc belongs to.

Edit: In case your are not familiar with the commands, run as root from terminal, presuming /dev/sg3 is mapped to /dev/sdc:

Code:

# openSeaChest_Info -d /dev/sg3 -i

Let me know if you an read the disk info. If you do, post the output using [CODE]text[/CODE]. No more screenshots from now on.

If the above command fails, you can try the Seagate proprietary SCSI command:

Code:

# cd /mnt/yourpool/yourdataset
# git clone https://github.com/Seagate/ToolBin.git
# cd ToolBin/SeaChest/Info/v2.1.2/Linux
# chmod 0755 *
# ./SeaChest_Info_x86_64-alpine-linux-musl_static -d /dev/sg3 -i
OUTPUT HERE

 

[jgreco]

connected to a P420i controller in HBA mode.

There is no such thing as a "P420i controller in HBA mode." You have a P420i in lobotomized RAID controller mode, and it is not acceptable for use with TrueNAS. Please see the following article.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

I know that HP desperately wants you to believe that there's an HBA mode, but that mode is NOT what we mean when we say that ZFS needs an LSI HBA in IT mode. Their "HBA mode" is just RAID mode with all the RAID functionality disabled.

 

[Daisuke]

Their "HBA mode" is just RAID mode with all the RAID functionality disabled.

Learning very day new things, from smart people. I'm curious why the disks still show with a 0byte.

 

[João Fernandes]

Is not. Seagate ST900MM0 are mostly formatted with proprietary firmware, try using Seagate's SeaChest and see if you can read the disk. openSeaChest commands are already included into Scale, but we might need the proprietary SCSI commands.

Are you sure /dev/sg3 is the right disk? Run sg_map to see where /dev/sdc belongs to.

Edit: In case your are not familiar with the commands, run as root from terminal, presuming /dev/sg3 is mapped to /dev/sdc:

Code:

# openSeaChest_Info -d /dev/sg3 -i

Let me know if you an read the disk info. If you do, post the output using [CODE]text[/CODE]. No more screenshots from now on.

If the above command fails, you can try the Seagate proprietary SCSI command:

Code:

# cd /mnt/yourpool/yourdataset
# git clone https://github.com/Seagate/ToolBin.git
# cd ToolBin/SeaChest/Info/v2.1.2/Linux
# chmod 0755 *
# ./SeaChest_Info_x86_64-alpine-linux-musl_static -d /dev/sg3 -i
OUTPUT HERE

Here is the output, finding really odd the drive capacity being 0/0

Code:

/dev/sg3 - ST900MM0 CLAR900 - S0N3M80A - LS5C - SCSI
        Vendor ID: SEAGATE
        Model Number: ST900MM0 CLAR900
        Serial Number: S0N3M80A
        Firmware Revision: LS5C
        World Wide Name: 5000C50088ABD667
        Copyright: Copyright (c) 2014 Seagate All rights reserved
        Drive Capacity (B/B): 0.00/0.00
        Temperature Data:
                Current Temperature (C): Not Reported
                Highest Temperature (C): Not Reported
                Lowest Temperature (C): Not Reported
        Power On Time: Not Reported
        Power On Hours: Not Reported
        MaxLBA: 0
        Native MaxLBA: Not Reported
        Logical Sector Size (B): 0
        Physical Sector Size (B): 0
        Sector Alignment: 0
        Rotation Rate (RPM): 10000
        Form Factor: 2.5"
        Last DST information:
                Not supported
        Long Drive Self Test Time:  1 hour 33 minutes
        Interface speed:
                Not Reported
        Annualized Workload Rate (TB/yr): Not Reported
        Total Bytes Read (B): Not Reported
        Total Bytes Written (B): Not Reported
        Encryption Support: Not Supported
        Cache Size (MiB): Not Reported
        Read Look-Ahead: Not Supported
        Write Cache: Not Supported
        SMART Status: Unknown or Not Supported
        ATA Security Information: Not Supported
        Firmware Download Support: Full, Deferred
        Number of Logical Units: 1
        Specifications Supported:
                SPC-3
                ----
                ----
                ----
                ----
                ----
                ----
                ----
                ----
        Features Supported:
                None reported or an error occurred while trying to determine
                the features.
        Adapter Information:
                Adapter Type: PCI
                Vendor ID: 103Ch
                Product ID: 323Bh
                Revision: 0001h

Thank you to showing me the right direction

 

[João Fernandes]

There is no such thing as a "P420i controller in HBA mode." You have a P420i in lobotomized RAID controller mode, and it is not acceptable for use with TrueNAS. Please see the following article.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

I know that HP desperately wants you to believe that there's an HBA mode, but that mode is NOT what we mean when we say that ZFS needs an LSI HBA in IT mode. Their "HBA mode" is just RAID mode with all the RAID functionality disabled.

I have to trust your opinion but the fact is, even if it's a sub-par HBA it's working with the provided hard drives that came with the server. My issue is with the new Seagate disks which are reporting 0 bytes of available space.

I guess my next investment will be on a LSI HBA card.

 

[Daisuke]

Here is the output

@jgreco explained where is the issue. Can you read the output of your command? Everything is detailed there. And you did not posted what you did exactly, you are supposed to insert the executed command and the output, like described. Please learn how to post properly the information.

 

[jgreco]

the fact is, even if it's a sub-par HBA it's working with the provided hard drives that came with the server.

It's not a sub-par HBA. It's a sub-par RAID controller. And it isn't working correctly with the hard drives that came with the server. You just haven't wrecked yet. That doesn't mean it is working correctly.

 

[ChrisRJ]

You just haven't wrecked yet. That doesn't mean it is working correctly.

The definition of "it works" is really critical. This is, by the way, one of the big problems I personally have with most of the YouTube videos on TrueNAS. Someone wants clicks and comes up with a story along the lines of "Here is how to run this great enterprise-grade piece of software on small/cheap hardware". The implication being that you can have reliable storage from this setup.

Of course this is plain wrong. It is similar to someone promising you a 20% return on investment and saying it is totally safe. If it sounds too good to be true, it isn't.

Working correctly does not mean that I finished the installation and initial setup steps successfully. Working means that whatever problems come the way will not lead to lost or (worse!) corrupted data. Think of HDDs dying or not working properly, brown-outs in electricity, PSU aging, contacts being corroded or having a borderline-level of resistance that make things crash when humidity is over a certain level, etc.

 

[João Fernandes]

@jgreco explained where is the issue. Can you read the output of your command? Everything is detailed there. And you did not posted what you did exactly, you are supposed to insert the executed command and the output, like described. Please learn how to post properly the information.

I've executed the command you recommended me to execute but I'll take note for the next time.
Regarding the ability to read, no, I'm not 100% sure what are the final conclusions but my wild guess is that as @jgreco mentioned, my card might be interfering. In the meanwhile, I've ordered a LSI in IT mode to replace the onboard raid controller.

 

[Daisuke]

I've executed the command you recommended me to execute but I'll take note for the next time.

Not a problem, I posted 2 SeaChest commands, one proprietary and I was expecting to see the commands and the outputs, like I posted into previous post. This way there is no confusion and is also useful for other users to clearly understand what you did.

Regarding the ability to read, no, I'm not 100% sure what are the final conclusions

The first command output shows only the disk basic information like name and serial, anything else is not readable/accessible. I expect you get an identical result with SeaChest_Info_x86_64-alpine-linux-musl_static.

What LSI card model you ordered?

 

[João Fernandes]

Not a problem, I posted 2 SeaChest commands, one proprietary and I was expecting to see the commands and the outputs, like I posted into previous post. This way there is no confusion and is also useful for other users to clearly understand what you did.

The first command output shows only the disk basic information like name and serial, anything else is not readable/accessible. I expect you get an identical result with SeaChest_Info_x86_64-alpine-linux-musl_static.

What LSI card model you ordered?

I got this one "LSI 9205-8i H220 6Gbps SAS PCI-E 3.0 IT Mode", it seems to be HP branded and will prevent the fans to spin up like crazy (something painful with this server version).

Regarding the command, I've ran both but they did in fact output similar information. What I posted was the alpine-linux-musl_static version.

 

[Daisuke]

I got this one "LSI 9205-8i H220 6Gbps SAS PCI-E 3.0 IT Mode"

It does not matter if is HP branded, you can flash it the way you like. That's an internal card, you need an internal model? On my server setup I use the equivalent of LSI 9207 internal and external.

I've ran both but they did in fact output similar information.

Great, that's what I wanted to know. Appreciated.

 

[João Fernandes]

It does not matter if is HP branded, you can flash it the way you like. That's an internal card, you need an internal model? On my server setup I use the equivalent of LSI 9207 internal and external.

Great, that's what I wanted to know. Appreciated.

Dl380p gen8 is known for having lousy fans spinning up when non HP cards are used, that's why I went for the H220 LSI one. I picked the internal model to disconnect the 8 disks from the P420i and connect it to the internal one. I've also ordered a external one for the upcoming upgrade with an external disk array.

 

[João Fernandes]

so I just got my HP H220 HBA card and connected the disks there. Here is ./SeaChest_Info_x86_64-alpine-linux-musl_static -d /dev/sde -i output.

Code:

root@truenas[~/ToolBin/SeaChest/Info/v2.1.2/Linux]# ./SeaChest_Info_x86_64-alpine-linux-musl_static -d /dev/sde -i
==========================================================================================
 SeaChest_Info - Seagate drive utilities - NVMe Enabled
 Copyright (c) 2014-2022 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
 SeaChest_Info Version: 2.1.2-3_2_1 X86_64
 Build Date: Jul 26 2022
 Today: Thu Jan 12 17:52:10 2023        User: root
==========================================================================================

/dev/sg5 - ST900MM0 CLAR900 - S0N3SB73 - LS5C - SCSI
        Vendor ID: SEAGATE
        Model Number: ST900MM0 CLAR900
        Serial Number: S0N3SB73
        Firmware Revision: LS5C
        World Wide Name: 5000C50088AF5F4F
        Copyright: Copyright (c) 2014 Seagate All rights reserved
        Drive Capacity (B/B): 0.00/0.00
        Temperature Data:
                Current Temperature (C): Not Reported
                Highest Temperature (C): Not Reported
                Lowest Temperature (C): Not Reported
        Power On Time: Not Reported
        Power On Hours: Not Reported
        MaxLBA: 0
        Native MaxLBA: Not Reported
        Logical Sector Size (B): 0
        Physical Sector Size (B): 0
        Sector Alignment: 0
        Rotation Rate (RPM): 10000
        Form Factor: 2.5"
        Last DST information:
                Not supported
        Long Drive Self Test Time:  1 hour 33 minutes
        Interface speed:
                Not Reported
        Annualized Workload Rate (TB/yr): Not Reported
        Total Bytes Read (B): Not Reported
        Total Bytes Written (B): Not Reported
        Encryption Support: Not Supported
        Cache Size (MiB): Not Reported
        Read Look-Ahead: Not Supported
        Write Cache: Not Supported
        SMART Status: Unknown or Not Supported
        ATA Security Information: Not Supported
        Firmware Download Support: Full, Deferred
        Number of Logical Units: 1
        Specifications Supported:
                SPC-3
                ----
                ----
                ----
                ----
                ----
                ----
                ----
                ----
        Features Supported:
                None reported or an error occurred while trying to determine
                the features.
        Adapter Information:
                Adapter Type: PCI
                Vendor ID: 1000h
                Product ID: 0087h
                Revision: 0005h

It's pretty much similar to the original behaviour. Any additional hints of what I could do differently? I already have a disk array on the way and I will be able to test there as well but I believe I will have the exact same outcome. The only difference I've noticed is the amber light (disk issues) on the tray is no longer on.