Command sesutil

[Tony-1971]

Hello,

I have FreeNAS 11.2 U7 on SuperServer 5028D-TN4T with this backplane https://www.supermicro.com/manuals/other/CSE-SAS733TQ.pdf

If I try sesutil command it doesn't work:

Code:

root@freenas-sm[~]# sesutil status
sesutil: No SES devices found

root@freenas-sm[~]# ls -la /dev/ses*
zsh: no matches found: /dev/ses*

root@freenas-sm[~]# ls -la /var/run/dmesg.boot
-rw-r--r--  1 root  wheel  15747 Jan 15 21:24 /var/run/dmesg.boot
root@freenas-sm[~]# grep -i ses /var/run/dmesg.boot
root@freenas-sm[~]# grep -i enc /var/run/dmesg.boot | grep -v -i Frequency | grep -v Encryption

root@freenas-sm[~]# kldstat -v | grep -i -E "enc|ses"
                29 enc

In the BIOS I set SATA ports as AHCI.

I miss something? Is the backplane compatible with the installed version of FreeNAS?

Best Regards,
Antonio

 

[danb35]

If I try sesutil command it doesn't work:

No, it won't--the -TQ backplanes are dumb devices with no logic on them.

 

[Tony-1971]

What do you mean with dumb devices?
In manual I found:
7. MG9071 Chip The MG9071 is an enclosure management chip that supports the SES-2 controller and protocols. (Page 2-3)

Maybe this chip is not supported?

Best Regards,
Antonio

 

[cobrakiller58]

in the same manual it states

3. Sideband Header The sideband header is designated JP51. For SES-2 to work properly, you must connect an 8-pin sideband cable. See the table to the right for pin definitions.

Is it connected to your controller?

 

[Tony-1971]

No, it is connected directly to the motherboard (I don't have a controller like LSI HBA).
From manual of the motherboard (https://www.supermicro.com/manuals/superserver/mid-tower/MNL-5028D-TN4T.pdf) seems connected in the right place: I-SGPIO1 (never modified it).

Best Regards,
Antonio

 

[cobrakiller58]

When I pulled up the manual for the motherboard I was expecting to see mention of a SAS controller but it does not appear to have one without that I don't *THINK* sesutil will work. I don't have any experience with that setup however so I'm not sure sorry.

Edit: well apparently that assumption was incorrect.

 

[Tony-1971]

You are right: no SAS controller; only six SATA ports on motherboard.
Maybe this is the problem...

Thank you!

Best Regards,
Antonio

 

[Tony-1971]

After upgrading Freenas to version 11.3 sesutil start to work:

Code:

root@freenas-sm[~]# sesutil status
ses0: OK

root@freenas-sm[~]# sesutil map
ses0:
        Enclosure Name: AHCI SGPIO Enclosure 2.00
        Enclosure ID: 3061686369656d30
        Element 0, Type: Array Device Slot
                Status: Unsupported (0x00 0x00 0x00 0x00)
                Description: Drive Slots
        Element 1, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 00
                Device Names: ada0,pass0
        Element 2, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 01
                Device Names: ada1,pass1
        Element 3, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 02
                Device Names: ada2,pass2
        Element 4, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 03
                Device Names: ada3,pass3
        Element 5, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 04
                Device Names: ada4,pass4
        Element 6, Type: Array Device Slot
                Status: OK (0x01 0x00 0x00 0x00)
                Description: Slot 05
                Device Names: ada5,pass5

But the mapping disk <-> slot is wrong:

Code:

root@freenas-sm[~]# grep -i ses /var/run/dmesg.boot
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ses0: ada0 in 'Slot 00', SATA Slot: scbus0 target 0
ses0: ada1 in 'Slot 01', SATA Slot: scbus1 target 0
ses0: ada2 in 'Slot 02', SATA Slot: scbus2 target 0
ses0: ada3 in 'Slot 03', SATA Slot: scbus3 target 0
ses0: ada4 in 'Slot 04', SATA Slot: scbus4 target 0
ses0: ada5 in 'Slot 05', SATA Slot: scbus5 target 0

root@freenas-sm[~]# camcontrol devlist
<SuperMicro SSD SOB20R>            at scbus0 target 0 lun 0 (pass0,ada0)
<WDC WD80EFZX-68UW8N0 83.H0A83>    at scbus1 target 0 lun 0 (pass1,ada1)
<WDC WD80EFAX-68KNBN0 81.00A81>    at scbus2 target 0 lun 0 (pass2,ada2)
<WDC WD80EFAX-68LHPN0 83.H0A83>    at scbus3 target 0 lun 0 (pass3,ada3)
<INTEL SSDSC1BG400G4 G2010150>     at scbus4 target 0 lun 0 (pass4,ada4)
<WDC WD80EFZX-68UW8N0 83.H0A83>    at scbus5 target 0 lun 0 (pass5,ada5)
<AHCI SGPIO Enclosure 2.00 0001>   at scbus6 target 0 lun 0 (pass6,ses0)

root@freenas-sm[~]# getencstat -v /dev/ses0
/dev/ses0: Enclosure Status <OK>
Element 0x0: Array Device Slot, status: Unsupported (0x00 0x00 0x00 0x00), descriptor: 'Drive Slots'
Element 0x1: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 00', dev: 'ada0,pass0'
Element 0x2: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 01', dev: 'ada1,pass1'
Element 0x3: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 02', dev: 'ada2,pass2'
Element 0x4: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 03', dev: 'ada3,pass3'
Element 0x5: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 04', dev: 'ada4,pass4'
Element 0x6: Array Device Slot, status: OK (0x01 0x00 0x00 0x00), descriptor: 'Slot 05', dev: 'ada5,pass5'

The locate subcommand of sesutil turn on and off led for slot 0 to slot 3, but ada0 is an internal SataDOM (not in slot 0).
In my case slot 0 is the lowest one and slot 3 is the highest one and they are populated with WDC disks (see camcontrol command above).

Best Regards,
Antonio

 

[cobrakiller58]

that's interesting FreeNAS 11.2 U7 did not recognize the CSE-SAS-733TQ but 11.3 does?

 

[Tony-1971]

Yes, the only modification was upgrade FreeNAS to 11.3
But sesutil is not very useful because of wrong mapping and I think that the mapping is made in hardware: I must connect Sata0 to slot0, Sata1 to slot1, etc.
Unfortunately Sata0 (ada0 in my configuration) was for SataDOM without the need of external power, and I don't want to use this port for HDD.

Best Regards,
Antonio

 

[cobrakiller58]

ada* numbers can move to different ports, as far as I know it is a software thing I never worry about. The fact that sesutil now works for you means you can easily identify a specific failed device which is useful.

 

[Tony-1971]

Not sure if I can identify a specific failed device.
Now if ada0 fails (Sata DOM that contains FreeNAS OS) and I use sesutils to locate the slot it shows slot 0 that contain a WD Drive part of another pool.

So ada* can move to different ports in FreeNAS OS but I think that my backplane want SATA 0 on SLOT 0, etc
For example if I connect SATA 3 to SLOT 0 in backplane, sesutil think that in SLOT 0 is SATA 0, but from OS view SATA 3 is ada 3

I'm not sure about this: I have to investigate some more about my hw configuration.
I want to poweroff the system and check which HDD is in which slot using HDD Serial Number and then maybe I understand better what is going on.

Best Regards,
Antonio

 

[Johnny Fartpants]

This is what I do https://www.ixsystems.com/community/threads/identify-missing-disk.70197/#post-484505

 

[cobrakiller58]

There is a script on here that is supposed to flash the locate light automatically when a drive fails in a pools which I have running, most of my trays have the drives serial number on them. Really need to do the ones I was lazy with.....

 

[Johnny Fartpants]

There is a script on here that is supposed to flash the locate light automatically when a drive fails in a pools which I have running

That sounds cool, although I would say 99% of my drive replacements are done before they are considered 'failed' by the system. I pull them based on SMART stats and if they look like they're going to pop to avoid any unnecessary headaches.

 

[cobrakiller58]

@Johnny Fartpants same that's why i had to say "supposed to flash" I haven't had any straight out die to test how well it works aside from basic testing to make sure all the commands functioned on my system. here's the link to it zpscan

Tony this uses sas2ircu so I would bet it would not work on your system sadly may be convertible to use sesutil though

 

[Tony-1971]

I try the command but doesn't work:

Code:

root@freenas-sm[~]# sas2ircu LIST
LSI Corporation SAS2 IR Configuration Utility.
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved.

SAS2IRCU: No Controllers Found.

I had checked HDD positions inside slots.
Physical slots (slot 00 is the lowest one) from command sesutil map:

Code:

Slot 03 -> pass3 OK -> ada3 OK
Slot 02 -> pass2 OK -> ada2 OK
Slot 01 -> pass1 OK -> ada1 OK
Slot 00 -> pass0 OK -> ada0 WRONG (is ada5)

sesutil locate pass<n> on -> work as expected

sesutil locate ada5 on -> NO Blinking red light (WRONG because disk is on slot 00)
sesutil locate ada0 on -> Slot 00 Blinking red light (WRONG because disk is attached directly on SATA 0 port)

And this is the real position of eache WD HDD:

Code:

Name    Pool        Serial                    Disk Size
ada0    Boot Pool   XXXX2015               14.75 GiB
ada1    tank-big    XXXXWB9Y                7.28 TiB            -> Slot 01
ada2    tank-big    XXXX58JL                7.28 TiB            -> Slot 02
ada3    tank-big    XXXX0J4W                7.28 TiB            -> Slot 03
ada4    tank-big    XXXX0RGN              372.61 GiB (CACHE)
ada5    tank-big    XXXX53EY                7.28 TiB            -> Slot 00
nvd0    tank-big    XXXX0AGN              260.83 GiB (LOG)

So I think that is mandatory to connect Sata0 to Slot 00, Sata1 to Slot 01, etc, to have sesutli working well.

Best Regards,
Antonio

 

[cobrakiller58]

I understand now, I guess this is the dumb part of the TQ series backplanes that danb35 mentioned. Even if sesutil knows it is there if it activates the wrong devices locator light, like you said it isn't very useful.