Script to report zpool drive information. v1.0.5 "zpool-drive-info.py"

Status
Not open for further replies.

rluiten

Cadet
Joined
Jan 26, 2013
Messages
6
Friend was getting annoyed with figuring out physical drive information in all his zpools so wrote this.

It dumps all the devices in a zpool looking up the device id, device info and serial number.
Its been tested on FreeNAS 8.3 and 9.3 so far.

To use it copy the attached file and rename it to remove the ".txt" extension.
Login as root on your FreeNas box and execute it with python.
For my zpool is called tank.
"python zpool-drive-info.py tank"

Have another friend using it capture state of his build, in case he cant get the same information back later if something is in an error state.

I have attached v1.0.5
I am a developer though havn't used python much so maybe its a bit naive.
I am also not an expert on FreeNAS or ZFS but I believe I havn't made serious errors.

This script does not write any data to disk or change any confguration state.
It only executes the following commands to collect information and present it.

# "zpool status"
# "camcontrol devlist"
# "glabel status"
# "smartctl -i /dev/<device>" - this is run for each device to lookup serial


WARNING - Any script you find on the internet might be doing bad things so in general you should not just grab it and run it. Either you should understand it or someone you trust understands it before running it.

Example output. I hid the serial numbers in the example.
Code:
[root@tonk] /mnt/tank/b1# python zpool-drive-info.py tank
      pool:  tank
      vdev:    raidz1-0 (1)

        id:      gptid/9c686292-6436-11e2-bf42-3cd92b042ab1
    device:        ada0
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]

        id:      gptid/9d22503a-6436-11e2-bf42-3cd92b042ab1
    device:        ada1
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]

        id:      gptid/9dd96a31-6436-11e2-bf42-3cd92b042ab1
    device:        ada2
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]

        id:      gptid/9e8ffacf-6436-11e2-bf42-3cd92b042ab1
    device:        ada3
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]

        id:      gptid/9f4ca51b-6436-11e2-bf42-3cd92b042ab1
    device:        ada4
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]

        id:      gptid/a00e3056-6436-11e2-bf42-3cd92b042ab1
    device:        ada5
driveinfo:        <Hitachi HDS5C3030ALA630 MEAOA5C0>
    serial:        [Serial Hidden]



 

Attachments

  • zpool-drive-info.py.txt
    5.5 KB · Views: 1,006
Last edited:

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
WARNING - Any script you find on the internet might be doing bad things so in general you should not just grab it and run it. Either you should understand it or someone you trust understands it before running it.


I ran your script and it turned my box into a gayporn streaming service!!!!!

Oh no wait, my bad, I was ssh'd into Cyberjock's box. The script seems fine.
 

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
Not sure if the OP's friend cares about getting the script running on 9.10, but all 6 drives in my raidz2 vdev have the same driveinfo output:
Code:
[root@freenas] ~# python zpool-drive-info.py pool
      pool:  pool
      vdev:    raidz2-0 (1)

        id:      gptid/b5d788fd-cd82-11e2-b330-002590ae2bed
    device:        da0
 driveinfo:        Weird "da0" not found in device list
    serial:        13K82P0AS
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Not sure if the OP's friend cares about getting the script running on 9.10, but all 6 drives in my raidz2 vdev have the same driveinfo output:
Code:
[root@freenas] ~# python zpool-drive-info.py pool
      pool:  pool
      vdev:    raidz2-0 (1)

        id:      gptid/b5d788fd-cd82-11e2-b330-002590ae2bed
    device:        da0
driveinfo:        Weird "da0" not found in device list
    serial:        13K82P0AS
That's unlikely to be a 9.10 vs 9.3 problem. Can we see the output of "camcontrol devlist", please, on your box?
 

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
Code:
[root@freenas] ~# camcontrol devlist
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 2 lun 0 (da0,pass0)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 3 lun 0 (da1,pass1)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 4 lun 0 (da2,pass2)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 5 lun 0 (da3,pass3)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 10 lun 0 (da4,pass4)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 11 lun 0 (da5,pass5)
<SATA SSD S9FM01.6>                at scbus1 target 0 lun 0 (ada0,pass6)
 

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
The formatting inside the ()'s is different on my 8.3.2 boxes.
Code:
[root@freenas01] ~# camcontrol devlist
<LSI SAS2X28 0e0b>                 at scbus0 target 8 lun 0 (ses0,pass0)
<LSI SAS2X36 0e0b>                 at scbus0 target 9 lun 0 (ses1,pass1)
<SEAGATE ST33000650SS 0004>        at scbus0 target 10 lun 0 (pass2,da0)
<SEAGATE ST33000650SS 0004>        at scbus0 target 11 lun 0 (pass3,da1)
<SEAGATE ST33000650SS 0004>        at scbus0 target 12 lun 0 (pass4,da2)
<SEAGATE ST33000650SS 0004>        at scbus0 target 13 lun 0 (pass5,da3)
<SEAGATE ST33000650SS 0004>        at scbus0 target 14 lun 0 (pass6,da4)
<SEAGATE ST33000650SS 0004>        at scbus0 target 15 lun 0 (pass7,da5)
<SEAGATE ST33000650SS 0004>        at scbus0 target 16 lun 0 (pass8,da6)
<SEAGATE ST33000650SS 0004>        at scbus0 target 17 lun 0 (pass9,da7)
<snip>
< Patriot Memory PMAP>             at scbus8 target 0 lun 0 (pass42,da36)
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
The formatting inside the ()'s is different on my 8.3.2 boxes.
I'll take a look at the script myself--it's probably an easy fix.

But I'd love to ask: Why in Muhammad, Buddha, and Jesus' name are you running 8.3.2 boxes?
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
take the three lines that are "commented" in here with the #, and take out the # temporarily. Then show me the output when you run it please.
Code:
# print camcontrol
camcontrolStucture = {}
for line in camcontrol.splitlines():
  # print "a", "\"" + line + "\""
  match = re.search(r'^\s*(.*)\s*\sat\s.*,([^)]+)\)$', line)
  if match:
  camcontrolStucture[match.group(2)] = match.group(1).strip()
# print "camcontrolStucture", camcontrolStucture
 

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
Why in Muhammad, Buddha, and Jesus' name are you running 8.3.2 boxes?
I know, it's nasty. They're Supermicro e1r36n boxes at work. For the most part one serves up the storage and the other is a replication target. The primary hosts a bunch of nfs storage for vsphere, 20+ tb storage for veeam backups, misc direct-attach nfs and iscsi, etc. Taking them off line is a major event. I did it once when a usb key failed, but otherwise I just let them chug along.

I'm not going to even try to update them. I went through that with my home freenas box and I don't consider the major version upgrade process reliable enough to use in production.

I ordered a couple of new e1cr36l boxes. When I get them online I'm going to migrate the data off the existing boxes, then i'm going to wipe them and set them up as replication targets.
 

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
take the three lines that are "commented" in here with the #, and take out the # temporarily. Then show me the output when you run it please.
Code:
# print camcontrol
camcontrolStucture = {}
for line in camcontrol.splitlines():
  # print "a", "\"" + line + "\""
  match = re.search(r'^\s*(.*)\s*\sat\s.*,([^)]+)\)$', line)
  if match:
  camcontrolStucture[match.group(2)] = match.group(1).strip()
# print "camcontrolStucture", camcontrolStucture
Code:
[root@freenas] ~# python ./zpool-drive-info.py pool
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 2 lun 0 (da0,pass0)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 3 lun 0 (da1,pass1)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 4 lun 0 (da2,pass2)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 5 lun 0 (da3,pass3)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 10 lun 0 (da4,pass4)
<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 11 lun 0 (da5,pass5)
<SATA SSD S9FM01.6>                at scbus1 target 0 lun 0 (ada0,pass6)

a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 2 lun 0 (da0,pass0)"
a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 3 lun 0 (da1,pass1)"
a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 4 lun 0 (da2,pass2)"
a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 5 lun 0 (da3,pass3)"
a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 10 lun 0 (da4,pass4)"
a "<ATA TOSHIBA DT01ACA3 ABB0>        at scbus0 target 11 lun 0 (da5,pass5)"
a "<SATA SSD S9FM01.6>                at scbus1 target 0 lun 0 (ada0,pass6)"
camcontrolStucture {'pass4': '<ATA TOSHIBA DT01ACA3 ABB0>', 'pass5': '<ATA TOSHIBA DT01ACA3 ABB0>', 'pass6': '<SATA SSD S9FM01.6>', 'pass0': '<ATA TOSHIBA DT01ACA3 ABB0>', 'pass1': '<ATA TOSHIBA DT01ACA3 ABB0>', 'pass2': '<ATA TOSHIBA DT01ACA3 ABB0>', 'pass3': '<ATA TOSHIBA DT01ACA3 ABB0>'}
      pool:  pool
      vdev:    raidz2-0 (1)

        id:      gptid/2ae3a80c-de34-11e5-a364-0015175d4010
    device:        da4
 driveinfo:        Weird "da4" not found in device list
    serial:        Y41TRV7GS

        id:      gptid/addd78e8-e01e-11e5-821e-0015175d4010
    device:        da5
 driveinfo:        Weird "da5" not found in device list
    serial:        168T3NGKS

        id:      gptid/b3bfc440-cd82-11e2-b330-002590ae2bed
    device:        da3
 driveinfo:        Weird "da3" not found in device list
    serial:        13F7AESAS

        id:      gptid/b46f2b33-cd82-11e2-b330-002590ae2bed
    device:        da2
 driveinfo:        Weird "da2" not found in device list
    serial:        13K84SXGS

        id:      gptid/b5210424-cd82-11e2-b330-002590ae2bed
    device:        da1
 driveinfo:        Weird "da1" not found in device list
    serial:        13K8LLNGS

        id:      gptid/b5d788fd-cd82-11e2-b330-002590ae2bed
    device:        da0
 driveinfo:        Weird "da0" not found in device list
    serial:        13K82P0AS
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
well I think we see what the problem is. It's pulling down the second tag instead of the first. The script should be written so that it determines which coordinate has the (a)da tag in it, not just assume it's in the second place.

If I get a chance later I'll fix it.
 

rluiten

Cadet
Joined
Jan 26, 2013
Messages
6
Wow, greatwork DrKK.

I had made assumptions on data format.
I have update and attached the script so it should not care about the order of those last two fields from "camcontrol devlist".
 

Attachments

  • zpool-drive-info.py.txt
    6 KB · Views: 956

KevinM

Contributor
Joined
Apr 23, 2013
Messages
106
Wow, greatwork DrKK.

I had made assumptions on data format.
I have update and attached the script so it should not care about the order of those last two fields from "camcontrol devlist".
Works on 9.10!
Code:
[root@freenas] ~# python zpool-drive-info.py pool
      pool:  pool
      vdev:    raidz2-0 (1)

        id:      gptid/2ae3a80c-de34-11e5-a364-0015175d4010
    device:        da4
 driveinfo:        <ATA TOSHIBA DT01ACA3 ABB0>
    serial:        Y41TRV7GS
...
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630

MrToddsFriends

Documentation Browser
Joined
Jan 12, 2015
Messages
1,338
Nice utility, unfortunately v1.0.6 (and maybe the versions before) doesn't display the desired information for pools made out of a single vdev that consists of a single disk.

In such a case there is no line stating 'mirror-0', 'raidz2' or whatsoever.
Code:
[root@blunzn] ~# zpool status volrep
  pool: volrep
state: ONLINE
  scan: scrub repaired 0 in 1h57m with 0 errors on Sat Apr 16 00:57:39 2016
config:

        NAME                                          STATE     READ WRITE CKSUM
        volrep                                        ONLINE       0     0     0
          gptid/61f19d27-e65e-11e4-afbd-0cc47a317e3c  ONLINE       0     0     0

errors: No known data errors


This leads to no match for a dev level 2 in your code with the following output for a single disk:
Code:
[root@blunzn] ~# python ./zpool-drive-info_orig.py volrep
      pool:  volrep
      vdev:    gptid/61f19d27-e65e-11e4-afbd-0cc47a317e3c (1)


I created a small hack to your code to circumvent this situation, but be warned: I'm an absolute beginner in python. And: My patch results in handling single disks as dev level 1 and dev level 2 at the same time (I don't know if this could harm in combination with other disk configurations than mine).

The patch:
Code:
[root@blunzn] ~# diff zpool-drive-info_orig.py zpool-drive-info.py
89a90
>  matchgptid = -1 # single disk marker
91a93,94
>  # single disk is handled as dev level 1 and dev level 2 at the same time
>  matchgptid = newDev1.find("gptid")
96c99
<  if match and match.start(1) == 4:  # found dev level 2
---
>  if (match and match.start(1) == 4) or (matchgptid >= 0):  # found dev level 2


With this patch the output for the single disk (and single vdev) pool is:
Code:
[root@blunzn] ~# python ./zpool-drive-info.py volrep
      pool:  volrep
      vdev:    gptid/61f19d27-e65e-11e4-afbd-0cc47a317e3c (1)

        id:      gptid/61f19d27-e65e-11e4-afbd-0cc47a317e3c
    device:        ada2
driveinfo:        <WDC WD20EFRX-68AX9N0 80.00A80>
    serial:        WD-WMC300241919


All testing done with FreeNAS 9.10, 2-way mirror vdevs (one vdev per pool) and this single disk pool. Maybe this helps for further development.
 

rluiten

Cadet
Joined
Jan 26, 2013
Messages
6
That seems a very reasonable way to resolve it, thanks for the feedback.
I did change your variable name.
Updated script v1.0.7
 

Attachments

  • zpool-drive-info.py.txt
    6.3 KB · Views: 1,058

MrToddsFriends

Documentation Browser
Joined
Jan 12, 2015
Messages
1,338
Updated script v1.0.7

Assuming you don't have such a degenerated vdev at hand: Output of v1.0.7 is identical to that of my patched v1.0.6 with my disk configuration.
 

rluiten

Cadet
Joined
Jan 26, 2013
Messages
6
You are right I do not have access to any setup like that. I can substitute the return result of the run commands to get specific scenarios for for testing, (you can see import test commented out at top and some other test lines commented out) to check it "should" work, its nice to have confirmation though, Thanks.

I should look at using a test runner, but I don't do much python and just couldn't be bothered.
 
Status
Not open for further replies.
Top