no such pool available, but the pools are listed and so are the disks.

EnCrypt3r · Oct 3, 2022

I know this has been gone over several times but i can't seem to find a solution or an instance that's really the same as mine, also i can't seem to get any less general errors so i can actually troubleshoot it.

I had to physically relocate my server as well as my DS4243 (running as JBOD). after relocating i put all the disks back in i believe in the exact order they were removed and for whatever reason while my freenas shows the pools, it says they're in an unknown status and offline. but all of the disks are there. i have two pools but only one that i actually care about.

Server info:
HP DL360 Gen7
32gb memory
2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
LSI Logic / Symbios Logic LSI2008

The VM has 8gb ram.

FreeNAS is ran as a VM on ESXI6.5 with the LSI2008 ran in passthrough mode on the vm; i've been using it in this configuration for several years and has been great until now.

At first I thought the disks were maybe put back in the DS4234 in the wrong order but i verified they should be correct and read later that freenas doesn't even care and uses some identifier to determine which disk belongs where. i'm getting the following output:

Code:

root@freenas[~]# zpool import
root@freenas[~]#
root@freenas[~]# zpool status -v
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:20 with 0 errors on Sat Jul 30 03:45:20 2022
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

Any help would be greatly appreciated!

Arwen · Oct 4, 2022

You supplied the 2 key pieces of information, no pools to import and other than the boot pool, nothing already imported. The "pools.png" simply shows what was their before. That is why they are UNKNOWN.

Checking your "dmesg.txt" output, it appears you have a problem with your hardware configuration;

Code:

GEOM: multipath/disk1: corrupt or invalid GPT detected.
GEOM: multipath/disk1: GPT rejected -- may not be recoverable.

Did you have 2 cables to the disk tray?
Or just 1 cable?

EnCrypt3r · Oct 4, 2022

I currently have one cable going from the LSI to the "square" indicated port on the array, the circle port is then plugged into the square of the second half of the array; when I re-looked through documentation for plugging this back in I was under the impression this was the correct way.

Also thanks for commenting I'm hoping to be able to get this resolved!

Arwen · Oct 4, 2022

Well, at present, I don't have any suggestions. The output of "dmesg.txt" does should a lot of disks. Its not clear why zpool import does not show a pool, (even one missing a few disks).

In general, ZFS does not care about the disk order. It generally tries to use the drive serial number to complete a pool disk list.

It is possible that your pool(s) are not showing up because the stock import does not scan the correct directories. This can be overcome with the following option. However, I would not know what option(s) would be appropriate for your server.

Code:

-d dir|device
     Uses device or searches for devices or files in dir.  The -d option can be specified multi-
     ple times.  This option is incompatible with the -c option.

EnCrypt3r · Oct 4, 2022

Arwen said:
Well, at present, I don't have any suggestions. The output of "dmesg.txt" does should a lot of disks. Its not clear why zpool import does not show a pool, (even one missing a few disks).

In general, ZFS does not care about the disk order. It generally tries to use the drive serial number to complete a pool disk list.

It is possible that your pool(s) are not showing up because the stock import does not scan the correct directories. This can be overcome with the following option. However, I would not know what option(s) would be appropriate for your server.

Code:
-d dir|device Uses device or searches for devices or files in dir. The -d option can be specified multi- ple times. This option is incompatible with the -c option.

is the -d option for zpool or what? i can't seem to find the reference you're using here.

EnCrypt3r · Oct 4, 2022

I ended up finding what you were referring to, and found the -c switch which allows for using the cache file that i already had, although it's claiming that the drives aren't working? or something. i unplugged a couple drives just to see if they went away and they do on the disks menu so i dont understand what's going on here.

Code:

root@freenas[/data/zfs]# zpool import -c zpool.cache
   pool: HawkEye
     id: 4685264409575521207
  state: UNAVAIL
 status: One or more devices are missing from the system.
 action: The pool cannot be imported. Attach the missing
        devices and try again.
   see: http://illumos.org/msg/ZFS-8000-3C
 config:

        HawkEye                   UNAVAIL  insufficient replicas
          raidz2-0                UNAVAIL  insufficient replicas
            3617874740870750488   UNAVAIL  cannot open
            13077544351036226941  UNAVAIL  cannot open
            8826285399754106850   UNAVAIL  cannot open
            14202738197255575510  UNAVAIL  cannot open
            14449837756396824126  UNAVAIL  cannot open
            1329651091363077540   UNAVAIL  cannot open
            2491121919027193420   UNAVAIL  cannot open
            5562359860256280278   UNAVAIL  cannot open
            8393652391275275353   UNAVAIL  cannot open
            12496207986016518407  UNAVAIL  cannot open
            14530922262134524121  UNAVAIL  cannot open
            8007533003357230128   UNAVAIL  cannot open

   pool: HoneyCut
     id: 11985135574227840896
  state: UNAVAIL
 status: One or more devices are missing from the system.
 action: The pool cannot be imported. Attach the missing
        devices and try again.
   see: http://illumos.org/msg/ZFS-8000-3C
 config:

        HoneyCut                  UNAVAIL  insufficient replicas
          raidz2-0                UNAVAIL  insufficient replicas
            1979269319180743079   UNAVAIL  cannot open
            3374053863492469773   UNAVAIL  cannot open
            2760031973361069160   UNAVAIL  cannot open
            7477590995145616912   UNAVAIL  cannot open
            17219062780889152897  UNAVAIL  cannot open
            4467947160299885910   UNAVAIL  cannot open

EnCrypt3r · Oct 4, 2022

I also verified the disks are actually communicating and ran smartctl on them. i don't know what else i can do to show that they're active but just not being used for some reason.

Code:

root@freenas[/data/zfs]# smartctl -i /dev/da41
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SA
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x50000c900002cad4
Serial number:        ZFL330WR
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:41:56 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

root@freenas[/data/zfs]# smartctl -i /dev/da35
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SA
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x50000c90000254b8
Serial number:        ZFL330F1
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:42:01 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

root@freenas[/data/zfs]# smartctl -i /dev/da30
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SA
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x50000c9000026848
Serial number:        ZFL33CRV
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:42:06 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

root@freenas[/data/zfs]# smartctl -i /dev/da29
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SA
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x50000c90000256ec
Serial number:        ZFL33126
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:42:10 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

root@freenas[/data/zfs]# smartctl -i /dev/da23
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SA
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x50000c900002cb0c
Serial number:        ZFL333WV
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:42:17 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

root@freenas[/data/zfs]# smartctl -i /dev/da20
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               ST2000DM
Product:              008-2FR102    SM
Revision:             4321
User Capacity:        2,000,398,934,016 bytes [2.00 TB]
Logical block size:   512 bytes
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x500605ba004b3eec
Serial number:        ZFL331CH
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Tue Oct  4 20:42:22 2022 CDT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled

Arwen · Oct 4, 2022

The whole option would be something like;

Code:

zpool import -d /dev/disk/by-id

But, that is specific to Linux and I don't know what your VMWare instance is supplying for disk device paths.

You could poke around in /dev/ to see if all your disks are in a different path.

Based on your SMART command line, I am guessing you could use something like this to get a clue;

Code:

zpool import -d /dev/da35

If that works to get 1 disk recognized, then you repeat the -d /dev/daX, replacing X with another known working disk. Repeat ad nauseam until you have them all, (if it works). Whence a pool has all it's disks listed correctly, you can then add the pool name to the command line at the end to actually import it.

EnCrypt3r · Oct 7, 2022

Arwen said:
The whole option would be something like;

Code:
zpool import -d /dev/disk/by-id

But, that is specific to Linux and I don't know what your VMWare instance is supplying for disk device paths.

You could poke around in /dev/ to see if all your disks are in a different path.

Based on your SMART command line, I am guessing you could use something like this to get a clue;

Code:
zpool import -d /dev/da35

If that works to get 1 disk recognized, then you repeat the -d /dev/daX, replacing X with another known working disk. Repeat ad nauseam until you have them all, (if it works). Whence a pool has all it's disks listed correctly, you can then add the pool name to the command line at the end to actually import it.

i tried this switch and it just tells me those are not directories.

Code:

root@freenas[/dev]# zpool import -d da30
cannot open '/dev/da30/': Not a directory

Arwen · Oct 7, 2022

You might use the prefix;
zpool import -d /dev/da35
Beyond that, I don't have any suggestions.

It's almost like your combination of VMWare, device pass through & TrueNAS is not picking up enough details to work properly. My skills with VMWare are quite limited today.

Important Announcement for the TrueNAS Community.

no such pool available, but the pools are listed and so are the disks.

EnCrypt3r

Cadet

Attachments

Arwen

MVP

EnCrypt3r

Cadet

Arwen

MVP

EnCrypt3r

Cadet

EnCrypt3r

Cadet

EnCrypt3r

Cadet

Arwen

MVP

EnCrypt3r

Cadet

Arwen

MVP

Similar threads