SOLVED Missing SSD cache drive

[mlinton]

Hey all, I'm asking for some grace on this, I normally wear a NetApp hat and FreeNAS is a little new to me.

As an experiment I set up a FreeNAS box with 2x 500gb drives (zfs, mirrored) and 1x 60gb SSD. FreeNAS is installed on. A USB stick.

I had the SSD set up as a cache drive and things were working well. The whole system was setup as an iSCSI device serving an Esxi host.

Now for the bad news, since my test was successful I destroyed the volume (marked disk as new) and removed the 500gb drives. I then installed my 2x 2tb drives that I intended to use for serving data. Once again, I mirrored the drives, however the SSD no longer shows up in volume manager. If I click - "view disks" I can see the drive (it shows up at ada0).

I also checked the shell but there appears to be a disconnect between the shell and the GUI, so I'm not sure which ones better. Running a diskinfo does show all the drives.

Do I need to kick that drive out and bring it back in somehow?

Side note, I was using full disk encryption.

Thanks again,
Mark

 

[paleoN]

I also checked the shell but there appears to be a disconnect between the shell and the GUI, so I'm not sure which ones better.

CLI always wins. However, you often need to deal with the disconnect later.

Code:

zpool status -v

zpool import

camcontrol devlist

glabel status

gpart show

Have you rebooted? I assume the cache drive was also encrypted in the test pool. Perhaps GELI is still holding onto it?

 

[mlinton]

I am about to reboot the box, I have included the output for each of those commands below. The OCZ is the one that I'm using for cache. As of right now, I can use volume manager to create a volume with the 2x 2tb drives, but can't add the ssd.

Thanks for your help,
Mark

Code:

Welcome to FreeNAS
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# zpool status -v
no pools available
[root@freenas] ~# zpool import
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# camcontrol devlist
<OCZ-AGILITY2 1.37>                at scbus3 target 0 lun 0 (pass0,ada0)
<TOSHIBA DT01ACA200 MX4OABB0>      at scbus4 target 0 lun 0 (pass1,ada1)
<TOSHIBA DT01ACA200 MX4OABB0>      at scbus5 target 0 lun 0 (ada2,pass3)
<PNY USB 2.0 FD 1100>              at scbus7 target 0 lun 0 (pass2,da0)
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# glabel status
                                      Name  Status  Components
                            ufs/FreeNASs3    N/A  da0s3
                            ufs/FreeNASs4    N/A  da0s4
                            ufs/FreeNASs1a    N/A  da0s1a
gptid/13b2b547-d94a-11e2-b111-001bfcb158bc    N/A  ada1p2
gptid/1473726e-d94a-11e2-b111-001bfcb158bc    N/A  ada2p2
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# gpart show
=>      63  15950529  da0  MBR  (7.6G)
        63  1930257    1  freebsd  [active]  (942M)
  1930320        63      - free -  (31k)
  1930383  1930257    2  freebsd  (942M)
  3860640      3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  12045600      - free -  (5.8G)
 
=>      0  1930257  da0s1  BSD  (942M)
        0      16        - free -  (8.0k)
      16  1930241      1  !0  (942M)
 
=>        34  3907029101  ada1  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834696    2  freebsd-zfs  (1.8T)
  3907029128          7        - free -  (3.5k)
 
=>        34  3907029101  ada2  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834696    2  freebsd-zfs  (1.8T)
  3907029128          7        - free -  (3.5k)
 
[root@freenas] ~# 

 

[mlinton]

I rebooted. Still no success. I have attached screen shots in case those will help.

Mark

 

[dlavigne]

Does the device show if you extend the volume using Volume Manager?

 

[mlinton]

I created a volume with the 2x 2tb drives, then went back in to expand it and the SSD was still missing from the list... I tried wiping the drive, that didn't work either, FreeNAS seems to have a lock on it.

More Info... As you can see, the system knows that the drive is there. I just can't use it.

Code:

[root@freenas] ~#
[root@freenas] ~# diskinfo -v ada1
ada1
        512            # sectorsize
        60022480896    # mediasize in bytes (55G)
        117231408      # mediasize in sectors
        0              # stripesize
        0              # stripeoffset
        116301          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        OCZ-I93484L650T6N53B    # Disk ident.
 
[root@freenas] ~# zpool list
NAME  SIZE  ALLOC  FREE    CAP  DEDUP  HEALTH  ALTROOT
zfs1  1.81T  576K  1.81T    0%  1.00x  ONLINE  /mnt
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# zpool add zfs1 cache ada1
cannot add to 'zfs1': no such pool or dataset
[root@freenas] ~#

 

[mlinton]

Got it!

This command fixed the problem.

Code:

sysctl kern.geom.debugflags=0x10

And based on the following output, you can see that I now have a cache drive again. Thank you for the help!

Code:

[root@freenas] ~# sysctl kern.geom.debugflags=0x10
kern.geom.debugflags: 0 -> 16
[root@freenas] ~#
[root@freenas] ~#
[root@freenas] ~# zpool add zfs1 cache ada1
[root@freenas] ~#
[root@freenas] ~# zpool status zfs1
  pool: zfs1
state: ONLINE
  scan: none requested
config:
 
        NAME                                            STATE    READ WRITE CKS                                            UM
        zfs1                                            ONLINE      0    0                                                  0
          mirror-0                                      ONLINE      0    0                                                  0
            gptid/fbbeaed4-dadc-11e2-8045-001bfcb158bc  ONLINE      0    0                                                  0
            gptid/fc411771-dadc-11e2-8045-001bfcb158bc  ONLINE      0    0                                                  0
        cache
          ada1                                          ONLINE      0    0                                                  0
 
errors: No known data errors
[root@freenas] ~#
[root@freenas] ~#

Is there a way to mark a thread as "solved"?

Mark

 

[paleoN]

Good job on getting it working.

Code:

sysctl kern.geom.debugflags=0x10

This is shouldn't be required as GEOM normally shouldn't be holding onto it. If you haven't yet, do a reboot to make sure whatever it was doesn't grab the drive before ZFS get's to it. If you still see the cache drive then you should be working.

Is there a way to mark a thread as "solved"?

Once upon a time. It may be broken currently. Did you try editing the first post?

 

[Soljia]

Removed