What is best way to add hot swap drive to freenas ZFS pool?

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
Ok I think I have most of this down but I'm confused out process.

I have one zpool=tank

My hot swap or spare is located at /dev/ada3

When I look at zpool status everything is listed by a gptid/<long number>

I look up how the devices IDs were being mapped to gpt IDs and a post recommended gpart list. Unfortunately since /dev/ada3 is unpartitioned and empty, this device isn't showing.

Do I need to partition drive or add by gptID? or can I add by devID?
Oracle's documentation is a little sketchy on this one: https://docs.oracle.com/cd/E53394_01/html/E54801/gpegp.html#SVZFSgjfbs

Code:
# zpool status
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:02:35 with 0 errors on Tue Feb  4 03:47:35 2020
config:

    NAME        STATE     READ WRITE CKSUM
    freenas-boot  ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada0p2  ONLINE       0     0     0
        ada1p2  ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 05:08:32 with 0 errors on Sun Feb  2 05:08:35 2020
config:

    NAME                                            STATE     READ WRITE CKSUM
    tank                                            ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/2e48e04a-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/2eff3431-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/2fad6079-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/305f9785-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/310fd248-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/31c62952-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/32845d1c-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/3338ea10-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
    cache
      gptid/3385f967-d2f0-11e6-8e60-0cc47a84a594    ONLINE       0     0     0

errors: No known data errors
 
Last edited:

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
Ok I'll answer my own question. I'll first put this post as a reference since it really helped despite the very humorous tone of the discussion: https://www.ixsystems.com/community...gptid-instead-of-da1p2-in-zpool-status.18441/

GPTIDs are something internal to Freenas -- or to simplify -- FreeNAS adds them. So basically you need to do everything through the GUI and not the CLI or the FreeNAS middleware will not be aware.

So specifically for the GUI
Storage->Pools->Extend

Screen Shot 2020-02-07 at 2.01.45 PM.png


The Empty Drive should be listed Under Available Disk
Push the button at the bottom and select Add Spare.
Now the really confusing part in the GUI is you want to add the available drive to only the Spare VDev. and not the DataVDev.
So In order to do this you'll need to mess around with hitting the blue arrows and select the Left to Right for the Spare VDev and UNSELECT the Left to Right Arrow for the Data VDev.

Once you've gotten things setup, hit the Extend button at the bottom. There will be a warning about erasing the drive but keep the data in the original pool.

Once done:
Code:
# zpool status
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:02:35 with 0 errors on Tue Feb  4 03:47:35 2020
config:

    NAME        STATE     READ WRITE CKSUM
    freenas-boot  ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada0p2  ONLINE       0     0     0
        ada1p2  ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 05:08:32 with 0 errors on Sun Feb  2 05:08:35 2020
config:

    NAME                                            STATE     READ WRITE CKSUM
    tank                                            ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/2e48e04a-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/2eff3431-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/2fad6079-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/305f9785-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/310fd248-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/31c62952-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/32845d1c-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
        gptid/3338ea10-d2f0-11e6-8e60-0cc47a84a594  ONLINE       0     0     0
    cache
      gptid/3385f967-d2f0-11e6-8e60-0cc47a84a594    ONLINE       0     0     0
    spares
      gptid/a203d5ad-49e3-11ea-9739-0cc47a84a594    AVAIL

errors: No known data errors


Now a couple of shoutouts for some scripts I found during the process. (I didn't create any of these scripts and do not support them, however I really found them useful when trying to hunt down devIDs, gptIDs, Serial Numbers of drives, etc.

The first is entitled lsblk.sh which gives output like the following:
Screen Shot 2020-02-07 at 2.10.22 PM.png


The second I call serial_num_drive.sh which gives output similar to the following:
Screen Shot 2020-02-07 at 2.12.42 PM.png


I list these here since they might help find out which drive in your case is associated with the devID (ie da0, etc).
Code:
#! /bin/sh

# Copyright (c) 2016-2019 Slawomir Wojciech Wojtczak (vermaden)
# All rights reserved.
#
# THIS SOFTWARE USES FREEBSD LICENSE (ALSO KNOWN AS 2-CLAUSE BSD LICENSE)
# https://www.freebsd.org/copyright/freebsd-license.html
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that following conditions are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS 'AS IS' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# ------------------------------
# REPLACEMENT FOR LINUX lsblk(8)
# ------------------------------
# vermaden [AT] interia [DOT] pl
# https://vermaden.wordpress.com

PATH=${PATH}:/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin

# DISPLAY HELP/USAGE/EXAMPLES
__usage() {
  local NAME="${0##*/}"
  cat << __EOF
usage:
  BASIC USAGE INFORMATION
  =======================
  # ${NAME} [DISK]
example(s):
  LIST ALL BLOCK DEVICES IN SYSTEM
  --------------------------------
  # ${NAME}
  DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
  ada0             0:92  932G GPT                           - -
    ada0p1         0:100 200M efi                    efiboot0 -
    ada0p2         0:101 512K freebsd-boot           gptboot0 -
    <FREE>         -:-   492K -                             - -
    ada0p3         0:102 931G freebsd-zfs                zfs0 <ZFS>
    ada0p3.eli     0:106 931G freebsd-zfs                   - <ZFS>
  LIST ONLY da1 BLOCK DEVICE
  --------------------------
  # ${NAME} da1
  DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
  da1              0:80  2.0G MBR                           - -
    da1s1          0:80  2.0G freebsd                       - -
      da1s1a       0:81  1.0G freebsd-ufs                root /
      da1s1b       0:82  1.0G freebsd-swap               swap SWAP
hint(s):
  DISPLAY PHYSICAL DISKS
  ----------------------
  # sysctl kern.disks
  kern.disks: ada0 da0 da1
  DISPLAY MEMORY BACKED DISKS
  ---------------------------
  # mdconfig -l
__EOF
  exit 1
}
# __usage() ENDED

# GET MAJOR/MINOR NUMBERS
__major_minor() { # 1=DEV
  local DEV=${1}
  MAJ=$( stat -f "%Hr" /dev/${DEV} )
  MIN=$( stat -f "%Lr" /dev/${DEV} )
}
# __major_minor() ENDED

# DETECT IF SWAP IS BEING USED
__swap_mount() { # 1=DEV
  local DEV="${1}"
  SWAP_FOUND=0

  # DETECT SWAP BY RAW DEVICE
  if swapctl -l | grep -q -- ${DEV} 2> /dev/null
  then
    TYPE=freebsd-swap
    MOUNT=SWAP
    LABEL="-"
    SWAP_FOUND=1
    return
  fi

  # DETECT SWAP BY LABEL
  while read LABELER GARBAGE PROVIDER
  do
    if [ "${DEV}" = "${PROVIDER}" ]
    then
      if swapctl -l | grep -q -- ${LABELER} 2> /dev/null
      then
        TYPE=freebsd-swap
        LABEL=${LABELER}
        MOUNT=SWAP
        SWAP_FOUND=1
        break
      fi
    fi
  done << __EOF
    $( echo "${GLABEL}" )
__EOF
}
# __swap_mount() ENDED

# DETECT TYPE WITH fstyp(8) TOOL
__type() { # 1=DEV
  local DEV=${1}
  TYPE=$( fstyp -u /dev/${DEV} 2> /dev/null )
  [ "${TYPE}" = "" ] && __swap_mount ${DEV}
  [ "${TYPE}" = "" ] && TYPE="-"
}
# __type() ENDED

# DETECT MOUNT/LABEL IF POSSIBLE
__mount_label() { # 1=TARGET
  local TARGET="${1}"
  local MOUNT_FOUND=0
  local LABEL_FOUND=0
  LABEL="-"
  if [ "${SWAP_FOUND}" = "1" ]
  then
    local MOUNT_FOUND=1
  else
    MOUNT="-"
  fi

  # CHECK IF DEVICE EXISTS
  [ ! -e /dev/${TARGET} ] && return

  # TRY CLASSIC MOUNT POINT WITH DEVICE NAME
  if [ "${MOUNT_FOUND}" != "1" ]
  then
    MOUNT=$( mount | grep "/dev/${TARGET} " | awk 'END{print $3}' )
    if [ "${MOUNT}" = "" ]
    then
      MOUNT="-"
    else
      local MOUNT_FOUND=1
    fi
  fi

  # GET LABEL/MOUNT FOR UFS/zfs(8)/msdosfs(8) FILESYSTEM
  case ${TYPE} in
    (freebsd-ufs)
      LABEL=$( tunefs -p /dev/${TARGET} 2>&1 | awk '/volume label/ {print $5}' )
      if [ "${LABEL}" = "" ]
      then
        LABEL="-"
      else
        LABEL="ufs/${LABEL}"
        local LABEL_FOUND=1
      fi
      ;;
    (freebsd-zfs)
      # zfs(8) IS NEVER MOUNTED BY RAW DEVICE NAME
      # SO NO NEED TO CHECK IF MOUNT_FOUND = 1 HERE
      MOUNT="<ZFS>"
      local MOUNT_FOUND=1
      ;;
    (msdosfs)
      LABEL=$( file -s /dev/${DEV} | tr ',' '\n' | awk -F '"' '/label:/ {print $2}' )
      if [ "${LABEL}" = "" ]
      then
        LABEL="-"
      else
        LABEL="msdosfs/${LABEL}"
        local LABEL_FOUND=1
      fi
      ;;
    (exfat)
      if which exfatlabel 1> /dev/null 2> /dev/null
      then
        LABEL=$( exfatlabel /dev/${DEV} 2> /dev/null )
        if [ "${LABEL}" = "" ]
        then
          LABEL="-"
        else
          LABEL="exfat/${LABEL}"
          local LABEL_FOUND=1
        fi
      fi
      ;;
  esac

  # GET LABEL USING glabel(8)
  if [ "${LABEL_FOUND}" != "1" ]
  then
    # TRY GLABEL MOUNTPOINT AS msdosfs MOUNT IS NOT POSSIBLE WITHOUT GPT/MBR
    local GLABEL_GREP=$( echo "${GLABEL}" | grep -v ufsid | grep "${TARGET}\$" )
    if [ "${GLABEL_GREP}" = "" ]
    then
      LABEL="-"
    else
      while read PROVIDER STATUS DEVICE
      do
        LABEL=$( echo "${GLABEL_GREP}" | grep -m 1 "${TARGET}"  | awk '{print $1}' )
        local LABEL_FOUND=1
        break
      done << ______EOF
        $( echo "${GLABEL_GREP}" )
______EOF
    fi
  fi

  # GET MOUNT USING glabel(8)
  if [ "${MOUNT_FOUND}" != "1" ]
  then
    # TRY GLABEL MOUNTPOINT AS msdosfs MOUNT IS NOT POSSIBLE WITHOUT GPT/MBR
    local GLABEL_GREP=$( echo "${GLABEL}" | grep -v ufsid | grep "${TARGET}\$" )
    if [ "${GLABEL_GREP}" != "" ]
    then
      while read PROVIDER STATUS DEVICE
      do
        MOUNT=$( mount | grep "${PROVIDER}" | awk 'END{print $3}' )
        if [ "${MOUNT}" = "" ]
        then
          MOUNT="-"
        else
          local MOUNT_FOUND=1
        fi
        break
      done << ______EOF
        $( echo "${GLABEL_GREP}" )
______EOF
    fi
  fi

  # GET MOUNT FROM fuse(8)
  if [ "${MOUNT_FOUND}" != "1" ]
  then
    local FUSE_PIDS=$( pgrep mount.exfat ntfs-3g | tr '\n' ',' | sed '$s/.$//' )
    if [ "${FUSE_PIDS}" != "" ]
    then
      local FUSE_MOUNTS=$( ps -p "${FUSE_PIDS}" -o command | sed 1d | sort -u )
      MOUNT=$( echo "${FUSE_MOUNTS}" | grep "/dev/${TARGET} " | awk '{print $3}' )
      if [ "${MOUNT}" = "" ]
      then
        MOUNT="-"
      else
        local MOUNT_FOUND=1
      fi
    fi
  fi

  # GET LABEL FROM gpart(8)
  if [ "${LABEL_FOUND}" != "1" ]
  then
    LABEL=$( gpart show -p -l ${DEV} 2> /dev/null | sed 's|=>||g' | sed 's-\[active\]--g' | grep "${TARGET} " | awk '{print $4}' )
    if [ "${LABEL}" = "" -o "${LABEL}" = "(null)" ]
    then
      LABEL="-"
    else
      LABEL="gpt/${LABEL}"
    fi
  fi
  SWAP_FOUND=0
}
# __mount_label() ENDED

# LIST BLOCK DEVICES
__list_block_devices() {
  # FIRST 1000 DEVICES OF EACH CLASS SHOULD DO
  for I in ada da mmcsd md
  do
    ls -1 /dev/${I}[0-9]           2> /dev/null
    ls -1 /dev/${I}[0-9][0-9]      2> /dev/null
    ls -1 /dev/${I}[0-9][0-9][0-9] 2> /dev/null
  done \
    | sed 's|/dev/||g' \
    | sort -u \
    | sed '/^s*$/d'
}
# __list_block_devices() ENDED

# PRINT DEVICES AND PARTITIONS
__print_parts() { # 1=NAME 2=TYPE 3=SIZE 4=SIZE_FREE
  local NAME=${1}
  local TYPE=${2}
  local SIZE=${3}
  local SIZE_FREE=${4}

  case ${TYPE} in
    (free)
      # REMOVE BRACKETS FROM SIZE
      local SIZE_FREE=$( echo ${SIZE_FREE} | tr -d '()' )
      # FIRST 16 SECTORS ARE ALWAYS EMPTY ON bsdlabel(8) PARTITION SCHEMA
      [ "${SIZE_FREE}" = "8.0K" ] && return
      # OUTPUT
      printf "${FORMAT}" "<FREE>" - - ${SIZE_FREE} "-" ${LABEL} ${MOUNT}
      ;;
    (*)
      # NEEDED FOR bsdlabel(8) ON WHOLE DISK WITHOUT PARTITION
      # WORKAROUND FOR gpart(8) BUG 241004
      # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241004
      if [ "${TYPE}" = "!0" ]
      then
        local TYPE=freebsd-ufs
      fi

      # REMOVE BRACKETS FROM SIZE
      local SIZE=$( echo ${SIZE} | tr -d '()' )
      [ "${LABEL}" = "" ] && local LABEL="-"
      [ "${MOUNT}" = "" ] && local MOUNT="-"
      __major_minor ${NAME}
      printf "${FORMAT}" ${NAME} ${MAJ} ${MIN} ${SIZE} ${TYPE} ${LABEL} ${MOUNT}
      if [ -e /dev/${NAME}.eli ]
      then
        # GET GELI MOUNT/LABEL/MAJOR/MINOR
        __major_minor ${NAME}.eli
        __type ${NAME}.eli
        __mount_label ${NAME}.eli
        printf "${FORMAT}" ${NAME}.eli ${MAJ} ${MIN} ${SIZE} ${TYPE} ${LABEL} ${MOUNT}
        MOUNT="-"
      fi
      ;;
  esac
}
# __print_parts() ENDED

# WHEN GPART OUTPUT EXISTS (partition table / not entire device)
__gpart_present() { # 1=DEV
  local DEV=${1}
  local TYPE="-"
  local SIZE="-"
  local SIZE_FREE="-"
  # CHECK IF DEVICE EXISTS
  [ ! -e /dev/${DEV} ] && return

  # WORKAROUND FOR gpart(1) BUG 240998
  # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240998
  TYPE_EXFAT_HELPER_DEVICE=$( gpart show -p ${DEV} | head -1 | sed 's-\[active\]--g' | awk '{print $5}' 2> /dev/null )
  if [ "${TYPE_EXFAT_HELPER_DEVICE}" = "MBR" ]
  then
    TYPE_EXFAT_HELPER=$( fstyp -u /dev/${DEV} 2> /dev/null )
    if [ "${TYPE_EXFAT_HELPER}" = "exfat" ]
    then
      TYPE=exfat
      SIZE=$( diskinfo -v ${DEV} 2> /dev/null| awk '/mediasize in bytes/ {print $NF}' | tr -d '()' )
      __print_parts ${DEV} ${TYPE} ${SIZE} -
      continue
    fi
  fi

  # READ PARTITIONS OF PROVIDER
  local GPART=$( gpart show -p ${DEV} 2> /dev/null | sed 's-\[active\]--g' | sed 's|=>||g' )
  # PARSE gpart(8) OUTPUT
  echo "${GPART}" \
    | while read BEG END NAME TYPE SIZE SIZE_FREE
      do
        # VISITED ONLY ONCE FOR WHOLE DEVICE
        if [ ${LEVEL_DEV} = "0" ]
        then
          # WHOLE DEVICE
          LEVEL_DEV=1
          FORMAT="${FORMAT_L0}"
          __print_parts ${NAME} ${TYPE} ${SIZE} ${SIZE_FREE}
        else
          # PARTITION
          FORMAT="${FORMAT_L1}"
          # READ PARTITIONS OF PROVIDER
          local GPART_PARTS=$( gpart show -p ${NAME} 2> /dev/null | sed 's-\[active\]--g' | sed 's|=>||g' )
          # PARSE gpart(8) OUTPUT
          if [ "${GPART_PARTS}" != "" ]
          then
            # PARTITION IN PARTITION
            [ ! -e /dev/${NAME} ] && return
            # PARSE DEVICE
            echo "${GPART_PARTS}" \
              | while read BEG END NAMEONE TYPE SIZE SIZE_FREE
                do
                  # VISITED ONLY ONCE FOR WHOLE DEVICE
                  if [ ${LEVEL_PAR} = "0" ]
                  then
                    # WHOLE DEVICE
                    LEVEL_PAR=1
                    FORMAT="${FORMAT_L1}"
                  else
                    # PARTITION
                    FORMAT="${FORMAT_L2}"
                  fi
                  # CHECK IF SWAP IS ENABLED FOR SWAP DEVICES
                  [ "${TYPE}" = "freebsd-swap" ] && __swap_mount ${NAMEONE}
                  __mount_label ${NAMEONE}
                  __print_parts ${NAMEONE} ${TYPE} ${SIZE} ${SIZE_FREE}
                done
          else
            # FILESYSTEM
            [ "${TYPE}" = "freebsd-swap" ] && __swap_mount ${DEV}
            __mount_label ${NAME}
            __print_parts ${NAME} ${TYPE} ${SIZE} ${SIZE_FREE}
          fi
        fi
      done
}
# __gpart_present() ENDED

# WHEN GPART OUTPUT DO NOT EXISTS (no partition table / entire device)
__gpart_absent() { # 1=DEV
  local DEV=${1}
  # EXIT IF DEVICE DOES NOT EXISTS
  [ ! -e /dev/${DEV} ] && return
  local SIZE=$( diskinfo -v ${DEV} 2> /dev/null | awk '/mediasize in bytes/ {print $NF}' | tr -d '()' )
  local SIZE_FREE="-"
  __type ${DEV}
  __mount_label ${DEV}
  [ "${SIZE}"  = "" ] && local SIZE="-"
  [ "${TYPE}"  = "" ] && local TYPE="-"
  [ "${LABEL}" = "" ] && local LABEL="-"
  [ "${MOUNT}" = "" ] && local MOUNT="-"
  __print_parts ${DEV} ${TYPE} ${SIZE} ${SIZE_FREE}
}
# __gpart_absent() ENDED

# PARSE ARGUMENTS IF THERE ARE ANY
LEVEL_DEV=0
LEVEL_PAR=0
GLABEL=$( glabel status | sed 1d )
DISKS=$( __list_block_devices )
case ${#} in
  (0)
    # LIST ALL DISKS
    :
    ;;
  (1)
    # SINGLE ARGUMENT MEANS SINGLE DISK OR HELP
    case ${1} in
      (h|-h|--h|help|-help|--help)
        __usage
        ;;
    esac
    # SINGLE DISK CHECK
    if echo "${DISKS}" | grep -q -- "${1}"
    then
      DISKS="${1}"
    else
      echo "NOPE: disk '${1}' does not exist in the system"
      echo
      __usage
    fi
    ;;
  (*)
    # ONLY '0' and '1' ARGUMENTS ARE COVERED
    __usage
    ;;
esac

# SET OUTPUT FORMAT AND PRINT HEADER
    FORMAT_L0="%-14s %3s:%-3s %4s %-14s %20s %s\n"
  FORMAT_L1="  %-12s %3s:%-3s %4s %-14s %20s %s\n"
FORMAT_L2="    %-10s %3s:%-3s %4s %-14s %20s %s\n"
FORMAT="${FORMAT_L0}"
printf "${FORMAT}" DEVICE MAJ MIN SIZE TYPE LABEL MOUNT

# DO THE JOB
echo "${DISKS}" \
  | while read DEVICE
    do
      if gpart show ${DEVICE} 1> /dev/null 2> /dev/null
      then
        __gpart_present ${DEVICE}
      else
        __gpart_absent ${DEVICE}
      fi
    done


Code:
#!/bin/bash

#Use smartctl to get Serial Number - else dmesg is used
USESMART=2
#save glabel status to temporary file
CACHEGLABEL=0

if  [ $CACHEGLABEL -eq 0 ]
then
GLCMD="glabel status"
else
GLTMP=/var/tmp/$0.glabel
glabel status > $GLTMP
GLCMD="cat $GLTMP"
fi

ADALOW=`ls /dev/ada[0-9] 2>/dev/null`
ADAHIGH=`ls /dev/ada[0-9][0-9] 2>/dev/null`
DALOW=`ls /dev/da[0-9] 2>/dev/null`
DAHIGH=`ls /dev/da[0-9][0-9] 2>/dev/null`
#check if all device nodes exist or skip
if  [[ $ADALOW == *ls* ]]
then
$DALOW=
fi
if  [[ $ADAHIGH == *ls* ]]
then
$ADAHIGH=
fi
if  [[ $DALOW == *ls* ]]
then
$DALOW=
fi
if  [[ $DAHIGH == *ls* ]]
then
$DAHIGH=
fi
for FILE in $ADALOW $ADAHIGH $DALOW $DAHIGH
do
DEV=${FILE##'/dev/'}
#echo -n "${DEV}: "
if  [ $USESMART -eq 0 ]
then
SERIAL=`grep $DEV: /var/log/dmesg.today |grep -i Serial | awk '{print $(NF)}'`
elif [ $USESMART -eq 1 ]
then
SERIAL=`smartctl -a $FILE | grep -i 'Serial Number'| awk '{print $(NF)}'`
else
SERIAL=$(smartctl -a $FILE | grep -i 'serial number' | awk '{print $(NF)}')
fi
#this skips all ufs drives
#GPTID=`$GLCMD |grep 2$ |grep ${DEV}p|cut -d' ' -f1`
GPTID=$($GLCMD | awk -v pat="^${DEV}" '$3~pat{print $1, $3"\n"}')
if [ "${GPTID}x" == 'x' ]
then
GPTID="No GPTID"
fi
if [ "${SERIAL}x" == 'x' ]
then
$SERIAL="Not found"
fi
echo  ${DEV}: Serial $SERIAL \; GPTID=$GPTID
done

if  [ $CACHEGLABEL -eq 1 ]
then
rm $GLTMP
fi
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
It would probably clarify if your question were "What is best way to add hot spare" rather than "hot swap"--I was wondering why you thought it mattered that the prospective disk was in a hot-swap bay (because it certainly doesn't matter to FreeNAS). But in short, yes, any manipulation of the pool should be done through the GUI. As a bonus, that removes the need for gptid-to-device name mapping, since the GUI doesn't display the gptids. It doesn't obviate the need for physical device-to-device name mapping, but in the context you're dealing with, it only shows devices that aren't already in use.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35
I find it sometimes really confusing the way FreeNAS does things in situations like these. Oftentimes I'm looking up the Oracle documentation and what I'm seeing in FreeNAS isn't the same. In this situation for example the gptid-to-device name mapping. I couldn't for a long time figure out how a device was assigned a gptid. Perhaps finer points like this should be elucidated in the FreeNAS documentation. Frankly I find the FreeNAS documentation severely lacking in areas like this -- as compared to like an Arch Wiki which is specific down to many points. I'm sure the designers have made choices how to implement things within FreeNAS for good reasons, however it would be great if they would provide better instructions how to deal with certain situations. I'm aware the usual recommendation is to use the GUI so the middleware is aware of the changes. The problem is however it's often a combination of using the CLI and middleware since the GUI doesn't reveal things or a lot of properties.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
Perhaps finer points like this should be elucidated in the FreeNAS documentation.
Maybe, but you never need to know about gptids when using the GUI, and there just wasn't any reason to not use the GUI for this task. This isn't a difference between Oracle ZFS and OpenZFS; it's a question of how devices are designated (the Oracle docs don't talk about using da*/ada* devices either).
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35 - You are correct that all I needed to use was the GUI for this task. However the only way I discovered FreeNAS was using gptid's was the CLI, not the documentation.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
However the only way I discovered FreeNAS was using gptid's was the CLI
But why was it important for you to know that? Really, I'm not trying to be difficult, but if you're working with the GUI (as FreeNAS intends), that just isn't information you need. It's hard to read and complicates matters, and the GUI transparently maps the IDs to device IDs (da*/ada*), so use the GUI and don't worry about what's going on under the hood. Now, if you're like a lot of us here and want to understand what's going on under the hood anyway, good for you, but the manual isn't going to answer those questions--that simply isn't its purpose.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35 -- Its not particularly important in this scenario, however when looking up how to add a hot spare, the first thing that pops up in google is the oracle docs. In fact with effort I found Storage->Disks only after looking at the Oracle docs and realizing the examples I was seeing didn't correspond to the information I was getting on the FreeNAS CLI. In terms of why I need to know GPT -- I don't, but I do unfortunately need to work from the CLI for example to do zfs send/receive commands for certain zvols. After I do this manually, the zvols will show up in the GUI menu under storage->pools. I'm not trying to mix and match arguments here -- I could see on the one hand you're saying to yourself -- 1st this guy is talking about GPTs, and then out of the blue he's talking on point #2) Zvols. My point is that the GUI can't manage everything. Somethings you have to manage from the CLI. If going CLI however you actually need to know some parameters FreeNAS is working with since they in some cases differ from "general ZFS" commands. Documentation on these parameters in many cases is missing or I'll rephrase obscure. You find talk of them in various internet posts, these forums, reddit, for example. It would be great if the official documentation would even hint of some of these parameters -- similar to an Arch Wiki -- but it does not. In fact to my knowledge -- which I could be wrong -- I don't think the documentation can be community appended or accepts submissions from the community for review about topics that might require more knowledge of what's going on behind the scenes. I'm aware the "behind the scenes" could be a potential landmine, however other projects like the Arch Wiki somehow in my opinion seem to be able to strike this balance.

I hope this wasn't perceived as a rant or being overly critical. I think FreeNAS is great as a project, however I do think some of their documentation could be more comprehensive. (For example -- your acme.sh script to add Lets Encrypt certs into the middleware -- you're post is wonderful -- official documentation -- information isn't even listed).
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
I don't think the documentation can be community appended or accepts submissions from the community for review about topics that might require more knowledge of what's going on behind the scenes.
No, the docs aren't a wiki, but you can submit tickets against the docs which may result in updates. If you want a wiki, well, someone on this thread hosts one...

For example -- your acme.sh script to add Lets Encrypt certs into the middleware -- you're post is wonderful -- official documentation -- information isn't even listed
No, and I wouldn't expect it would be, any more than any of the community-generated documentation on setting up certain apps in jails. But it is in an "official" place for community docs, which is the Resources section of this very forum.
 
Top