TrueNAS-13.0-U3.1 unable to Offline a disk

[denis4o]

I have a 4 x 2TB drive pool which I want to replace with 4 x 8 TB drives. I have no spare SATA ports so I am planning to offline one disk at a time, replace, reslver.
However, when I choose the offline option on any of my disks from the UI, nothing happens. I am not getting an error, however, the disk is not offlined either.

Here is the pool status:

Code:

zpool status
  pool: freenas-boot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:01:57 with 0 errors on Sun Jan  8 03:46:57 2023
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2       ONLINE       0     0     0

errors: No known data errors

  pool: home_server
 state: ONLINE
  scan: resilvered 216K in 00:00:01 with 0 errors on Thu Jan 12 10:17:10 2023
config:

        NAME                                            STATE     READ WRITE CKSUM
        home_server                                     ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/e8de723b-1374-11e9-8a9c-7085c25f11d1  ONLINE       0     0     0
            ada0p2                                      ONLINE       0     0     0
            ada2p2                                      ONLINE       0     0     0
            ada3p2                                      ONLINE       0     0     0

 

[denis4o]

Code:

zpool list -v
NAME                                             SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
freenas-boot                                      14G  5.66G  8.34G        -         -      -    40%  1.00x    ONLINE  -
  da0p2                                         14.1G  5.66G  8.34G        -         -      -  40.4%      -    ONLINE
home_server                                     7.25T  5.88T  1.37T        -         -    26%    81%  1.00x    ONLINE  /mnt
  raidz1-0                                      7.25T  5.88T  1.37T        -         -    26%  81.1%      -    ONLINE
    gptid/e8de723b-1374-11e9-8a9c-7085c25f11d1  1.82T      -      -        -         -      -      -      -    ONLINE
    ada0p2                                      1.82T      -      -        -         -      -      -      -    ONLINE
    ada2p2                                      1.82T      -      -        -         -      -      -      -    ONLINE
    ada3p2                                      1.82T      -      -        -         -      -      -      -    ONLINE

 

[sretalla]

does it work if you run it yourself in CLI?

zpool offline home_server gptid/e8de723b-1374-11e9-8a9c-7085c25f11d1

 

[denis4o]

Thanks @sretalla that worked. I removed the disk and replaced it with the new one, however, in the UI, now I don't see it as a member disk:

 

[denis4o]

Edit: I shut down the server and the disk showed up. It is resilvering now. I am not sure why the hot-swap didn't work.

 

[sretalla]

I don't have the time right now to go searching, but there is a command that can be run to rescan for disks... maybe something with camcontrol

Perhaps your system (SATA controller or backplane) doesn't (fully) support hot-plug.

 

[denis4o]

Could be the case.

The saga continues however. The first disk was replaced successfully. The pool got resilvered very fast (I'm impressed by the WD Red Plus performance).

Code:

zpool list -v
NAME                                             SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
freenas-boot                                      14G  5.66G  8.34G        -         -      -    40%  1.00x    ONLINE  -
  da0p2                                         14.1G  5.66G  8.34G        -         -      -  40.4%      -    ONLINE
home_server                                     7.25T  5.88T  1.37T        -         -    26%    81%  1.00x    ONLINE  /mnt
  raidz1-0                                      7.25T  5.88T  1.37T        -         -    26%  81.1%      -    ONLINE
    gptid/fb50dea3-9265-11ed-be0b-7085c25f11d1  7.28T      -      -        -         -      -      -      -    ONLINE
    ada0p2                                      1.82T      -      -        -         -      -      -      -    ONLINE
    ada2p2                                      1.82T      -      -        -         -      -      -      -    ONLINE
    ada3p2                                      1.82T      -      -        -         -      -      -      -    ONLINE

Now not only that I can't OFFLINE any of the remaining disks via the GUI, but I also can't do that from the CLI:

Code:

  pool: home_server
 state: ONLINE
  scan: resilvered 120K in 00:00:01 with 0 errors on Thu Jan 12 18:11:38 2023
config:

        NAME                                            STATE     READ WRITE CKSUM
        home_server                                     ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/fb50dea3-9265-11ed-be0b-7085c25f11d1  ONLINE       0     0     0
            ada0p2                                      ONLINE       0     0     0
            ada2p2                                      ONLINE       0     0     0
            ada3p2                                      ONLINE       0     0     0

errors: No known data errors


root@freenas:~ # zpool offline home_server ada0p2


root@freenas:~ # zpool status
  pool: freenas-boot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:01:57 with 0 errors on Sun Jan  8 03:46:57 2023
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2       ONLINE       0     0     0

errors: No known data errors

  pool: home_server
 state: ONLINE
  scan: resilvered 120K in 00:00:01 with 0 errors on Thu Jan 12 18:11:38 2023
config:

        NAME                                            STATE     READ WRITE CKSUM
        home_server                                     ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/fb50dea3-9265-11ed-be0b-7085c25f11d1  ONLINE       0     0     0
            ada0p2                                      ONLINE       0     0     0
            ada2p2                                      ONLINE       0     0     0
            ada3p2                                      ONLINE       0     0     0

errors: No known data errors

Any ideas?

 

[Redcoat]

Any ideas?

zpool offline home_server ada0

 

[sretalla]

Or possibly you need to specify /dev/ada0p2

 

[denis4o]

Unfortunately nothing from what I tried worked - ada0, /dev/ada0p2, /ada0p2:

Code:

zpool offline home_server ada0
cannot offline ada0: no such device in pool

Code:

root@freenas:~ # zpool offline home_server /dev/ada0p2
root@freenas:~ # zpool status
  pool: freenas-boot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:01:57 with 0 errors on Sun Jan  8 03:46:57 2023
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2       ONLINE       0     0     0

errors: No known data errors

  pool: home_server
 state: ONLINE
  scan: resilvered 120K in 00:00:01 with 0 errors on Thu Jan 12 18:39:59 2023
config:

        NAME                                            STATE     READ WRITE CKSUM
        home_server                                     ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/fb50dea3-9265-11ed-be0b-7085c25f11d1  ONLINE       0     0     0
            ada0p2                                      ONLINE       0     0     0
            ada2p2                                      ONLINE       0     0     0
            ada3p2                                      ONLINE       0     0     0

errors: No known data errors

I'm running out of ideas here ... it doesn't error out but it doesn't offline the disks either ...

 

[sretalla]

OK, so get the gptid from glabel status

 

[denis4o]

Seems like an issue with the gptid not being correctly added to the pool as described here: https://www.truenas.com/community/t...ows-device-id-da0-only-truenas-12-0-u1.90124/

My glabel status shows the following:

Code:

glabel status
                                      Name  Status  Components
gptid/fb50dea3-9265-11ed-be0b-7085c25f11d1     N/A  ada1p2
gptid/d6b7c845-4a66-11e8-841c-30f9edbae477     N/A  da0p1
                              label/efibsd     N/A  da0p1

Where da0 is my boot device, so none of the other 3 drives show up

 

[sretalla]

You can get the gptid from rawuuid of partition 2 from gpart list /dev/ada0

 

[denis4o]

Code:

gpart list /dev/ada0
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 3907029127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,9e51ede7-2030-11ea-b2ff-7085c25f11d1,0x80,0x400000)
   rawuuid: 9e51ede7-2030-11ea-b2ff-7085c25f11d1
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada0p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(2,GPT,a799872b-2030-11ea-b2ff-7085c25f11d1,0x400080,0xe8a08808)
   rawuuid: a799872b-2030-11ea-b2ff-7085c25f11d1
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: ada0
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e4

Code:

root@freenas:~ # zpool offline home_server a799872b-2030-11ea-b2ff-7085c25f11d1
cannot offline a799872b-2030-11ea-b2ff-7085c25f11d1: no such device in pool
root@freenas:~ # zpool offline home_server gptid/a799872b-2030-11ea-b2ff-7085c25f11d1
cannot offline gptid/a799872b-2030-11ea-b2ff-7085c25f11d1: no such device in pool
root@freenas:~ # zpool offline home_server /dev/gptid/a799872b-2030-11ea-b2ff-7085c25f11d1
cannot offline /dev/gptid/a799872b-2030-11ea-b2ff-7085c25f11d1: no such device in pool