After boot-pool upgrade: (...) features are enabled (...) despite not being requested by the 'compatibility' property

CoolWolf

Dabbler
Joined
Mar 2, 2023
Messages
10
Hello Community!
After staying for quite a time on Bluefin 22.12.4, I finally upgraded yesterday to Cobia 23.10.1. A zpool status suggested me to possible upgrade my pools to pickup new features .... well ... should I have done so? Maybe not. Have I done so? Yes I did ...

All went "ok" for my 4 Storage Pools and also for my boot-pool (no error message or similar seen), but latter one shows now following status:

Code:
admin@FatBoy[~]$ sudo zpool status boot-pool
  pool: boot-pool
 state: ONLINE
status: One or more features are enabled on the pool despite not being
    requested by the 'compatibility' property.
action: Consider setting 'compatibility' to an appropriate value, or
    adding needed features to the relevant file in
    /etc/zfs/compatibility.d or /usr/share/zfs/compatibility.d.
  scan: scrub repaired 0B in 00:00:31 with 0 errors on Fri Dec 22 03:45:32 2023
config:

    NAME        STATE     READ WRITE CKSUM
    boot-pool   ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        sdc3    ONLINE       0     0     0
        sdl3    ONLINE       0     0     0

errors: No known data errors


Pool compatibility shows:
Code:
admin@FatBoy[~]$ sudo zpool get compatibility
NAME           PROPERTY       VALUE          SOURCE
BigTank        compatibility  off            default
FastTank       compatibility  off            default
SmallTank      compatibility  off            default
SuperFastTank  compatibility  off            default
boot-pool      compatibility  grub2          local


The "features" in /usr/share/zfs/compatibility.d/grub2 are listed as:
Code:
admin@FatBoy[~]$ cat /usr/share/zfs/compatibility.d/grub2
# Features which are supported by GRUB2
allocation_classes
async_destroy
block_cloning
bookmarks
device_rebuild
embedded_data
empty_bpobj
enabled_txg
extensible_dataset
filesystem_limits
hole_birth
large_blocks
livelist
log_spacemap
lz4_compress
project_quota
resilver_defer
spacemap_histogram
spacemap_v2
userobj_accounting
zilsaxattr
zpool_checkpoint


boot-pool features are:
Code:
admin@FatBoy[~]$ sudo zpool get all boot-pool           
NAME       PROPERTY                       VALUE                          SOURCE
boot-pool  size                           216G                           -
boot-pool  capacity                       4%                             -
boot-pool  altroot                        -                              default
boot-pool  health                         ONLINE                         -
boot-pool  guid                           15503437371621097426           -
boot-pool  version                        -                              default
boot-pool  bootfs                         boot-pool/ROOT/23.10.1         local
boot-pool  delegation                     on                             default
boot-pool  autoreplace                    off                            default
boot-pool  cachefile                      -                              default
boot-pool  failmode                       wait                           default
boot-pool  listsnapshots                  off                            default
boot-pool  autoexpand                     off                            default
boot-pool  dedupratio                     1.00x                          -
boot-pool  free                           207G                           -
boot-pool  allocated                      9.26G                          -
boot-pool  readonly                       off                            -
boot-pool  ashift                         12                             local
boot-pool  comment                        -                              default
boot-pool  expandsize                     -                              -
boot-pool  freeing                        0                              -
boot-pool  fragmentation                  3%                             -
boot-pool  leaked                         0                              -
boot-pool  multihost                      off                            default
boot-pool  checkpoint                     -                              -
boot-pool  load_guid                      16544597345875423744           -
boot-pool  autotrim                       off                            default
boot-pool  compatibility                  grub2                          local
boot-pool  bcloneused                     0                              -
boot-pool  bclonesaved                    0                              -
boot-pool  bcloneratio                    1.00x                          -
boot-pool  feature@async_destroy          enabled                        local
boot-pool  feature@empty_bpobj            active                         local
boot-pool  feature@lz4_compress           active                         local
boot-pool  feature@multi_vdev_crash_dump  enabled                        local
boot-pool  feature@spacemap_histogram     active                         local
boot-pool  feature@enabled_txg            active                         local
boot-pool  feature@hole_birth             active                         local
boot-pool  feature@extensible_dataset     active                         local
boot-pool  feature@embedded_data          active                         local
boot-pool  feature@bookmarks              enabled                        local
boot-pool  feature@filesystem_limits      enabled                        local
boot-pool  feature@large_blocks           enabled                        local
boot-pool  feature@large_dnode            enabled                        local
boot-pool  feature@sha512                 enabled                        local
boot-pool  feature@skein                  enabled                        local
boot-pool  feature@edonr                  enabled                        local
boot-pool  feature@userobj_accounting     active                         local
boot-pool  feature@encryption             enabled                        local
boot-pool  feature@project_quota          active                         local
boot-pool  feature@device_removal         enabled                        local
boot-pool  feature@obsolete_counts        enabled                        local
boot-pool  feature@zpool_checkpoint       enabled                        local
boot-pool  feature@spacemap_v2            active                         local
boot-pool  feature@allocation_classes     enabled                        local
boot-pool  feature@resilver_defer         enabled                        local
boot-pool  feature@bookmark_v2            enabled                        local
boot-pool  feature@redaction_bookmarks    enabled                        local
boot-pool  feature@redacted_datasets      enabled                        local
boot-pool  feature@bookmark_written       enabled                        local
boot-pool  feature@log_spacemap           active                         local
boot-pool  feature@livelist               enabled                        local
boot-pool  feature@device_rebuild         enabled                        local
boot-pool  feature@zstd_compress          enabled                        local
boot-pool  feature@draid                  enabled                        local
boot-pool  feature@zilsaxattr             active                         local
boot-pool  feature@head_errlog            disabled                       local
boot-pool  feature@blake3                 disabled                       local
boot-pool  feature@block_cloning          enabled                        local
boot-pool  feature@vdev_zaps_v2           disabled                       local


Two questions:
1.) What would be considered as an appropriate compatibility value for the boot-pool?
- or -
2.) What feature(s) would be needed in /usr/share/zfs/compatibility.d/grub2 (if any)?

I am not a ZFS Expert and I have quite a vanilla TrueNAS installation - nothing special or weird settings.
Any help or suggestion is therefore highly appreciated.

THANKS and have a great day!
CoolWolf
 

tprelog

Patron
Joined
Mar 2, 2016
Messages
297
You should not be upgrading the boot-pool. It's my understanding, and a few posts on the forum suggest that will prevent TrueNAS from booting. If you have not rebooted yet, I suggest you back up your configuration. You will likely need to reinstall TrueNAS and restore your configuration to get back up and running.



 
Last edited:

CoolWolf

Dabbler
Joined
Mar 2, 2023
Messages
10
Thanks for your feedback @tprelog !

Sigh ... seems I am quite a "lucky" boy then as both of my systems showing this issue are rebooting. But I agree mabye take this as a lessons learned and start from scratch re-installing TrueNAS again and keep the fingers off the boot pools.
It is Xmas ... best time for doing so. Hohohoho

THANKS again and Merry Christmas!
 
Top