Pool Unavailable after Migration to Scale (but I can access it on Core)

[apfelcake]

Hello,

I can't import my Pool after migrating from TrueNAS Core to TrueNAS Scale.

Show : zpool import on Scale

Code:

# zpool import
   pool: apfelcake-home-pool
     id: 16058675992404351851
  state: UNAVAIL
status: One or more devices contains corrupted data.
 action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
 config:

    apfelcake-home-pool                       UNAVAIL  insufficient replicas
      raidz2-0                                UNAVAIL  insufficient replicas
        3989863d-bef4-11ec-8168-001517d6061e  UNAVAIL
        9283d139-7a9e-11eb-a8e0-b06ebf3732d8  UNAVAIL
        929d53a4-7a9e-11eb-a8e0-b06ebf3732d8  UNAVAIL
        92b8bda8-7a9e-11eb-a8e0-b06ebf3732d8  UNAVAIL
        86adcff9-4fbb-11ec-b633-b06ebf3732d8  UNAVAIL
        dfe5e425-825b-11eb-908e-b06ebf3732d8  UNAVAIL
        sdh2                                  ONLINE

Show : lsblk on SCALE

Code:

lsblk
NAME     MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda        8:0    0   2.7T  0 disk
sdb        8:16   0   2.7T  0 disk
sdc        8:32   0   2.7T  0 disk
sdd        8:48   0   2.7T  0 disk
sde        8:64   0   2.7T  0 disk
sdf        8:80   0   2.7T  0 disk
sdg        8:96   0 476.9G  0 disk
sdh        8:112  0   2.7T  0 disk
├─sdh1     8:113  0     2G  0 part
└─sdh2     8:114  0   2.7T  0 part
sdi        8:128  0   2.7T  0 disk
sdj        8:144  0  59.6G  0 disk
├─sdj1     8:145  0   260M  0 part
├─sdj2     8:146  0  43.4G  0 part
└─sdj3     8:147  0    16G  0 part
  └─sdj3 253:0    0    16G  0 crypt [SWAP]

Show : blkid on SCALE

Code:

blkid
/dev/sdh2: LABEL="apfelcake-home-pool" UUID="16058675992404351851" UUID_SUB="11960437850125860992" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="563f616d-7daa-11eb-aeac-b06ebf3732d8"
/dev/sdj1: SEC_TYPE="msdos" UUID="6D65-1806" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="c8754764-216e-11ed-885b-b06ebf3732d8"
/dev/sdj2: LABEL="boot-pool" UUID="4760369027261807198" UUID_SUB="10406371361967250506" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c8f2abab-216e-11ed-885b-b06ebf3732d8"
/dev/mapper/sdj3: UUID="392100f4-dc43-44a9-8625-2519261f784e" TYPE="swap"
/dev/sdh1: PARTUUID="562c819a-7daa-11eb-aeac-b06ebf3732d8"
/dev/sdj3: PARTUUID="c8e5a283-216e-11ed-885b-b06ebf3732d8"

The devices are marked as UNAVAILABLE. The disks are still visible, but they don't have a Pool assigned to. It seems like TrueNAS can't find the disks. The Pool is not GELI encrypted, but the Root Dataset is encrypted. I don't know if this is relevant.

I can go back to TrueNAS Core and import the Pool and everything is fine. Its interesting, that all the drives, marked as "UNAVAIL" in Scale, are Drives attached via an HBA SAS2008.

Is there a way to point the "missing" disks in the pool to the right Disks?

 

[apfelcake]

Does it has something to do with the label?

This is the output with the Mounted Pool in TrueNAS Core:

Code:

glabel status
                                      Name  Status  Components
gptid/c8754764-216e-11ed-885b-b06ebf3732d8     N/A  ada0p1
gptid/563f616d-7daa-11eb-aeac-b06ebf3732d8     N/A  ada1p2
gptid/92b8bda8-7a9e-11eb-a8e0-b06ebf3732d8     N/A  da1p2
gptid/929d53a4-7a9e-11eb-a8e0-b06ebf3732d8     N/A  da0p2
gptid/9283d139-7a9e-11eb-a8e0-b06ebf3732d8     N/A  da3p2
gptid/9156801c-7a9e-11eb-a8e0-b06ebf3732d8     N/A  da2p2
gptid/dfe5e425-825b-11eb-908e-b06ebf3732d8     N/A  da4p2
gptid/3989863d-bef4-11ec-8168-001517d6061e     N/A  da6p2
gptid/86adcff9-4fbb-11ec-b633-b06ebf3732d8     N/A  da5p2
gptid/9104a7e0-7a9e-11eb-a8e0-b06ebf3732d8     N/A  da2p1
gptid/562c819a-7daa-11eb-aeac-b06ebf3732d8     N/A  ada1p1
gptid/c8e5a283-216e-11ed-885b-b06ebf3732d8     N/A  ada0p3

And this is the output from geom disk list on TrueNAS Core:

Show : geom disk list CORE

Code:

geom disk list
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 64023257088 (60G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   descr: SAMSUNG SSD 830 Series
   lunid: 5002538043584d30
   ident: S0XXNEAC615927
   rotationrate: 0
   fwsectors: 63
   fwheads: 16

Geom name: ada2
Providers:
1. Name: ada2
   Mediasize: 512110190592 (477G)
   Sectorsize: 512
   Mode: r0w0e0
   descr: SPCC Solid State Disk
   lunid: 5000000000003480
   ident: 30080962488
   rotationrate: 0
   fwsectors: 63
   fwheads: 16

Geom name: ada1
Providers:
1. Name: ada1
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e3
   descr: TOSHIBA HDWD130
   lunid: 5000039fe6eb0d26
   ident: 98T1PXBAS
   rotationrate: 7200
   fwsectors: 63
   fwheads: 16

Geom name: da1
Providers:
1. Name: da1
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST33000651SS  E
   lunid: 5000c50042204917
   ident: Z293L8JK0000C250BY6S
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST33000651SS  E
   lunid: 5000c50042203e03
   ident: Z293L8CQ0000C2485FSD
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da3
Providers:
1. Name: da3
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST33000651SS  E
   lunid: 5000c500422032ef
   ident: Z293L8460000C2503MZK
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da2
Providers:
1. Name: da2
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r0w0e0
   descr: IBM-ESXS ST33000651SS  E
   lunid: 5000c50055c0bfef
   ident: Z295ET2B000093160SN6
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da4
Providers:
1. Name: da4
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST3000NM0043  E
   lunid: 5000c50062e5784f
   ident: Z1Y2JM090000W514R21X
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da6
Providers:
1. Name: da6
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST3000NM0043  E
   lunid: 5000c5008493065f
   ident: Z1Y3WXP50000R616N4A8
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da5
Providers:
1. Name: da5
   Mediasize: 3000592982016 (2.7T)
   Sectorsize: 512
   Mode: r2w2e5
   descr: IBM-ESXS ST3000NM0043  E
   lunid: 5000c500849d2203
   ident: Z1Y3X9XJ0000R61768VD
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

 

[Arwen]

First off, ZFS marks the pool disks as belonging to a specific host. Just loading up TrueNAS SCALE won't cause SCALE to automatically bring in your pre-existing pool. For practical purposes, a different OS, (aka SCALE), is a different host.

In this case, you generally have to import the pool. If I remember correctly, under the storage section their is an import pool button. Try that.

If that fails, it is still possible that your pool can be imported. But, you may have to resort to a more complicated method. So, if that simple GUI import fails, please supply the output of zpool import in code tags.

 

[apfelcake]

Thank you for your Answer.

The first thing i've tried was migrating with the Pool still imported in TrueNAS Core. After migrating, it showed up in TrueNAS Scale, but it was marked as unavailable.

So I went back to TrueNAS Core and exported the pool and migrated again. In SCALE it didn't showed up in the GUI, when I tried to import it. So I tried to import it via "zpool import", the output can be found in my first post.

 

[morganL]

It seems there is a problem with that LSI 2008 driver. Feel free to report a bug.

Reddit - The heart of the internet

www.reddit.com

 

[apfelcake]

The interesting thing is, that I can see the disks, but they are not assigned to the pool.

I've got one spare SAS Drive, which is on the same controller. When I create a pool on TrueNAS Core with it ( Pool: "test") it still shows up after migrating to TrueNAS Scale. But after migrating the 7 disks from Pool 1 are not assigned anymore (only 1 is still assigned, but its a drive, which is hooked up via SATA).

Do you think it is because it is encrypted (its not GELI encrypted)? The "test" pool is not encrypted.

 

[morganL]

None of the drives are working through the SAS HBA?

 

[apfelcake]

They are, but they are not assigned to the pool. I just recognized an error:

It seems that the drives are formatted with 8 bytes of "Protection Information" and TrueNAS Scale cannot handle this.

Troubleshooting disk format warnings in TrueNAS SCALE

Synopsis In today's world, reliable data storage is the most valuable asset, treat it accordingly and always purchase your CMR hard drives from reputable sellers. If you cannot successfully perform the formatting procedure detailed below, it usually means your hard drive has custom firmware...

www.truenas.com

Do I have to reformat them one by one and resilver afterward or is there a better way to get these drives working in Scale?

 

[Arwen]

Ah, that UNAVAIL now makes sense. I was not aware that FreeBSD, (used in TrueNAS Core), supported that DIF / Data Integrity Feature directly, without extra work.

It appears the Linux kernel used in SCALE does not support that feature, at least automatically.

So I would think yes, you have to reformat them one by one, and resilver afterwards. With RAID-Z2 in theory you can do 2 at once, but have no redundancy on your pool during the procedure... not something I would recommend.

 

[yucong111]

Hi，@Arwen
When I upgraded the scale from core I get the error too, how can I save my data?

 

[Arwen]

Hi，@Arwen
When I upgraded the scale from core I get the error too, how can I save my data?
View attachment 73274

In general, unless you know it is the same problem, (Data Integrity Feature), start a new forum thread.

Further, in that new forum thread, please include all the hardware details, including make and model of the storage drive, and how they are connected to the computer.