All disks "GPT rejected -- may not be recoverable" after migrating from bare-metal to ESXi Passthrough

S

sunshine931

Explorer
Joined
Jan 23, 2018
Messages
54
I've run FreeNAS on bare metal for years, but recently had need for more VMs with capabilities that exceed what I could do with bhyve or iocage jails.

I decided I'd migrate to ESXi on bare metal, with FreeNAS in a VM, and my HBA passed through to the VM.

I built things as I describe above - installed ESXi on bare metal, then created a FreeNAS VM and passed through my HBA, booted it up and noticed that my dmesg output is full of scary messages.

Any suggestions as what might be happening here?!

Code:
da0: <VMware Virtual disk 2.0> Fixed Direct Access SPC-4 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 12288MB (25165824 512 byte sectors)
da0: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
da7 at mps0 bus 0 scbus33 target 44 lun 0
da7: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number YVGG161D       
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 2861588MB (5860533168 512 byte sectors)
da14 at mps0 bus 0 scbus33 target 51 lun 0
da14: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da14: Serial Number YVG08VSD       
da14: 600.000MB/s transfers
da14: Command Queueing enabled
da14: 2861588MB (5860533168 512 byte sectors)
da1 at mps0 bus 0 scbus33 target 38 lun 0
da1: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number YHKWT6TD       
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 2861588MB (5860533168 512 byte sectors)
da16 at mps0 bus 0 scbus33 target 53 lun 0
da16: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da16: Serial Number YVGP6J9D       
da16: 600.000MB/s transfers
da16: Command Queueing enabled
da16: 2861588MB (5860533168 512 byte sectors)
da4 at mps0 bus 0 scbus33 target 41 lun 0
da4: <HITACHI HUS72303CLAR3000 C442> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number YXG52N9K       
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 2861588MB (5860533168 512 byte sectors)
da5 at mps0 bus 0 scbus33 target 42 lun 0
da5: <HITACHI HUS72303CLAR3000 C442> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number YXG5JE6K       
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 2861588MB (5860533168 512 byte sectors)
da3 at mps0 bus 0 scbus33 target 40 lun 0
da3: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number YVHT4K3K       
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 2861588MB (5860533168 512 byte sectors)
da9 at mps0 bus 0 scbus33 target 46 lun 0
da9: <HITACHI HUS72303CLAR3000 C442> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number YXG554YK       
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 2861588MB (5860533168 512 byte sectors)
da6 at mps0 bus 0 scbus33 target 43 lun 0
da6: <HITACHI HUS72303CLAR3000 C442> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number YXG5JEBK       
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 2861588MB (5860533168 512 byte sectors)
da8 at mps0 bus 0 scbus33 target 45 lun 0
da8: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number YVGHJUTD       
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 2861588MB (5860533168 512 byte sectors)
da12 at mps0 bus 0 scbus33 target 49 lun 0
da12: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number YHJUXDVG       
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: 2861588MB (5860533168 512 byte sectors)
da10 at mps0 bus 0 scbus33 target 47 lun 0
da10: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number YHJEXGHD       
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 2861588MB (5860533168 512 byte sectors)
da15 at mps0 bus 0 scbus33 target 52 lun 0
da15: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da15: Serial Number YHJVR0BG       
da15: 600.000MB/s transfers
da15: Command Queueing enabled
da15: 2861588MB (5860533168 512 byte sectors)
da11 at mps0 bus 0 scbus33 target 48 lun 0
da11: <HITACHI HUS72303CLAR3000 C442> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number YXG5H3SK       
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 2861588MB (5860533168 512 byte sectors)
da13 at mps0 bus 0 scbus33 target 50 lun 0
da13: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number YVHJD88K       
da13: 600.000MB/s transfers
da13: Command Queueing enabled
da13: 2861588MB (5860533168 512 byte sectors)
da2 at mps0 bus 0 scbus33 target 39 lun 0
da2: <HITACHI HUS72303CLAR3000 C1D6> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number YVGBAX8K       
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 2861588MB (5860533168 512 byte sectors)
ses0: da12,pass13: SAS Device Slot Element: 1 Phys at Slot 0
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca01a9fdbcd
ses0: da1,pass2: SAS Device Slot Element: 1 Phys at Slot 1
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca01adb9dc9
ses0: da2,pass3: SAS Device Slot Element: 1 Phys at Slot 2
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e14a42d
ses0: da3,pass4: SAS Device Slot Element: 1 Phys at Slot 3
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e6445fd
ses0: da4,pass5: SAS Device Slot Element: 1 Phys at Slot 8
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca046093f49
ses0: da5,pass6: SAS Device Slot Element: 1 Phys at Slot 9
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca0460a0e11
ses0: da6,pass7: SAS Device Slot Element: 1 Phys at Slot 10
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca0460a0e25
ses0: da7,pass8: SAS Device Slot Element: 1 Phys at Slot 11
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e1986b9
ses0: da8,pass9: SAS Device Slot Element: 1 Phys at Slot 12
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e1c4301
ses0: da9,pass10: SAS Device Slot Element: 1 Phys at Slot 13
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca0460964e5
ses0: da10,pass11: SAS Device Slot Element: 1 Phys at Slot 14
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca01a8a0ac9
ses0: da11,pass12: SAS Device Slot Element: 1 Phys at Slot 15
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca04609fa79
ses0: da13,pass14: SAS Device Slot Element: 1 Phys at Slot 20
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
GEOM: da1: the secondary GPT table is corrupt or invalid.
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e562df5
ses0: da14,pass15: SAS Device Slot Element: 1 Phys at Slot 21
GEOM: da1: using the primary only -- recovery suggested.
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e008515
ses0: da15,pass16: SAS Device Slot Element: 1 Phys at Slot 22
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca01aa14d25
ses0: da16,pass17: SAS Device Slot Element: 1 Phys at Slot 23
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: Initiator( None ) Target( SSP )
ses0:  phy 0: parent 500a098004347dbf addr 5000cca03e269129
random: unblocking device.
Trying to mount root from zfs:freenas-boot/ROOT/default []...
GEOM: da2: the secondary GPT table is corrupt or invalid.
GEOM: da2: using the primary only -- recovery suggested.
GEOM: da3: the secondary GPT table is corrupt or invalid.
GEOM: da3: using the primary only -- recovery suggested.
GEOM: da4: the secondary GPT table is corrupt or invalid.
GEOM: da4: using the primary only -- recovery suggested.
GEOM: da5: the secondary GPT table is corrupt or invalid.
GEOM: da5: using the primary only -- recovery suggested.
GEOM: da6: the secondary GPT table is corrupt or invalid.
GEOM: da6: using the primary only -- recovery suggested.
GEOM: da7: the secondary GPT table is corrupt or invalid.
GEOM: da7: using the primary only -- recovery suggested.
GEOM: da8: the secondary GPT table is corrupt or invalid.
GEOM: da8: using the primary only -- recovery suggested.
GEOM: da9: the secondary GPT table is corrupt or invalid.
GEOM: da9: using the primary only -- recovery suggested.
GEOM: da10: the secondary GPT table is corrupt or invalid.
GEOM: da10: using the primary only -- recovery suggested.
GEOM: da11: the secondary GPT table is corrupt or invalid.
GEOM: da11: using the primary only -- recovery suggested.
GEOM: da12: the secondary GPT table is corrupt or invalid.
GEOM: da12: using the primary only -- recovery suggested.
GEOM: da13: the secondary GPT table is corrupt or invalid.
GEOM: da13: using the primary only -- recovery suggested.
GEOM: da14: the secondary GPT table is corrupt or invalid.
GEOM: da14: using the primary only -- recovery suggested.
GEOM: da15: the secondary GPT table is corrupt or invalid.
GEOM: da15: using the primary only -- recovery suggested.
GEOM: da16: the secondary GPT table is corrupt or invalid.
GEOM: da16: using the primary only -- recovery suggested.
vmx_init: processor does not support VMX operation
module_register_init: MOD_LOAD (vmm, 0xffffffff82a1c470, 0) error 6
GEOM_MULTIPATH: disk16 created
GEOM_MULTIPATH: da16 added to disk16
GEOM_MULTIPATH: da16 is now active path in disk16
GEOM_MULTIPATH: disk15 created
GEOM_MULTIPATH: da15 added to disk15
GEOM_MULTIPATH: da15 is now active path in disk15
GEOM_MULTIPATH: disk14 created
GEOM_MULTIPATH: da14 added to disk14
GEOM_MULTIPATH: da14 is now active path in disk14
GEOM_MULTIPATH: disk13 created
GEOM_MULTIPATH: da13 added to disk13
GEOM_MULTIPATH: da13 is now active path in disk13
GEOM_MULTIPATH: disk12 created
GEOM_MULTIPATH: da12 added to disk12
GEOM_MULTIPATH: da12 is now active path in disk12
GEOM_MULTIPATH: disk11 created
GEOM_MULTIPATH: da11 added to disk11
GEOM_MULTIPATH: da11 is now active path in disk11
GEOM_MULTIPATH: disk10 created
GEOM_MULTIPATH: da10 added to disk10
GEOM_MULTIPATH: da10 is now active path in disk10
GEOM_MULTIPATH: disk9 created
GEOM_MULTIPATH: da9 added to disk9
GEOM_MULTIPATH: da9 is now active path in disk9
GEOM_MULTIPATH: disk8 created
GEOM_MULTIPATH: da8 added to disk8
GEOM_MULTIPATH: da8 is now active path in disk8
GEOM_MULTIPATH: disk7 created
GEOM_MULTIPATH: da7 added to disk7
GEOM_MULTIPATH: da7 is now active path in disk7
GEOM_MULTIPATH: disk6 created
GEOM_MULTIPATH: da6 added to disk6
GEOM_MULTIPATH: da6 is now active path in disk6
GEOM_MULTIPATH: disk5 created
GEOM_MULTIPATH: da5 added to disk5
GEOM_MULTIPATH: da5 is now active path in disk5
GEOM_MULTIPATH: disk4 created
GEOM_MULTIPATH: da4 added to disk4
GEOM_MULTIPATH: da4 is now active path in disk4
GEOM_MULTIPATH: disk3 created
GEOM_MULTIPATH: da3 added to disk3
GEOM_MULTIPATH: da3 is now active path in disk3
GEOM_MULTIPATH: disk2 created
GEOM_MULTIPATH: da2 added to disk2
GEOM_MULTIPATH: da2 is now active path in disk2
GEOM_MULTIPATH: disk1 created
GEOM_MULTIPATH: da1 added to disk1
GEOM_MULTIPATH: da1 is now active path in disk1
GEOM: multipath/disk16: corrupt or invalid GPT detected.
GEOM: multipath/disk16: GPT rejected -- may not be recoverable.
GEOM: multipath/disk15: corrupt or invalid GPT detected.
GEOM: multipath/disk15: GPT rejected -- may not be recoverable.
GEOM: multipath/disk14: corrupt or invalid GPT detected.
GEOM: multipath/disk14: GPT rejected -- may not be recoverable.
GEOM: multipath/disk13: corrupt or invalid GPT detected.
GEOM: multipath/disk13: GPT rejected -- may not be recoverable.
GEOM: multipath/disk12: corrupt or invalid GPT detected.
GEOM: multipath/disk12: GPT rejected -- may not be recoverable.
GEOM: multipath/disk11: corrupt or invalid GPT detected.
GEOM: multipath/disk11: GPT rejected -- may not be recoverable.
GEOM: multipath/disk10: corrupt or invalid GPT detected.
GEOM: multipath/disk10: GPT rejected -- may not be recoverable.
GEOM: multipath/disk9: corrupt or invalid GPT detected.
GEOM: multipath/disk9: GPT rejected -- may not be recoverable.
GEOM: multipath/disk8: corrupt or invalid GPT detected.
GEOM: multipath/disk8: GPT rejected -- may not be recoverable.
GEOM: multipath/disk7: corrupt or invalid GPT detected.
GEOM: multipath/disk7: GPT rejected -- may not be recoverable.
GEOM: multipath/disk6: corrupt or invalid GPT detected.
GEOM: multipath/disk6: GPT rejected -- may not be recoverable.
GEOM: multipath/disk5: corrupt or invalid GPT detected.
GEOM: multipath/disk5: GPT rejected -- may not be recoverable.
GEOM: multipath/disk4: corrupt or invalid GPT detected.
GEOM: multipath/disk4: GPT rejected -- may not be recoverable.
GEOM: multipath/disk3: corrupt or invalid GPT detected.
GEOM: multipath/disk3: GPT rejected -- may not be recoverable.
GEOM: multipath/disk2: corrupt or invalid GPT detected.
GEOM: multipath/disk2: GPT rejected -- may not be recoverable.
GEOM: multipath/disk1: corrupt or invalid GPT detected.
GEOM: multipath/disk1: GPT rejected -- may not be recoverable.
 
Last edited:
Chris Moore

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Instead of using [ icode ] for your text, you should use regular [ code ] tags (without the spaces); it allows the code to flow into a box with a scroll bar.

Do you have a backup?
What version of FreeNAS?
Did you follow a guide? Which one?
Do you actually have multipath?
Did you have multipath before?
Did the pool import?
 
S

sunshine931

Explorer
Joined
Jan 23, 2018
Messages
54
Fixed the CODE thing, thanks for the tip!

Do you have a backup?
Yes, but at 20+ TB, it's not something I look forward to recovering over the internet (cloud backup).

What version of FreeNAS?
11.2 U1

Did you follow a guide? Which one?
No, I took my previously working system and did as I described.
  1. Installed ESXi on bare metal (onto a raid of SATA disks managed by my internal raid card, both are out of scope as far as my FreeNAS stuff goes)
  2. Assigned my HBA as Passthrough in ESXi
  3. Created VM to host FreeNAS and added the HBA to it
  4. Installed FreeNAS in the new VM. Gave it an IP and enabled SSH.
  5. Noticed all my disks show up, but appear "corrupt".

Do you actually have multipath?
I have a single SAS connection between my HBA (Dell SAS 6GBPS HBA) and my enclosure (NetApp DS4246).

Did you have multipath before?
I don't know how to determine that, but my configuration was the same in terms of HBA, enclosure, and enclosure connectivity.

Did the pool import?
I wish.. It doesn't show up anywhere in dmesg output, zpool list, etc. Is there a someplace else I can look?

I should have included this info, just in case it helps. I see I can access the drives, and fdisk seems to think they have partitions.

Code:
root@freenas[~]# fdisk /dev/da6
******* Working on device /dev/da6 *******
parameters extracted from in-core disklabel are:
cylinders=364801 heads=255 sectors/track=63 (16065 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=364801 heads=255 sectors/track=63 (16065 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 238 (0xee),(EFI GPT)
    start 1, size 4294967295 (2097151 Meg), flag 0
        beg: cyl 0/ head 0/ sector 2;
        end: cyl 1023/ head 255/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>



Thanks for your input on this.
 
Last edited:
S

sunshine931

Explorer
Joined
Jan 23, 2018
Messages
54
Update: In an attempt to assess the damage, I'm in the process or returning to a bare-metal FreeNAS. Maybe that'll help somehow? I'll report back soon.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "All disks "GPT rejected -- may not be recoverable" after migrating from bare-metal to ESXi Passthrough"

Similar threads

MatthewSteinhoff
  • Locked
SOLVED GPT rejected -- may not be recoverable.
Replies
5
Views
7K
MatthewSteinhoff
MatthewSteinhoff
k1n6b0b
  • Locked
run_interrupt_driven_hooks: still waiting after XX seconds for xpt_config - HBA NetApp / PMC-Sierra
Replies
1
Views
3K
dlavigne
D
mdrisser
GPT Rejected -- may not be recoverable
Replies
2
Views
2K
mdrisser
mdrisser
A
  • Locked
Is this Error processing log message any real concern?
Replies
2
Views
1K
Amsoil_Jim
A
KenwoodFox
GPT rejected -- may not be recoverable.
Replies
0
Views
1K
KenwoodFox
KenwoodFox
Top