Raidz2 pool no longer accessible after moving to USB

[MHOOO]

Hi all,

For some reason I get strange behaviour with a pool that I would like to use for backups.
I have decided to replace my raidz2 pool (old) with a striped mirror pool (new) and the replacement went just fine, except that:

• the old pool which lived inside my HP Microserver Gen8 is now connected via two USB docks
• the old pool - which worked just fine prior to me pulling the disks out - now shows as a striped mirror and is degraded, because disks can't be opened... how is that possible?:
  zpool status:
  
  Code:

     pool: tank1
  	 id: 5794346787872140487
    state: DEGRADED
   status: One or more devices are missing from the system.
   action: The pool can be imported despite missing or damaged devices.  The
  		fault tolerance of the pool may be compromised if imported.
     see: http://illumos.org/msg/ZFS-8000-2Q
   config:
  
  		tank1										   DEGRADED
  		  mirror-0									  DEGRADED
  			gptid/2512b179-1260-11e8-87cc-3464a99a6588  ONLINE
  			22011067663477126						   UNAVAIL  cannot open
  		  mirror-1									  DEGRADED
  			1259817509696220100						 UNAVAIL  cannot open
  			gptid/28687134-1260-11e8-87cc-3464a99a6588  ONLINE
  

• the old pool yields GPT errors in dmesg:
  
  Code:

  
  da1 at umass-sim1 bus 1 scbus8 target 0 lun 0
  da1: <WDC WD20 EFRX-68EUZN0 0> Fixed Direct Access SPC-4 SCSI device
  da1: Serial Number 1234567890DD
  da1: 40.000MB/s transfers
  da1: 1907729MB (3907029168 512 byte sectors)
  da1: quirks=0xa<NO_6_BYTE,4K>
  GEOM_MULTIPATH: disk2 created
  GEOM_MULTIPATH: da1 added to disk2
  GEOM_MULTIPATH: da1 is now active path in disk2
  da2 at umass-sim1 bus 1 scbus8 target 0 lun 1
  da2: <WDC WD20 EFRX-68EUZN0 0> Fixed Direct Access SPC-4 SCSI device
  da2: Serial Number 1234567890DD
  da2: 40.000MB/s transfers
  da2: 1907729MB (3907029168 512 byte sectors)
  da2: quirks=0xa<NO_6_BYTE,4K>
  GEOM: multipath/disk2: corrupt or invalid GPT detected.
  GEOM: multipath/disk2: GPT rejected -- may not be recoverable.
  ugen1.2: <ASMT ASM1156-PM> at usbus1
  umass2 on uhub2
  umass2: <ASMT ASM1156-PM, class 0/0, rev 3.00/1.00, addr 1> on usbus1
  umass2:  SCSI over Bulk-Only; quirks = 0x0000
  umass2:9:2: Attached to scbus9
  da3 at umass-sim2 bus 2 scbus9 target 0 lun 0
  da3: <ASMT ASM1156-PM 0> Fixed Direct Access SPC-4 SCSI device
  da3: Serial Number 00000000000000000000
  da3: 400.000MB/s transfers
  da3: 1907729MB (3907029168 512 byte sectors)
  da3: quirks=0x2<NO_6_BYTE>
  GEOM_MULTIPATH: disk1 created
  GEOM_MULTIPATH: da3 added to disk1
  GEOM_MULTIPATH: da3 is now active path in disk1
  da4 at umass-sim2 bus 2 scbus9 target 0 lun 1
  da4: <ASMT ASM1156-PM 0> Fixed Direct Access SPC-4 SCSI device
  da4: Serial Number 00000000000000000000
  da4: 400.000MB/s transfers
  da4: 1907729MB (3907029168 512 byte sectors)
  da4: quirks=0x2<NO_6_BYTE>
  GEOM: multipath/disk1: corrupt or invalid GPT detected.
  GEOM: multipath/disk1: GPT rejected -- may not be recoverable.
  

• the new pool appears to work just fine (all data is accessible) but gpart shows it as CORRUPT:
  
  Code:

   Name   Status  Components
   da0p1	   OK  da0
   da0p2	   OK  da0
  ada0p1  CORRUPT  ada0
  ada0p2  CORRUPT  ada0
  ada1p1  CORRUPT  ada1
  ada1p2  CORRUPT  ada1
  ada2p1	   OK  ada2
  ada2p2	   OK  ada2
  ada3p1	   OK  ada3
  ada3p2	   OK  ada3
   da2p1	   OK  da2
   da2p2	   OK  da2
   da4p1	   OK  da4
   da4p2	   OK  da4 

I had the new pool connected via the same USB docks that the old pool is now connected to. This way I was able to move all data to the new pool. Now, I'm wondering what exactly I did wrong to get the old pool into this faulty state and how to fix that.

Additional Info:

• camcontrol devlist
  This is strange, since ALL drives are WD REDs. No idea why da3/4 are shown as ASMT.
  
  Code:

  <WDC WD20EFRX-68AX9N0 80.00A80>	at scbus0 target 0 lun 0 (pass0,ada0)
  <WDC WD20EFRX-68AX9N0 80.00A80>	at scbus1 target 0 lun 0 (pass1,ada1)
  <WDC WD20EFRX-68EUZN0 82.00A82>	at scbus2 target 0 lun 0 (pass2,ada2)
  <WDC WD20EFRX-68EUZN0 82.00A82>	at scbus3 target 0 lun 0 (pass3,ada3)
  <SanDisk Ultra 1.00>			   at scbus7 target 0 lun 0 (pass4,da0)
  <WDC WD20 EFRX-68EUZN0 0>		  at scbus8 target 0 lun 0 (da1,pass5)
  <WDC WD20 EFRX-68EUZN0 0>		  at scbus8 target 0 lun 1 (da2,pass6)
  <ASMT ASM1156-PM 0>				at scbus9 target 0 lun 0 (da3,pass7)
  <ASMT ASM1156-PM 0>				at scbus9 target 0 lun 1 (da4,pass8)

Anyone here who might have some answers?
I'll probably just plug the old pool back in... just to see if that fixes the Problem. Though if it does, I'll be even more clueless.

Edit: Importing the old pool on my laptop works just fine... even though it still shows as being two vdevs with 2 mirrored drives in it. I can't explain how that is possible, since I was 100% sure I had a raidz2 setup... but I guess I was wrong. Still does not explain why I am unable to import the pool within freenas.

 

[Chris Moore]

USB is not an intended method of connecting a storage pool.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[MHOOO]

Why is that? Do you think that the reason for some drives (of the same brand) to work, and others not, is USB?
I've found different opinions (both good and bad) regarding external USB drives with freenas, but no explanation as to why USB is considered bad.

 

[Jailer]

Why is that?

For the reasons your experiencing in your first post.

 

[MHOOO]

There are no answers, only questions in my first post. I don't see how your response sheds any light on the issue.
Does that mean that the new drive is bad and I should switch back to the old ones, which were not created using a backup via USB?

 

[MHOOO]

I've replaced the new pool with the old pool again, but the multipath error persisted ("
corrupt or invalid GPT detected."). According to https://forums.freenas.org/index.ph...-after-adding-another-external-storage.28264/ I have tried:
gmultipath destroy disk1
gmultipath destroy disk2
After this, the old pool, was importable again. It did show a resilver via zpool status, but maybe that was due to destroying the multipath information? I'm not sure.

What I have learned:

• FreeNAS does not like USB and a existing GPT/Multipath configuration will probably break if you move a SATA connected pool to USB

 

[Jailer]

I don't see how your response sheds any light on the issue.

It means USB is troublesome and your experience so far with a USB connected pool is not unique.