6TB Disk works in one FreeNas but not another

[depasseg]

I have a strange problem, and I could really use help. I have 2 systems (freenas1 and freenas2). Specs are in my sig, but fn1 has an external SC847 enclosure, and fn2 is a repurposed Netgear Readynas pro6. I had 6x6TB Seagate drive in fn2. One died, did an RMA and got a recertified drive in return. To expedite replication, I moved all 6 drives into thefn1 sc847 for the initial replication. All was good. Detached the 6 drive backup pool from fn1 and installed the drives in fn2. The recertified drive wasn't detected. I swapped drives around in fn2 and no matter where it was, the recertified drive wasn't detected (at boot). So I move the drive back to fn1, and it's detected just fine. I'm kinda at a loss. Any ideas?

Show : dmseg from fn1

Code:

mpssas_get_sata_identify: error reading SATA PASSTHRU; iocstatus = 0x47
mps0: Sleeping 3 seconds after SATA ID error to wait for spinup
mps0: SAS Address for SATA device = 3b2d3b3a71ad5158
mps0: SAS Address from SATA device = 3b2d3b3a71ad5158
da41 at mps0 bus 0 scbus5 target 41 lun 0
da41: <ATA ST6000DX000-1H21 CC48> Fixed Direct Access SCSI-6 device
da41: Serial Number             Z4D07NJZ
da41: 600.000MB/s transfers
da41: Command Queueing enabled
da41: 5723166MB (11721045168 512 byte sectors: 255H 63S/T 729601C)
ses2: da41,pass46: Element descriptor: 'Slot 10'
ses2: da41,pass46: SAS Device Slot Element: 1 Phys at Slot 9
ses2:  phy 0: SATA device
ses2:  phy 0: parent 50030480017041ff addr 50030480017041d1
GEOM: da41: the secondary GPT header is not in the last LBA.
mps0: mpssas_prepare_remove: Sending reset for target ID 41
da41 at mps0 bus 0 scbus5 target 41 lun 0
da41: <ATA ST6000DX000-1H21 CC48> s/n             Z4D07NJZ detached
mps0: Unfreezing devq for target ID 41
(da41:mps0:0:41:0): Periph destroyed
mpssas_get_sata_identify: error reading SATA PASSTHRU; iocstatus = 0x47
mps0: Sleeping 3 seconds after SATA ID error to wait for spinup
mps0: SAS Address for SATA device = 3b2d3b3a71ad5158
mps0: SAS Address from SATA device = 3b2d3b3a71ad5158
da41 at mps0 bus 0 scbus5 target 41 lun 0
da41: <ATA ST6000DX000-1H21 CC48> Fixed Direct Access SCSI-6 device
da41: Serial Number             Z4D07NJZ
da41: 600.000MB/s transfers
da41: Command Queueing enabled
da41: 5723166MB (11721045168 512 byte sectors: 255H 63S/T 729601C)
ses2: da41,pass46: Element descriptor: 'Slot 10'
ses2: da41,pass46: SAS Device Slot Element: 1 Phys at Slot 9
ses2:  phy 0: SATA device
ses2:  phy 0: parent 50030480017041ff addr 50030480017041d1

Show : dmseg from fn2

Code:

(aprobe5:ahcich5:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 46 00
(aprobe5:ahcich5:0:0:0): CAM status: ATA Status Error
(aprobe5:ahcich5:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe5:ahcich5:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00
(aprobe5:ahcich5:0:0:0): Retrying command
(aprobe5:ahcich5:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 46 00
(aprobe5:ahcich5:0:0:0): CAM status: ATA Status Error
(aprobe5:ahcich5:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe5:ahcich5:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00
(aprobe5:ahcich5:0:0:0): Error 5, Retries exhausted

Show : camcontrol from recertified disk working in fn1

Code:

[root@freenas1] ~# camcontrol identify da41
pass46: <ST6000DX000-1H217Z CC48> ATA-10 SATA 3.x device
pass46: 600.000MB/s transfers, Command Queueing Enabled

protocol              ATA/ATAPI-10 SATA 3.x
device model          ST6000DX000-1H217Z
firmware revision     CC48
serial number         Z4D07NJZ
WWN                   5000c5005d25ec0b
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 4096, offset 0
LBA supported         268435455 sectors
LBA48 supported       11721045168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      no      0/0x00
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              yes      no      0/0x0
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      no
[root@freenas1] ~# 

Show : camcontrol from other disk working in fn2

Code:

[root@freenas2] ~# camcontrol identify ada0
pass0: <ST6000DX000-1H217Z CC48> ATA-10 SATA 3.x device
pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)

protocol              ATA/ATAPI-10 SATA 3.x
device model          ST6000DX000-1H217Z
firmware revision     CC48
serial number         Z4D1AADE
WWN                   5000c5007ac9f913
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 4096, offset 0
LBA supported         268435455 sectors
LBA48 supported       11721045168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      no      0/0x00
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              yes      no      0/0x0
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      no
[root@freenas2] ~#  

 

[Ericloewe]

That's an incredibly weird issue... What happens with a different OS?

 

[joeschmuck]

The recertified drive wasn't detected.

In the BIOS or just by FreeNAS?

 

[depasseg]

That's an incredibly weird issue... What happens with a different OS?

Windows sees it (although I have an old USB external dock, so it only saw 1.6TB instead of 6).

 

[depasseg]

In the BIOS or just by FreeNAS?

BIOS takes ~20-30 seconds detecting AHCI port #5 and then says nothing, so bios only sees the 5 drives (not the 6th). Freenas2 reports the errors via the dmesg error I attached above. Freenas1 is ok.

 

[joeschmuck]

What bugs me is that all the drives are 6TB, same model, and only the re-certified drive is giving you issue. The fact that the BIOS doesn't recognize it on freenas2 is the starting point of the problem. What if it were the only drive attached?

How about (grasping at straws here) performing a wipe of the drive in case it's got some weird format from the factory. I don't see how that would cause an issue but we have seen odd things before.

 

[Ericloewe]

What bugs me is that all the drives are 6TB, same model, and only the re-certified drive is giving you issue. The fact that the BIOS doesn't recognize it on freenas2 is the starting point of the problem. What if it were the only drive attached?

Right. Everything says they should be absolutely identical.

How about (grasping at straws here) performing a wipe of the drive in case it's got some weird format from the factory. I don't see how that would cause an issue but we have seen odd things before.

Better than anything I can come up with (i.e., nothing).

 

[solarisguy]

Did you try rearranging ports and/or cables in fn2 ?

 

[depasseg]

Did you try rearranging ports and/or cables in fn2 ?

Yep, I swapped that drive with 3 others. The failure followed the drive.

 

[tvsjr]

I would suggest possibly an odd sector size, but in that case, Windows shouldn't know what to do with it either.

This is a recertified drive... sure wouldn't be the first time that a recertified drive came back not working perfectly...

 

[solarisguy]

I would suggest possibly an odd sector size, but in that case, Windows shouldn't know what to do with it either.

This is a recertified drive... sure wouldn't be the first time that a recertified drive came back not working perfectly...

It works in fn1!

 

[depasseg]

It works in fn1!

Unfortunately, I need the drive in fn2 in my backup pool.

 

[solarisguy]

Unfortunately, I need the drive in fn2 in my backup pool. :)

Swap the some other drives between the servers first? :D

I have looked again at your initial post. So your fn1 has it connected to a SATA 3.x port, and Netgear ReadyNAS Pro6 has it on SATA 2.x port. Do you have a SATA 2.x port in fn1? You said you have moved the drive around in fn2, but if fn2 has a SATA 3.x port, did you try in it ?

 

[ChriZ]

Does the recertified drive have the same firmware as the others?

 

[solarisguy]

Does the recertified drive have the same firmware as the others?

Both the re-certified drive and a working drive have firmware revision CC48

 

[ChriZ]

Both the re-certified drive and a working drive have firmware revision CC48

Yeap, should have taken a closer look to camcontrol output...
While has happened in the past, (two different firmwares - same version number), I doubt it's the case now...
If cabling is out of the equation, through, perhaps a firmware reflash would worth a try, since everything else seems irrelevant to the issue...

 

[depasseg]

I've destroyed the partitions, and repartioned it in fn1. I've also wiped the disk in fn1. I don't have any sata 3 ports in fn2 nor sata 2 ports in fn1. As was noticed, the firmware is the same between all drives. the only difference from the factory, was that advanced power management was enabled. I disabled in using both camcontrol as well as via the GUI (I had to enable power off and then disable it).

The only thing I can think of is an incompatibility between the disk and the sata controller in fn2. but I don't know how to go about diagnosing that.

 

[Ericloewe]

The only thing I can think of is an incompatibility between the disk and the sata controller in fn2. but I don't know how to go about diagnosing that.

It wouldn't make sense, since the drives are supposedly identical.

Maybe Seagate can help you further...

 

[solarisguy]

[...] Maybe Seagate can help you further...

@depasseg, it appears that those drives have (undocumented) jumpers. You may try to ask Seagate, whether there is a jumper setting that would make your drive SATA 2 drive (some WD drives have such an option) or would initialize some deep hardware reset etc.

 

[Z300M]

@solarisguy: I guess that he can compare jumper settings on the two drives, but why would a differently set jumper affect the ability of only one machine to detect a drive?