Failing drive and slow pool problem.

[Kayman]

Hey guys and girls.

I've run into the dying drive that won't get dropped from the array problem again. Same one as in my previous thread (http://forums.freenas.org/index.php?threads/large-transfers-over-cif-always-fail.22507/). Pool is still healthy just very slow.

I've used the command (zpool iostat -v 5 | more) and looked through pages of data and I've identified the drive. It consistently has values very different from the rest. Now here's my problem. Iostat lists the disk in the same format that the command (zpool status) does.

NAME STATE READ WRITE CKSUM
Main ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/b71dbc93-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/b79b8b8b-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/b875b15e-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/b8ef3b45-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/b95234f5-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/b9b44e2f-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/ba703018-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/bb6bfc77-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/bc7e18c3-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/bd8ddaba-3f06-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
raidz2-1 ONLINE 0 0 0
gptid/a2d2d711-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a3c4aa61-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a41720ae-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a4de6b6c-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a56502a2-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a5ca983c-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a63851a0-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a72746b2-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a826ab77-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0
gptid/a8fe6043-3f07-11e4-b93c-00237dfbe0f0 ONLINE 0 0 0

The bad drive according to iostat is gptid/b8ef3b45-3f06-11e4-b93c-00237dfbe0f0. This may be a dumb question but how do I find out which drive this is in terms of either /dev/adxx or /dev/daxx or its serial number. So I know in the gui wich drive to offline, replace and resilver. Here is the list of the disks.


[root@NAS ~]# camcontrol dev list
<ATA WDC WD15EARS-00M AB51> at scbus0 target 0 lun 0 (da0,pass0)
<ATA WDC WD15EARS-00Z 0A80> at scbus0 target 3 lun 0 (da1,pass1)
<ATA WDC WD15EADS-00S 0A01> at scbus0 target 8 lun 0 (da2,pass2)
<ATA WDC WD15EADS-00S 5G04> at scbus0 target 9 lun 0 (da3,pass3)
<ATA WDC WD15EADS-00S 0A01> at scbus0 target 11 lun 0 (da4,pass4)
<ATA WDC WD15EADS-00S 0A01> at scbus0 target 12 lun 0 (da5,pass5)
<ATA WDC WD15EARS-00Z 0A80> at scbus0 target 13 lun 0 (da6,pass6)
<ATA WDC WD15EARS-00Z 0A80> at scbus0 target 14 lun 0 (da7,pass7)
<ATA WDC WD15EADS-00S 0A01> at scbus1 target 5 lun 0 (da8,pass8)
<ATA WDC WD15EADS-00R 0A01> at scbus1 target 8 lun 0 (da9,pass9)
<ATA WDC WD15EADS-00S 5G04> at scbus1 target 10 lun 0 (da10,pass10)
<ATA WDC WD15EARS-22M AB51> at scbus1 target 11 lun 0 (da11,pass11)
<ATA WDC WD15EARS-00Z 0A80> at scbus1 target 13 lun 0 (da12,pass12)
<ATA WDC WD15EARS-00Z 0A80> at scbus1 target 14 lun 0 (da13,pass13)
<ATA WDC WD15EARS-00Z 0A80> at scbus1 target 18 lun 0 (da14,pass14)
<ATA WDC WD15EADS-00S 0A01> at scbus1 target 19 lun 0 (da15,pass15)
<WDC WD15EARS-00J2GB0 80.00A80> at scbus2 target 0 lun 0 (ada0,pass16)
<WDC WD15EARS-00Z5B1 80.00A80> at scbus3 target 0 lun 0 (ada1,pass17)
<WDC WD15EARS-00Z5B1 80.00A80> at scbus4 target 0 lun 0 (ada2,pass18)
<WDC WD15EADS-00P8B0 01.00A01> at scbus5 target 0 lun 0 (ada3,pass19)
<SanDisk Cruzer Blade 1.26> at scbus9 target 0 lun 0 (pass20,da16)

System specs are.
FreeNAS 9.2.1.7 x64
Supermicro X9SCM-F
Xeon E3-1220v2
32 Gig ECC
Dual LSI 9211-8i cards.
20 Drives 16 on the cards and 4 on the motherboard.
Pool is Dual vdev z2.

Also my plan is to replace the 1.5tb greens with 2tb greens as they die. I've got quite a few spare 2TB and they are readily available but 1.5sTB are getting hard to find now. If 10 die and I shuffle them around so that all 10 2TB drives are in the same Vdev will the pool grown in size or does that only work if I replace both Vdevs (all 20 drives). And if this does work is it ok or a bad idea. Everywhere is says keep striped vdevs the same size and type.

 

[warri]

glabel status gives you the mapping of gptid to (a)daX.
smartctl -a /dev/adaX | grep Serial outputs the serial number for a given adaX.

 

[Kayman]

glabel status gives you the mapping of gptid to (a)daX.
smartctl -a /dev/adaX | grep Serial outputs the serial number for a given adaX.

Beautiful, exactly what I needed thank you.

 

[danb35]

If 10 die and I shuffle them around so that all 10 2TB drives are in the same Vdev will the pool grown in size or does that only work if I replace both Vdevs (all 20 drives). And if this does work is it ok or a bad idea. Everywhere is says keep striped vdevs the same size and type.

Keeping striped vdevs the same size and type may be best for performance, but it isn't essential. As you can see from my signature, my pool is comprised of two RAIDZ1 vdevs, one of 3 x 2 TB disks and the other of 3 x 3 TB disks. Capacity is as expected with this.

 

[Kayman]

Keeping striped vdevs the same size and type may be best for performance, but it isn't essential. As you can see from my signature, my pool is comprised of two RAIDZ1 vdevs, one of 3 x 2 TB disks and the other of 3 x 3 TB disks. Capacity is as expected with this.

So just to be clear your pool capacity is 10TB not 8TB.

Anyway the resilver finished overnight and the pool is back up to 110-120 MB/s on large files.

 

[danb35]

Correct.