hptnr drives enumeration

[roman mak]

Hello everyone,
I just installed FreeNAS 9.3 on a backblaze with two Rocket 750 cards and 45 x 4TB HDDs. I'm unable to start S.M.A.R.T. Which fails complaining about disk 21 on the second hptnr controller:


Dec 18 20:07:03 bbz01 notifier: Starting smartd.
Dec 18 20:08:02 bbz01 smartd[12033]: Unable to register ATA device /dev/hptnr [hpt_disk_2/21/1] at line 48 of file /usr/local/etc/smartd.conf
Dec 18 20:08:02 bbz01 smartd[12033]: Unable to register device /dev/hptnr [hpt_disk_2/21/1] (no Directive -d removable). Exiting.
Dec 18 20:08:02 bbz01 root: /usr/local/etc/rc.d/smartd: WARNING: failed to start smartd
Dec 18 20:08:02 bbz01 notifier: /usr/local/etc/rc.d/smartd: WARNING: failed to start smartd

[root@bbz01] ~# tail -5 /usr/local/etc/smartd.conf
/dev/hptnr -d hpt,2/17 -a -n sleep -W 0,0,0 -m me@me.com -s L/(01|02|03|04|05|06|07|08|09|10|11|12)/../(6)/..
/dev/hptnr -d hpt,2/18 -a -n sleep -W 0,0,0 -m me@me.com -s L/(01|02|03|04|05|06|07|08|09|10|11|12)/../(6)/..
/dev/hptnr -d hpt,2/19 -a -n sleep -W 0,0,0 -m me@me.com -s L/(01|02|03|04|05|06|07|08|09|10|11|12)/../(6)/..
/dev/hptnr -d hpt,2/20 -a -n sleep -W 0,0,0 -m me@me.com -s L/(01|02|03|04|05|06|07|08|09|10|11|12)/../(6)/..
/dev/hptnr -d hpt,2/21 -a -n sleep -W 0,0,0 -m me@me.com

However, there is no such disk on hptnr controller, the disk ID jumps from 19 to 23

dmesg | grep 'disk probed'
....
hptnr: [02:00 15] disk probed (spinup mode: 1).
hptnr: [02:00 16] disk probed (spinup mode: 1).
hptnr: [02:00 17] disk probed (spinup mode: 1).
hptnr: [02:00 18] disk probed (spinup mode: 1).
hptnr: [02:00 19] disk probed (spinup mode: 1).
hptnr: [02:00 23] disk probed (spinup mode: 1).

Here is the output of smartctl for disk 21

[root@bbz01] ~# smartctl /dev/hptnr -d hpt,2/21 -a
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p5 amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Unknown error

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
[root@bbz01] ~#

And here is the output for smartctl for disk 24

[root@bbz01] ~# smartctl /dev/hptnr -d hpt,2/24 -a
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p5 amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model: HGST HUS724040ALA640
Serial Number: PN2334PCGTY11B
LU WWN Device Id: 5 000cca 24ccb55e5
Firmware Version: MFAOAA70
User Capacity: 4,000,787,030,016 bytes [4.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: 7200 rpm
...<snip>...

camcontrol also shows incorrect drive ID

[root@bbz01] ~# camcontrol devlist
<WDC WD5000LPVX-22V0TT0 01.01A01> at scbus0 target 0 lun 0 (pass0,ada0)
<WDC WD5000LPVX-22V0TT0 01.01A01> at scbus1 target 0 lun 0 (pass1,ada1)
<HPT DISK 0_0 4.00> at scbus7 target 0 lun 0 (pass2,da0)
<HPT DISK 0_1 4.00> at scbus7 target 1 lun 0 (pass3,da1)
<HPT DISK 0_2 4.00> at scbus7 target 2 lun 0 (pass4,da2)
<HPT DISK 0_3 4.00> at scbus7 target 3 lun 0 (pass5,da3)
<HPT DISK 0_4 4.00> at scbus7 target 4 lun 0 (pass6,da4)
<HPT DISK 0_5 4.00> at scbus7 target 5 lun 0 (pass7,da5)
<HPT DISK 0_6 4.00> at scbus7 target 6 lun 0 (pass8,da6)
<HPT DISK 0_7 4.00> at scbus7 target 7 lun 0 (pass9,da7)
<HPT DISK 0_8 4.00> at scbus7 target 8 lun 0 (pass10,da8)
<HPT DISK 0_9 4.00> at scbus7 target 9 lun 0 (pass11,da9)
<HPT DISK 0_10 4.00> at scbus7 target 10 lun 0 (pass12,da10)
<HPT DISK 0_11 4.00> at scbus7 target 11 lun 0 (pass13,da11)
<HPT DISK 0_12 4.00> at scbus7 target 12 lun 0 (pass14,da12)
<HPT DISK 0_13 4.00> at scbus7 target 13 lun 0 (pass15,da13)
<HPT DISK 0_14 4.00> at scbus7 target 14 lun 0 (pass16,da14)
<HPT DISK 0_15 4.00> at scbus7 target 15 lun 0 (pass17,da15)
<HPT DISK 0_16 4.00> at scbus7 target 16 lun 0 (pass18,da16)
<HPT DISK 0_17 4.00> at scbus7 target 17 lun 0 (pass19,da17)
<HPT DISK 0_18 4.00> at scbus7 target 18 lun 0 (pass20,da18)
<HPT DISK 0_19 4.00> at scbus7 target 19 lun 0 (pass21,da19)
<HPT DISK 0_20 4.00> at scbus7 target 20 lun 0 (pass22,da20)
<HPT DISK 0_21 4.00> at scbus7 target 21 lun 0 (pass23,da21)
<HPT DISK 0_22 4.00> at scbus7 target 22 lun 0 (pass24,da22)
<HPT DISK 0_23 4.00> at scbus7 target 23 lun 0 (pass25,da23)
<HPT DISK 0_0 4.00> at scbus8 target 0 lun 0 (pass26,da24)
<HPT DISK 0_1 4.00> at scbus8 target 1 lun 0 (pass27,da25)
<HPT DISK 0_2 4.00> at scbus8 target 2 lun 0 (pass28,da26)
<HPT DISK 0_3 4.00> at scbus8 target 3 lun 0 (pass29,da27)
<HPT DISK 0_4 4.00> at scbus8 target 4 lun 0 (pass30,da28)
<HPT DISK 0_5 4.00> at scbus8 target 5 lun 0 (pass31,da29)
<HPT DISK 0_6 4.00> at scbus8 target 6 lun 0 (pass32,da30)
<HPT DISK 0_7 4.00> at scbus8 target 7 lun 0 (pass33,da31)
<HPT DISK 0_8 4.00> at scbus8 target 8 lun 0 (pass34,da32)
<HPT DISK 0_9 4.00> at scbus8 target 9 lun 0 (pass35,da33)
<HPT DISK 0_10 4.00> at scbus8 target 10 lun 0 (pass36,da34)
<HPT DISK 0_11 4.00> at scbus8 target 11 lun 0 (pass37,da35)
<HPT DISK 0_12 4.00> at scbus8 target 12 lun 0 (pass38,da36)
<HPT DISK 0_13 4.00> at scbus8 target 13 lun 0 (pass39,da37)
<HPT DISK 0_14 4.00> at scbus8 target 14 lun 0 (pass40,da38)
<HPT DISK 0_15 4.00> at scbus8 target 15 lun 0 (pass41,da39)
<HPT DISK 0_16 4.00> at scbus8 target 16 lun 0 (pass42,da40)
<HPT DISK 0_17 4.00> at scbus8 target 17 lun 0 (pass43,da41)
<HPT DISK 0_18 4.00> at scbus8 target 18 lun 0 (pass44,da42)
<HPT DISK 0_19 4.00> at scbus8 target 19 lun 0 (pass45,da43)
<HPT DISK 0_20 4.00> at scbus8 target 20 lun 0 (pass46,da44)

Please advise.

Thanks,
Roman.

 

[cyberjock]

Pretty sad that you bought a highpoint (which is about the worst choice in the world), but bought 45x4TB drives. Give our stickies a read for appropriate hardware. ;)

If you read our stickies you'd know that Highpoints are horrible choices BECAUSE they don't do SMART. I've even got bug tickets that are very old discussing the topic.

 

[roman mak]

Yeah, sometimes choices are done for you
We purchased damn thing for ~ $25K and it shipped with CentOS on it.

Anyway, looks like drive enumeration is done wrong and I would like to get SMART started. Is there a way to tell /etc/ix.rc.d/ix-smartd to use correct drive or make a change in freenas-v1.db so it won't override /usr/local/etc/smartd.conf

Thanks,
Roman.

 

[cyberjock]

Yes, edit the source code as necessary. Nobody has done the changes for you though because the bottom line is Highpoint is a terrible choice for more reasons than just SMART. You've set yourself up with some risks with the hardware you are using and you are MUCH better off buying an M1015 or two from ebay than try to make that Highpoint work.

 

[titan_rw]

Let alone the fact that this card doesn't properly pass the raw drive through (loss of smart, extra abstraction layer, etc), I read over on STH's forum that this might simply be a 8 port marvel sata controller with additional sata port multipliers built in.

Sata port multipliers are pretty well known for their flakey-ness.

An ibm m1015 with 2 intel 24 port expanders combined with some MB sata ports would support the drives, and cost about the same as ONE of the highpoint cards. And SAS port expanders have been proven to work quite well. Someone even posted a ~128 drive setup off of a single 8 port hba.