FreeBSD 8.2-RELEASE-p7 and HighPoint RR3***/4*** series controllers

[Eaglebird]

HighPoint RocketRaid 4310 and the FreeBSD driver? Need help installing

I have recently moved from Openfiler to FreeNAS with no prior BSD experience and suffice to say it has been interesting.

I decided to move since Openfiler seemed like it was going stale and had very little support for what few issues I would encounter. I switched after needing to replace the motherboard in the NAS because of sporadic "HBA detection" of a HighPoint RocketRaid 4310 controller card.

The install of FreeNAS went okay, as far as I can tell. I set up the system and gave it its hostname, IP address, timezone, things like that, and proceeded to try and re-add my disks attached to the RR4310. It was here that I realized the disks weren't being seen. I brought up a console with SSH and looked in /dev and all I saw were the OS disk and its partitions. I started looking around for solutions. The first thing I tried was adding in the "hptiop" driver for FreeBSD, which went as follows:
-Got "hptiop" .ko file from highpoint site at http://www.highpoint-tech.com/USA_new/rr4300_download.htm
-Did 'mount -uw /'
-Copied it to the NAS to /boot/modules/ and /boot/kernel/
-Added 'hptiop_load="YES"' to loader.conf after 'kern.cam.boot_delay=10000'
-Rebooted
...and surprise, still no drives. So I tried kldload hptiop and got:

Code:

kldload: can't load /boot/modules/hptiop.ko: Exec format error

...probably because the driver was for FreeBSD 7.2?

I tried this a few times after finding that hptiop.ko was not there after a reboot. I realized I needed to change its permissions.
At this point, the .ko file is in place and the appropriate line is in loader.conf. kldstat -m shows the following:

Code:

kldstat -m pci/hptiop
Id  Refs Name
114    1 pci/hptiop

So I'm not sure if that's it loaded or not, and if it is, why I don't see my drives.

There is one questionable line in dmesg:

Code:

pci1: <mass storage, RAID> at device 0.0 (no driver attached)

I'd much appreciate any help with this, especially if it gets this working. If there's anything relevant I might have left out please let me know!

 

[cyberjock]

I had a Rocketraid 4320 and I couldn't get it to work either. I got the same useless(to me anyway) error "Exec format error". I assumed it was because the 4320 isn't supported by FreeBSD 8.0. Someone did mention somewhere on another device issue to install FreeBSD 8.2(since that's what FreeNAS is currently built from) and see if the controller is detected. If so then figure out which driver is working and copy that to your FreeNAS USB stick. I don't know how you'd go about finding the driver but I don't have the 4320 anymore so I can't help you out much more.

I do know that Highpoint was a pretty good company and had reliable products 5 years ago, but I've had nothing but problems with them regarding customer support and warranty support. I no longer recommend them...

 

[Eaglebird]

I've found the hptiop driver for FreeBSD 8.2 just by installing FreeBSD 8.2 amd64 and grabbing the module hptiop.ko from there. I placed it in FreeNAS' '/boot/kernel/' and '/boot/modules', changed their perms to 0555, and added 'hptiop_load="YES"' to loader.conf. After the system boots, kldstat reports:

Code:

[root@FileNest] ~# kldstat -v | grep hptiop
 7    1 0xffffffff80e3e000 9888     hptiop.ko (/boot/kernel/hptiop.ko)
                 7 pci/hptiop

and kldload reports that the file exists, which to me indicates that it is indeed loaded.

Still, I cannot see my RAID array (da0? There is no "da#") and dmesg still reports this early on:

Code:

pci1: <mass storage, RAID> at device 0.0 (no driver attached)

I am not sure if I'm missing something or if there's a configuration option I need to set or what.

 

[Eaglebird]

I just found this in dmesg as well but don't know if it pertains to my problem; seems as if the module is trying to load more than once?:

Code:

kernel: module_register: module pci/hptiop already exists!
kernel: Module pci/hptiop failed to register: 17

 

[cyberjock]

Unfortunately, I am not familiar with FreeBSD and FreeNAS to help you. Maybe someone else can fill you in on what the error might mean. I'm definitely going to watch this thread though since I'm curious how this will all work out.

 

[Eaglebird]

[solved]

I've solved the issue. The problem exists in the source for the module, hptiop.c. There is a switch statement, "switch (id) {" that has a "case=0x4320:" followed by "sas = 1;". This determines what card(s) work with the hptiop driver module. To fix it, extra cases are added for the 4322 and the 4310.
The addition follows in hptiop.c:

Code:

	switch (id) {
		case 0x4320:
		case 0x4322: // NEW ADDITION
		case 0x4321: // NEW ADDITION
		case 0x4310: // NEW ADDITION, these cards use the hptiop driver.
			sas = 1;
		case 0x3220:
		case 0x3320:
		case 0x3410:
		case 0x3520:
		case 0x3510:
		case 0x3511:
		case 0x3521:
		case 0x3522:
		case 0x3540:
			ops = &hptiop_itl_ops;
			break;
		case 0x3120:
		case 0x3122:
		case 0x3020:
			ops = &hptiop_mv_ops;
			break;
		default:
			return (ENXIO);
	}

Once this change is made and the module built, it can be used in place of the kernel's module. At the next boot, dmesg will report on the controller in its jargon and pciconf will show the driver attached to the controller:

Code:

hptiop0: adapter at PCI 1:0:0, IRQ 16
hptiop0: <RocketRAID 4310 SAS Controller
> mem 0xe1000000-0xe17fffff irq 16 at device 0.0 on pci1
hptiop0: 0 RocketRAID 3***/4*** controller driver v1.3 (010208)
hptiop0: [GIANT-LOCKED]
hptiop0: [ITHREAD]

Code:

[root] /# pciconf -lvbc | grep hptiop
hptiop0@pci0:1:0:0:     class=0x010400 card=0x00001103 chip=0x43101103 rev=0x09 hdr=0x00

With the driver attached, any arrays or disks should show up in /dev/ as a da#, e.g. my HW RAID 1 shows up as /dev/da0.

 

[cyberjock]

You are the MAN! I might have to try this out just to learn how this all works and the experience. How did you build the module?

 

[cyberjock]

Ok. So I have a RocketRaid 3560. It's a 3540 with 24 ports instead of 16 ports. Since I have 2 of these rather expensive(and right now useless) controllers I was thinking I'd try to get them working, then maybe publish a guide on how to use them. The controller supports HBA mode and has battery backup with 2GB of onboard cache. It should make a good controller for a FreeNAS server.

So I got a spare machine together. I downloaded FreeBSD 8.3 x64 so I can grab the driver like Eaglebird did. I've found the driver in /boot/kernel(thanks to Eaglebird..lol). I copied all of the hpt files to a thumbdrive. So I have these files now:

hpt27xx.ko
hpt27xx.ko.symbols
hptiop.ko
hptiop.jo.symbols
hptmv.ko
hptmv.ko.symbols
hptrr.ko
hptrr.ko.symbols

I grabbed all of these because I figured it was better to grab too many than not all of them.

But the .ko file is in binary. I can't edit it nor is there the .c file anywhere I could find. Any help in the right direction? I'm doing this as a learning exercise. I'm fully aware i'm probably in way over my head, but I gotta learn somehow. I'm going to get this card to work even if it kills me :P

I then tried downloading the FreeBSD 7.2 drivers from highpoint's website http://www.highpoint-tech.com/BIOS_Driver/page/rr3560.htm. I don't get the .c files Eaglebird mentioned either.

I could be wrong, but my understanding is that I should use an 8.x version of FreeBSD drivers on an 8.x version of FreeBSD. I can't take the drivers from 9.0 and expect them to work in 8.x. Is this true? I was hoping to put together the FreeBSD drivers for 8.3 and 9.0 and host them on my dropbox for anyone to use. I figure at some point FreeNAS will be on FreeBSD 9.0 and I'll need the file anyway(assuming I can't use the 8.3 driver file).

Last question, once I have this hptio.c file and modified it, how do I compile it? If it's long and drawn out and you just want to point me in the direction that's fine. I'll enjoy the challenge of figuring it out.

I also found a copy of hptio.c at http://www.leidinger.net/FreeBSD/dox/dev_hptiop/html/dd/d51/hptiop_8c_source.html. Of course, I don't know what version of FreeBSD it is for(or if it matters). I found a version of FreeBSD 9's hptiop.c at http://svnweb.freebsd.org/base/stable/9/sys/dev/hptiop/hptiop.c?view=markup&pathrev=236655.

Yes, I've done alot of searching. Trying to make it a little easier for anyone that would like to assist.

Thanks.

 

[cyberjock]

Ok, update time. I couldn't find the hptiop.c and hptiop.h file because I hadn't installed the FreeBSD 8.3 developer options. I thought I had chosen a full install last time, but apparently not. Anyway, I have the files now. Just need to figure out how to compile them. I have the .c, .h and a related makefile.

I've changed the applicable portion as shown above by EagleBird to the following: I haven't done programming in any language since high school, so if anyone can tell me if I have anything wrong I'd be oblidged. I'm pretty sure I got it right though. I'm just adding support for their entire 35xx and 43xx line of cards. I've reordered the cards to make it easier to read.

Code:

 
switch (id) {
		case 0x4322:
		case 0x4321:
		case 0x4320:
		case 0x4311:
		case 0x4310:
			sas = 1;
		case 0x3220:
		case 0x3320:
		case 0x3410:
		case 0x3510:
		case 0x3511:
		case 0x3520:
		case 0x3521:
		case 0x3522:
		case 0x3530:
		case 0x3540:
		case 0x3560:
			ops = &hptiop_itl_ops;
			break;
		case 0x3120:
		case 0x3122:
		case 0x3020:
			ops = &hptiop_mv_ops;
			break;

Now just have to figure out how to compile and test.

 

[ProtoSD]

It's not easy to explain to a noob ;)

Very generally, if you're in the folder with the makefile, you can try just typing "make" and see what happens. You might have to search for the .ko file(s) if/when it finishes.

find /usr/src -type f -name "hptiop.ko" -print

What folder is the makefile you have in?

I'll try to give you some tips, but I'm up to my ears in plugin stuff right now.

 

[cyberjock]

The original hptiop.h and hptiop.c were in usr/src/sys/dev/hptiop. I've replaced the .c file with the one I modified. I went to the /usr/src/sys/modules/hptiop and ran a make command and I have an hptiop.ko file, several more .h files and a .o file. I'm about to test it and I hope it works. ;)

Edit: forgot to add that the Makefile was in /usr/src/sys/modules/hptiop.

 

[ProtoSD]

It sounds like you've got it. It really can be that simple. All you need is the hptiop.ko file.

 

[cyberjock]

Well, I got the hptiop.ko file in /boot/kernel as well as /boot/modules. I tried kldload hptiop.ko and I get the error "kldload: can't load hptiop: Exec format error". This is the same error I got when I tried to use the default driver it came with. I added "hptiop_load="YES" to loader.conf. I see no messages that appear to be related to my hptiop driver. In any case, I'm convinced if kldload wouldn't work it won't work from loader.conf. :( So I've done something wrong.

Question: When I ran "make" to compile the driver I was in the /usr/src/sys/modules/hptiop folder. I had copied the modified hptiop.c file to /usr/src/sys/dev/hptiop folder. Note that the .c file wsa NOT in the same folder as the makefile. Does the makefile know where to go to get the modified driver or do I need to do something with it? I've never compiled anything before and I'm thinking that's the first place to look for mistakes.

Edit: Is it possible that my .ko file won't work because I compiled it under FreeBSD 8.3 instead of 8.2?

 

[cyberjock]

Ok. I went back to FreeBSD 8.3. I took my compiled hptiop.ko file and overwrote the one that was in /boot/kernel. I used the command "find / -type f -name "hptiop.ko" -print to verify that the only copies are the ones I have updated. I'm convinced it's not loading or something else is wrong. I figured I'll first try to make this work in FreeBSD 8.3, then try for FreeNAS to rule out any potential issues with FreeNAS. Yes, i've never done anything like this before. LOL. Here's the outputs I get from various commands:

kldstat -m pci/hptiop

Code:

Id  Refs Name
135     1 pci/hptiop

I think this means that my controller is detected...

kldload hptiop.ko

Code:

module_register: module pci/hptiop already exists!
Module pci/hptiop failed to register: 17
kldload:  can't load hptiop.ko: File exists

I think this means that the driver is loading at bootup. I have no idea how to look at the bootup logs to see what error it gives.

kldstat -v | grep hptiop

Code:

135 pci/hptiop

pciconf -lvbc | grep hptiop

No output.

Based on the info Eaglebird left, I'm a little confused. It sounds like hptiop.ko is loading, but it's not working. Of course, I could be completely wrong since I've never done this correctly before so I'm not really sure what I'm looking for to identify the error.

One thing that caught my eye was Eaglebird said "Once this change is made and the module built, it can be used in place of the kernel's module. At the next boot, dmesg will report on the controller in its jargon and pciconf will show the driver attached to the controller"

If I assume I built the module correctly, how do I actually use it in place of the kernel's module? Did I do that just by copying my .ko file to /boot/kernel?

I found this dmesg output that is interesting:

Code:

pci3: <PCI bus> on pcib3
pci3: <mass storage, RAID> at device 0.0 (no driver attached)

I tried something sneaky.. I deleted the hptiop.ko file from /boot/kernel. I then rebooted. Since the file isn't there the OS should not be able to load it, right? After I logged in I copied my .ko file back to /boot/kernel and tried to do a kldload. I got the error that the module already esists and failed to register: 17. So I'm really confused now. I'm out of good ideas I think :(

 

[ProtoSD]

Edit: Is it possible that my .ko file won't work because I compiled it under FreeBSD 8.3 instead of 8.2?

Sorry for the delay, yes. The "symbols", names of the functions and variables compile differently and the kernel in 8.2 doesn't understand how to use/access them.

 

[ProtoSD]

Sorry, trying to catch up on what you've done.

You could try kldunload hptiop.ko, then try kldload path-to-your-module/hptiop.ko

but I'm still not sure that compiling it under 8.3 is going to work.

 

[cyberjock]

I'm actually trying to get it to load in 8.3 since that's what I compiled it in. I figure if that doesn't work then at least I've removed FreeNAS from the equation.

 

[cyberjock]

I tried a "kldunload hptiop.ko" and it says it can't find file. I tried pointing to the file in boot/kernel as well as the /usr location but it always says it can't find file.

 

[ProtoSD]

Sounds like a good plan. You seem to have the right ideas.

You need to look in the kernel config file and see if it's already been built into the kernel. My brain is mush at the moment so I can't remember the path or name, but I think it should have AMD64 in the name... sorry I should know it by now.

 

[ProtoSD]

Edit, its:

/usr/src/sys/amd64/conf/GENERIC



uggghhh