HBA card woes, need help

[HTLover]

Hello all. I'm new to the Truenas world as I'm coming from Unraid. I wanted to build a server using Truenas and get away from Unraid for several reasons. I have a SuperMicro X10SLL-F SERVER board, Intel E3-1276 V3, Kingston 32GB (4X8GB) DDR3 1600MHz PC3 12800 UDIMM ECC, 8 x 10TB drives, 1TB NVME (Cache). I think that is the relevant bits. I've tried 3 different expansion cards and none of them are being recognized. I've put them in all three pci-e slots with no configuration yielding results that Truenas will see. If I plug a couple hard drives into the MB sata ports they are recognized by Truenas so the drives work. When I plug the NVME into any of the pci-e slots it is recognized so the pci-e slots work. Below are the cards I've tried so far. It seems odd none of these are being recognized by Truenas so I have a feeling I have either a BIOS setting wrong or a Truenas setting wrong. I have a buddy that has been a Freenas guy for many years but has no advice for me and searching the forums didn't yield any useful info. I feel I am missing something simply here but hope someone can steer me in the right direction. I don't want to buy any more cards to try!!


1. 9272-8I 6Gbs PCI-E HBA FW: P20( LSI 9207-8i IT) Mode
2. LSI 9207-8i (IBM M5110) FLASHED TO IT MODE PCI-E
3. Old (11 years) 2 port Sata card (3 of them) (can get part number later)

Any advice would be greatly appreciated!!!

 

[firsway]

Hello all. I'm new to the Truenas world as I'm coming from Unraid. I wanted to build a server using Truenas and get away from Unraid for several reasons. I have a SuperMicro X10SLL-F SERVER board, Intel E3-1276 V3, Kingston 32GB (4X8GB) DDR3 1600MHz PC3 12800 UDIMM ECC, 8 x 10TB drives, 1TB NVME (Cache). I think that is the relevant bits. I've tried 3 different expansion cards and none of them are being recognized. I've put them in all three pci-e slots with no configuration yielding results that Truenas will see. If I plug a couple hard drives into the MB sata ports they are recognized by Truenas so the drives work. When I plug the NVME into any of the pci-e slots it is recognized so the pci-e slots work. Below are the cards I've tried so far. It seems odd none of these are being recognized by Truenas so I have a feeling I have either a BIOS setting wrong or a Truenas setting wrong. I have a buddy that has been a Freenas guy for many years but has no advice for me and searching the forums didn't yield any useful info. I feel I am missing something simply here but hope someone can steer me in the right direction. I don't want to buy any more cards to try!!


1. 9272-8I 6Gbs PCI-E HBA FW: P20( LSI 9207-8i IT) Mode
2. LSI 9207-8i (IBM M5110) FLASHED TO IT MODE PCI-E
3. Old (11 years) 2 port Sata card (3 of them) (can get part number later)

Any advice would be greatly appreciated!!!

Is your machine "seeing" the HBA at POST/Startup and just not the drives, or is it completely not seeing anything?

 

[HTLover]

Firsway, yes both LSI cards showed up during the post/startup. It went by pretty quick so i could read everything it said but it definitely saw the LSI cards when plugged in.

 

[Ericloewe]

So what's the output of sas2flash -listall and sas2ircu 0 display and sas2ircu 1 display?

 

[firsway]

Firsway, yes both LSI cards showed up during the post/startup. It went by pretty quick so i could read everything it said but it definitely saw the LSI cards when plugged in.

You don't specify whether you're using SAS or SATA drives (might not actually matter) but are you certain you are using the correct cable from the HBA to the drives? I ask this, because I made the same mistake a few years ago!
For example a Silverstone SST-CPS05-RE is not the right cable, whereas the SST-CPS05 works correctly!
Check that you are not using a reverse-cable...

 

[HTLover]

Interesting, well that is news to me! So I pulled the drives from an older windows machine where I had a RAID Card. the backplane of the Silverstone CS380B case has SAS and SATA connectors. So the drives are SATA I pulled from a windows computer I had. It was a Raid 5 using a RAID card. The breakout cables I used on that setup wouldn't work with the HAB LSI cards so I bought some of amazon. They are Cable Matters Internal Mini SAS to SATA Reverse Breakout Cable (SFF-8087 to SATA) 1.6 Feet. You stated NOT using a reverse-cable. Well slap me silly, is that the situation here?!?!?!?

 

[Ericloewe]

They are Cable Matters Internal Mini SAS to SATA Reverse Breakout Cable (SFF-8087 to SATA) 1.6 Feet. You stated NOT using a reverse-cable. Well slap me silly, is that the situation here?!?!?!?

Yeah, they're very difficult to tell apart and 100% incompatible. Reverse-breakout cables are only to combine host-side SATA-style ports into SAS-style (SFF-8087, SFF-8643, etc.) connectors on the backplane side; forward breakout cables split host SAS-style ports into individual SATA-style connectors for devices.

 

[HTLover]

Wow, I am embarrassed. But I learned something new today!! Thanks for everyone's assistance. I'll buy some forward breakout cables and test out Wednesday.

 

[firsway]

Interesting, well that is news to me! So I pulled the drives from an older windows machine where I had a RAID Card. the backplane of the Silverstone CS380B case has SAS and SATA connectors. So the drives are SATA I pulled from a windows computer I had. It was a Raid 5 using a RAID card. The breakout cables I used on that setup wouldn't work with the HAB LSI cards so I bought some of amazon. They are Cable Matters Internal Mini SAS to SATA Reverse Breakout Cable (SFF-8087 to SATA) 1.6 Feet. You stated NOT using a reverse-cable. Well slap me silly, is that the situation here?!?!?!?

Indeed - after a while tearing my hair out, that was my resolution. Both reverse and standard cables were available where I bought from, I'd just clicked the wrong code! As EricLoewe says, you can't really see the difference physically, perhaps the label gives away..
I'm using the CS380B as well.

 

[jgreco]

Wow, I am embarrassed.

Please, don't be. Coming from the guy who wrote the SAS Primer. You officially have permission to not be embarrassed, not that you really need it or anything. This is just one of those things you learn, and lots of people have run into it.

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop topologies and this kind of sucked. SATA and Serial...

www.truenas.com

 

[HTLover]

jgreco, I appreciate the words. I was pulling my hair out and knew I was missing something "silly" and the truenas community came through!!! I look forward to digging deep into this realm and having some fun.

 

[jgreco]

jgreco, I appreciate the words. I was pulling my hair out and knew I was missing something "silly"

I do this stuff professionally. Professionally is sort of a term that means "the embarrassing mistakes are much more obscure and arcane." If you spend much time here, you'll probably run into a bunch of stuff from people like @Ericloewe and other high volume posters, often consisting of those kinds of errors which are NOT obvious to someone coming into this new from the PC/server/hobbyist/gamer worlds. There's a lot to know. Mistakes are made. As long as they don't cook your hardware or kill your data, live and learn.

nd the truenas community came through!!! I look forward to digging deep into this realm and having some fun.

A bunch of super-smart people lurking around here. Lots to learn. It can be fun.

 

[firsway]

jgreco, I appreciate the words. I was pulling my hair out and knew I was missing something "silly" and the truenas community came through!!! I look forward to digging deep into this realm and having some fun.

Likewise the same hair pulling for me at first! I had to go back to basics although the answer was staring at me in the face!
@jgreco is right, nothing to be embarrassed about - it happens! Good luck with your rebuild!

 

[firsway]

Please, don't be. Coming from the guy who wrote the SAS Primer. You officially have permission to not be embarrassed, not that you really need it or anything. This is just one of those things you learn, and lots of people have run into it.

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop topologies and this kind of sucked. SATA and Serial...

www.truenas.com

Wish I'd seen that SAS Primer first I will take a good read of it now!

 

[jgreco]

Wish I'd seen that SAS Primer first I will take a good read of it now!

Also feel free to read the last post in this thread, quoted here

Yeah, that's kinda the nature of the business. We've been slowly upgrading from LSI 2008 to LSI 3008, and after upgrading one host, it developed a weird problem. After temporarily replacing the controller and cable on the bench, weird problems remained, and not having spare backplanes for that particular chassis, I ordered one for $25 on eBay from one of the usual suspects. Swapping it in, ... the problem remained. And on top of it, the replacement had one HDD LED that didn't work. So it turned out that there was something weird about the HP 8643-8087 cable I had used for four of the ports, and the alternate cable I had tried was the exact same part. Ugh. Fortunately, I did not feel bad returning the backplane because it did have a bad LED, and, yay, free eBay returns.

Home hobbyists look at me with a pained expression when I say that debugging is often a process of swapping things out until you find the actual culprit. Yes I know it sucks.

It really does happen to us all.