Help with R720 - Dell 12Gbs HBA - 3PAR - setup

[Zed1930]

I'm back to playing with TrueNAS and got an idea in my mind and bought some gear on ebay to try to build a little pet SAN. The end goal is to serve esxi VMs off of iscsi in my little home lab.

That being said, I had an old Dell R720 lying around, bough some new RAM/SSD/CPU/disks etc. I bought a Dell 0t93gd 12 Gigabit SAS HBA card, and a 3PAR StorServ 8000 enclosure, and I installed the latest version of TrueNAS onto a USB key (just to get my proof of concept and get all the gear recognized/talking to each other before actually installing, setting up a SLOG etc). I did research in the forums before buying everything and got the feeling that what I wanted to do was possible. I think TrueNAS sees the Dell HBA (9300e) card and the enclosure, but when I put two random SATA drives into the 3PAR I don't see any disks show up (on the web gui, or cli commands for what it's worth).

Running version 13.1:

My HBA card (Product ID says IT?):

The enclosure is seen as Xyratex?

gpart list only shows usb and one internal disk.

The HBA again:

Again only a USB key:

I think that the card is already in HBA/IT mode? I've been reviewing the process to "flash it back" to a stock LSI IT mode (from a ServeTheHome thread), however one of the first commands (FreeDOS) "megacli -adpallinfo -a0" was only showing the H710 mini in the system, so I pulled it out figuring maybe the command was getting confused . However rerunning it just shows "ERROR: Could not detect controller".

I feel like I'm either missing something really simple (I turned on only one of the Enclosure PSUs, or maybe got the SFF-8644 cabling wrong?), the DELL HBA proprietary firmware is not liking the HP enclosure or taking for granted that it's actually a Dell enclosure while it's not, or I'm completely wrong in my idea of being able to connect an external enclosure to TrueNAS.

Any ideas?

My next steps were:

• Try flashing the card away from Dell proprietary, but want to make sure to measure two or three times before cutting
• Try reading the 3PAR 8000 manual to see if there are any gotcha's I'm missing like position of the screws for SATA or ...

 

[Zed1930]

So going down the rabbithole of 3PAR documentation I noticed that the LED status of the disks was amber, indicating "defective" disk. I went to find an old SAS disk and now I get a green LED status on that tray, but still don't see it in TrueNAS, even after a "camcontrol rescan all"

So:

• Either the 3PAR (or it's last config?) doesn't like SATA
• Or the 3PAR doesn't like 512 disks
• Or the enclosure controller is acting as a sort of "raid" controller and obfuscating something from TrueNAS's perspective so it can't read the individual disks?

going to keep troubleshooting...

 

[Zed1930]

Ok I found the command "getencstat -v /dev/ses1" from https://people.freebsd.org/~mav/Enclosure_Management_en.pdf

ses1 is the device name I see when I turn on the 3PAR.

So I can see the 3 disks I have plugged in right now!
I wonder if the 3PAR controllers remember a previous config as one of the SATAs is Critical, while the other is swapped? Or Maybe it's cause I tried re-seating the second SATA drive.

And I wonder if it's a driver issue finally because some of the "SAS" stuff on the enclosure is unsupported or not installed. Or maybe it's because I only have one SFF-8644 connected now to eliminate sources of problems and simplify the setup.