What's all the noise about HBA's, and why can't I use a RAID controller?

[zbyte]

Lol, thanks, I am using the latest ESXi7, I would have go with Promox but I need VMware for certain enterprise VM's which are distributed in that format so ESXi is a must. This is for home and I can't have a dedicated TrueNAS appliance and separate Hypervisor so 1 small box will have to do

Found your post, I will read a few passages

thanks again

 

[zbyte]

@jgreco I went ahead and just completed reading your 2012 post, it is obsolete technology but valid conceptually, I understand the possible drawbacks and variables, I will think about it, my problem is this data although non mission critical I would like to preserve it and if this is important it might be best (like you stated) to run Free/True NAS on bare metal hardware and not on a VM even at the increased cost of two boxes .... thanks for the reference

 

[Yorick]

I seem to remember that ESXi 7 does not support LSI 9200 any more. It can still be made to work but might need a swift kick after every reboot. 9300 is the safer bet.

 

[Heracles]

I seem to remember that ESXi 7 does not support LSI 9200 any more. It can still be made to work but might need a swift kick after every reboot. 9300 is the safer bet.

Indeed, ESXi7 dropped support for non-native drivers. That means controller card driven by the megaraid driver like my Perc H700 can not be handled by the OS itself. Still, the goal here is to have it transferred to a VM using PCI passthrough. For that, ESXi does not need to understand the device or have any driver for it.

Here, my R820 is still running ESXi 6.7 U3 for that reason : for ESXi to manage its local storage by itself.
Should I wish to move to ESXi7, the way I would have to do would be to deploy a TrueNAS VM and give it the controller with PCI passthrough. Then I could do a virtual iSCSI link between it and ESXi and it would be that way that local storage would become available.

Another thing that is gone in ESXi7 is VNC access to VMs, but that is another story...

 

[Herr_Merlin]

If you have enough money to spend go for the LSI 9400 or 9500 Tri-Mode HBA (NVMe,SAS,SATA) :) can only recommand!

Host Bus Adapters

Broadcom Host Bus Adapter (HBA) cards can enable an easy, long-term storage growth strategy in practically any direct-attached storage scenario.

www.broadcom.com

 

[Ericloewe]

Any experience with the FreeBSD driver for them? They were somewhat flaky around launch.

(Sidenote: At LSI prices, throwing a dual-Epyc system at the problem to get a truckload of NVMe disks might be a better option for anything beyond what the card can do without expanders)

 

[Joe_Papa]

I have an HP DL380 G9 server running ESXI 7.0 that I plan to put FreeNAS on. It has 12x 3.5" 1TB drives and a P840 raid controller. From my reading on this forum I know that I need to replace the raid card with an HBA card or two... What would you recommend for my scenario? I plan to add an SSD for boot drive and some nvme drives for cache if that matters.

 

[Herr_Merlin]

Simple answer to those people wanting to use RAID Controller: https://www.youtube.com/watch?v=CXArovLJ60A

 

[Yorick]

I have an HP DL380 G9 server running ESXI 7.0 that I plan to put FreeNAS on. It has 12x 3.5" 1TB drives and a P840 raid controller.

We have a discussion about the cabling and options:https://www.truenas.com/community/threads/hp-gen9-server-w-p840-hba-mode-no-drives-visible.28620/

That person went with two controllers: “I ended up buying 3x SFF-8087 cables and two LSI 9207-8i cards, flashing the firmware, and that got me in business.”

9207 is 6gb sas, which is plenty for hard drives. If you do want 12gb, it’d be the 9300-8i or 9302-8i or 9300-16i if you want just one card.

you say “NVMe for cache”, that’s often misunderstood. L2ARC may not benefit you and you will want to max ARC first; SLOG is not a cache and only benefits sync writes (NFS or iSCSI from VMWare). We have good articles on when to use an SLOG and which drives make good SLOG drives.

 

[Joe_Papa]

Thanks for this. I have read a lot of the guides on the forum here, but I have a way to go before all the new terms sink in. :)

 

[ChrisRJ]

As a tangent to my tangent (which I guess is the same tangent, by definition of tangent), are there non-optical ATAPI devices? Is there some obscure piece of hardware out there that uses ATAPI and isn't an optical disk drive?

In the mid 1995s there were a few home-use tape drives with an ATAPI interface ;-) Had one, total crap and waste of money ...

 

[Ericloewe]

I noticed that Supermicro is shipping the SAS 9500 / SAS3808 in some new SuperStorage units, but I can't seem to find it in the FreeBSD mpr driver. Can anyone shed some light on this? Is it not yet supported, not documented...?

 

[Johnny Fartpants]

I noticed that Supermicro is shipping the SAS 9500 / SAS3808 in some new SuperStorage units, but I can't seem to find it in the FreeBSD mpr driver. Can anyone shed some light on this? Is it not yet supported, not documented...?

Is there anymore news on this and if this is supported on the latest version of TrueNAS Core?

 

[jgreco]

Is there anymore news on this and if this is supported on the latest version of TrueNAS Core?

UTSL

freebsd-src/sys/dev/mpr/mpr_pci.c at main · freebsd/freebsd-src

The FreeBSD src tree publish-only repository. Experimenting with 'simple' pull requests.... - freebsd/freebsd-src

github.com

 

[cstephan]

I have a SuperMicro 6036ST-LR chassis (Nexgen N5) dual-node shared storage bridge chassis. It uses a LSI-SAS2008-IR mezzanine card in each node to attach to a 16 disk shared SAS Expander on the front of the chassis.

I wanted to follow up, I had issues where the disks would not show up in the UI but would report in `camcontrol -devlist` and I received unique serial numbers for each disk using `camcontrol -S <devname>`.. smartctl also was able to read the smart data from each disk as well.

Assuming this was a firmware issue (I was on 11.00.01.00), I upgraded to 20.00.04.00 on each LSI SAS2008-IR mezz.

Retreieved DOS firmware at supermicro.com. I copied all the rom files in the IR firmware onto a freshly installed 13.0-U3 instance on each node and translated the `sas2flsh.exe` commands in the bat file to `sas2flash` commands on TrueNAS. All went well. Still no disks in the UI.

I was about to jump to IT firmware, when I noticed that DNS wasn't resolving at the CLI and middlewared was failing to start in the logs. There were also a bunch of failures to install plugins.

I updated the DNS configuration in my DHCP server, /etc/resolv.conf and in the UI to point to 1.1.1.1 and rebooted. This time, the primary node once rebooted began installing plugins and eventually middlewared completed its startup process. Now all disks are visible in the UI.

I am not sure if I was just being impatient and TrueNAS hadn't completed its installation, or if the lack of DNS was an issue. Either way, all is well now.

 

[jgreco]

20.00.04.00

Please note that you should be using 20.00.07.00. IR is OK, IT is preferred. I no longer recall what the fix was, presumably minor, but quite possibly not negligible.

 

[pixelwave]

I happen to have a LSI 9250-4i card installed. Unfortunetly it seems not to be compatible with the 2114/2118 it mode image. Is there a compatible firmware for that card .. already tried my luck at broadcoms page ...

UPDATE: Ok .. so after digging deeper it seems the 9250 is (as the 9260) using the LSI SAS2108 chip and apperently there never was an IT mode for that specific one. I guess I am going to get an 9211 instead ...

 

[Ericloewe]

Realistic answer: Get an SAS2008/2308

Craziness: Enterprising hackers have figured out how to load IT firmware on SAS2108 controllers, but the only widely-known method I've seen is for Dell cards only (and thus useful mostly to allow for the internal PCIe slot on Dell Gen 12 systems to be used with an HBA).

 

[pixelwave]

Also saw this post of reverse engineering the SBR:

Crossflashing the Fujitsu D2607

marcan.st

I ordered now a LSI 6 Gbit/s SAS HBA 9211-8i with P20 IT-Modus already flashed on ebay. Also gives me 8i instead of 4i .. which is nice to have.

 

[Ericloewe]

Nice find, that sure makes things a lot easier.