Full install on HPe server, P408i-a Controller - Works but errors

[Rod Martin]

I'm curious. I have installed FreeNAS directly on a HPe DL380 Gen10 server. I intend to use VMs in FreeNAS to drive PFSense, Unifi Protect (ubituiti products), and Unifi Controller. THe install of FreeNAS seemed to go fine. I was able to make a pool and all good. Pools reports "Healthy"

However, I hear repeated scans of the drives, and continual errors in the console.

[ERROR]::[177:655.0][CPU (MANY NUMBERS HERE)][os_aio_response_error][443]:SCSI_STATUS_CHECK_COND sense size 20 (sometimes it says 18)

Any idea on this error? Do I need to have my controller configured differently in BIOS?

Configuration:

HPe DL380 Gen10, Xeon 5218 CPU, 32GB RAM
P408i-a Controller
5x 1.8TB SFF SAS2 Drives

FreeNAS is installed on a MicroSD card installed on the motherboard, 64GB.

I'm sure there is more info you might want in order to help. Sorry I dont know to include everything that may be required without some kind request. Please be gentle.

 

[HoneyBadger]

P408i-a Controller

Likely your problem. This is a hybrid RAID/HBA controller.

Have you left the drives unconfigured in the BIOS, or did you make a virtual disk/RAID?

Leaving them unconfigured should cause the controller to behave in HBA mode and present them directly. Driver support will likely not be as robust as an LSI/Avago/Broadcom HBA though.

 

[Rod Martin]

I wanted to do all of the configuring in FreeNAS, so I left the controller and the drives completely unconfigured in BIOS.

Really, everything looked fine in FreeNAS. the drives show up, I can pool them, and it all looks perfect. If I didnt see the drives scanned every few seconds, and see the errors in the console, I wouldnt have anything to tell me things werent perfect.

Also worthy of note, I'm using 11.3 - RC1. I wanted to test this with all of the latest compatabilities.

 

[HoneyBadger]

There's an discussion in FreeBSD 12.0 here:

238149 – smartpqi error messages [os_aio_response_error][442]:SCSI_STATUS_CHECK_COND sense size 22

bugs.freebsd.org

As well as another thread here:

Errors with Adaptec-Controller

Hi @ all, I hope this is the right sub-forums for my question. The Backgroundstory: I am currently evaluating a FreeNAS-Build (Hardware in my signature) as a dedicated Storage System for Windows and Linux-VMs, and Files behind a Windows Standard Server 2016. The plan is to make 2 VDEVs as a...

www.ixsystems.com

It seems to suggest that the driver isn't interpreting things from the SMART data, or it's throwing a warning/alert that needs to be interpreted somehow. They're both showing Adaptec/Microsemi RAID cards as a common factor, and IIRC that's the same SoC at the heart of your HP's P408i card, so this could be driver-related.

Is throwing an LSI HBA in there a possibility? Reflashing a Dell H200/H310 or HP H220 is fairly easy, and those cards are available rather inexpensively; just make sure to get one that has the right SFF port style and orientation for your server.

 

[Rod Martin]

I was noping not to need to add a card. I was wanting to be able to add more drives, and I'm limited on slots. WIth the addition of a 10gbe card, and a 2-bay gtx card for passthrough for a VM (If you can do this, you can in ESXi... havent tested FreeNAS VM yet for this).

I know that card you mention is always referred too. If I add another two bays of drives, I'm going to run out of controller space. I was hoping there was something that would address this.

What you say about SMART does make sense.

I found a SMART service in FreeNAS I stopped it, and turned off auto start and rebooted. I thought for sure that would end the messages, but it did not. Drives are scanned every few seconds and errors still appear.

 

[Rod Martin]

I was wrong... the drives did a SMART scan and errored on boot, but no additional messages appeared. I started the service manually after boot, and errors flipped up immediately. I turned it back off.

So, there is still a scan on the drives every few seconds. But, no further errors on the console. The question is... are the drives supposed to do that even when there is no activity??

 

[HoneyBadger]

This is one of the downsides to the RAID/HBA hybrid cards that's often stated - SMART data isn't possible to be collected, or has to be couriered through the RAID card driver. You might be able to manually query it in a different way from the shell prompt, but it may not be able to be integrated with the FreeNAS UI for easy alerting.

So, there is still a scan on the drives every few seconds. But, no further errors on the console. The question is... are the drives supposed to do that even when there is no activity??

If by "scan" you mean "a small amount of drive activity" then yes - ZFS will by default flush any pending writes (including the "everything is okay, this transaction group was empty" update) every five seconds, even if no outside data was written. Think of it as the "resting heart rate" of ZFS. This is normal and expected behavior.

 

[HoneyBadger]

Sidebar: GPU passthrough under bhyve appears to work fine (although you'll have to fight with NVIDIA's drivers under Windows only wanting to run Quadros in a VM if you choose one of their cards) but passing through USB input devices seems to be the sticking point. Other users seem to have used add-on USB cards (which complicates your lack of PCIe slots as well) but if you can find a workaround temporarily in order to install an RDP/VNC style product, you might be able to get away with doing basic keyboard updates over VNC until you can get your GPU-streaming solution (Steam in-home streaming, I'm assuming?) going.

 

[Rod Martin]

VMWare ESXi environment really works well with VMs and passthrough. I have successfully tested taht environment. I would have gone with that, except with the caveat about FreeNAS. I wish that FreeNAS would work happily in that platform. Everything Ive read is that it is temperamental and risky. I'm struggling to even get a simple VM to work right in FreeNAS, even when following procedures exactly as described in a couple of youtube vids.

 

[HoneyBadger]

That might come down to the fact that bhyve is simply a (much) less mature hypervisor than VMware, Hyper-V, or KVM.

 

[blueether]

VMWare ESXi environment really works well with VMs and passthrough. I have successfully tested taht environment. I would have gone with that, except with the caveat about FreeNAS. I wish that FreeNAS would work happily in that platform. Everything I've read is that it is temperamental and risky. I'm struggling to even get a simple VM to work right in FreeNAS, even when following procedures exactly as described in a couple of youtube vids.

I have run freenas in ESXI for a while but I found that it was ESXI that seemed unstable for me, it would hang when VMs were under load - not sure why but it was a real pain in the ....

I moved to proxmox to host my VMs now and not had any problems
Server #1 (Main)

• pfSense
• FreeNAS 11.2- with pass through for the boot disk and PCI passthrough for the HBAs
  • Has a diskshelf attached through the HBA
• Windows 10
• TrueCommand

Server #2 (Test)

• FreeNAS 11.3 testing
• FreeNAS 11.2 test
• XPEnology (never fully installed)
• ...

FreeNAS can run as a VM but you have to take care with what you do and how it's set up, there are a few bod on here that know how to do it well and should be of some use if you wanted to go down that route

 

[Rod Martin]

Great reply Blueether,

I have not considered Proxmox. My goal is to provide a good quality server, with a strong set of features in one box, that does not have a continual monthly subscription. Something that is deployable to a small business that is not just overrun with cash, or a non-profit environment. A single outlay with minimal upkeep and no monthly subscription fees if at all possible.

I want to provide in one box pretty much what you show:

- pfSense linux VM
- NAS (FreeNAS, or anything that is stable)
- unifi Protect (for camera system) linux VM
- unifi Controller (for wifi) linux VM
- Windows10 with PCI passthrough for a GTX card (Not Quadro)


I'm curious how my HP HBA will work with the drives. I'll report back. Unless my post gets nuked for going off FreeNAS topic.