MPS (LSI) hw.mps.max_chains?

[MrAkai]

On my system (Dual Xeon, 256GB RAM, FreeNAS 9.2.1.5, 9.2.1.7 shortly) with two LSI cards (MPS drivers), One(mps0) is SAS2008 based for internal drives (boot drives, cache) and the second(mps1) is SAS2308 (for two 40 disc JBODs + 2 Zeusram ZILs)

I noticed in the dmesg output the following:

mps1: Out of chain frames, consider increasing hw.mps.max_chains.

The max_chains is set to 2048 (the default) and I haven't been able to find a lot of information about this situation online.

So my two questions for this esteemed body are:
1> Does bottoming out the chain frames have a big impact? I would imagine there would be a performance hit,but I'm more worried about data integrity.
2> Are there any caveats to simply doubling the # of DMA chain frames? After some of the issues with the Intel 10GB nics, I'm a bit gun shy on tunables :)

Hopefully I've provided enough information, but if I left anything out, please feel free to ask me

Thanks!

 

[cyberjock]

Yeah, I don't have the answers you are looking for and I'm not sure anyone here does. Your questions are probably better asked in either FreeBSD forums or to LSI themselves.

 

[Deleted User #319254452]

This is the best information I have found to date:

https://lists.freebsd.org/pipermail/freebsd-stable/2016-March/084316.html

"The message is harmless, it's a reminder that you should tune the kernel for your workload. When the message is triggered, it means that a potential command was deferred, likely for only a few microseconds, and then everything moved on as normal.

A command uses anywhere from 0 to a few dozen chain frames per I/o, depending on the size of the io. The chain frame memory is allocated at boot so that it's always available, not allocated on the fly. When I wrote this driver, I felt that it would be wasteful to reserve memory for a worst case scenario of all large io's by default, so I put in this deferral system with a console reminder to for tuning.

Yes, you actually do have 900 io's outstanding. The controller buffers the io requests and allows the system to queue up much more than what sata disks might allow on their own. It's debatable if this is good or bad, but it's tunable as well.

Anyways, the messages should not cause alarm. Either tune up the chain frame count, or tune down the max io count."

I am seeing this on two systems with multiple pools. I tried:

System: Tunables: Add Tunable

Variable: dev.mps.0.max_chains
Value: 4906
Type: Sysctl

... and double that, but the message did not go away.

 

[Deleted User #319254452]

The accompanying ticket: https://redmine.ixsystems.com/issues/28235

Thank you Jamie for reporting this.