Extremely High z_rd_int CPU Usage Under Load

[xioustic]

Image summarizes the issue relatively well.

When downloading data from TrueNAS, `z_rd_int` causes CPU usage to go through the roof. I suspect that's not supposed to happen. I'm also guessing this is why I'm getting much worse read speeds than I'd expect. This also suggests running Plex or other containers simultaneously is not going to be feasible on this setup.

Some info:
- Running TrueNAS-SCALE-20.10-ALPHA
- Board is a SuperMicro X9SCL/X9SCM
- 8-core Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz
- 32GB RAM
- Disks are connected via native SATA ports
- The pool is encrypted.
- Pool layout is effectively a RAID0 of 2x12TB drives (this is a test node, thus the "caution to the wind" layout and low RAM vs Pool Size)
- Connection to client from TrueNAS server is a 10Gbit SFP+ connection; however, running this same test over the built-in ethernet cards has the same exact issue
- EDIT: No, a scrub is not happening

I'm guessing either the CPU is missing some native instructions to keep with encryption or some other feature, this machine is under-spec'd, and/or I'm missing something driver/hardware-wise that Linux needs installed. Hopefully it is all solved by the latter.

Thanks!

 

[xioustic]

Pretty sure I have this fixed. After struggling to get into the BIOS (hooray for the Supermicro 2021 bug!) I did in fact find AES-NI disabled. I toggled it and suspect this will resolve the issue. Will report back if the headache still exists.

EDIT (since I don't want to bump again): Stress testing this and it still pushes the CPU, but each instance hits 6-7% max instead of the 20% before. Might be still high but this is enough improvement for me!