SOLVED CPU Thread 0 running at 100% ALL THE TIME

[Azsumbre13]

Hi all I am new to the TrueNas community and configuration as a whole.
It may be a stupid question ,but can someone give some ideas on why would my CPU always run at 25 percent:
ast pid: 8237; load averages: 1.03, 0.98, 0.95; b up 0+05:10:12 13:40:48
92 processes: 1 running, 91 sleeping
CPU: 0.1% user, 0.0% nice, 0.0% system, 24.0% interrupt, 75.9% idle
Mem: 338M Active, 3370M Inact, 1310M Wired, 10G Free
ARC: 622M Total, 370M MFU, 219M MRU, 4887K Header, 23M Other
511M Compressed, 1055M Uncompressed, 2.06:1 Ratio
Swap: 2048M Total, 2048M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
400 root 30 20 0 376M 267M kqread 3 0:45 0.15% python3.
4326 88 59 20 0 2330M 617M select 2 0:34 0.10% mysqld
8237 root 1 20 0 14M 4212K CPU3 3 0:00 0.03% top
3042 535 4 20 0 34M 9792K kqread 2 0:03 0.02% redis-se
3011 root 7 20 0 65M 33M select 2 0:02 0.01% python3.
1514 www 1 20 0 38M 10M kqread 1 0:00 0.00% nginx
1427 ntpd 1 20 0 21M 7144K select 2 0:00 0.00% ntpd
1456 root 8 20 0 52M 12M select 1 0:11 0.00% rrdcache
3046 root 1 20 0 616M 32M kqread 1 0:00 0.00% php-fpm
1383 root 1 -52 r0 13M 13M nanslp 2 0:00 0.00% watchdog
1523 root 11 20 0 114M 52M nanslp 2 0:23 0.00% collectd

I am running nextcloud on it ,but even before i configured and installed nexcloud i had the same issue.
I am running a baremetal:
Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
16gb RAM
with TrueNas core
TrueNAS-13.0-U5.1

Here is also top -P:
last pid: 8288; load averages: 1.05, 1.01, 0.96; b up 0+05:13:56 13:44:32
92 processes: 1 running, 91 sleeping
CPU 0: 0.0% user, 0.0% nice, 0.0% system, 97.6% interrupt, 2.4% idle
CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.5% interrupt, 99.5% idle
CPU 2: 0.5% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.5% idle
CPU 3: 0.5% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.5% idle
Mem: 336M Active, 3376M Inact, 1310M Wired, 10G Free
ARC: 625M Total, 370M MFU, 223M MRU, 4882K Header, 23M Other
511M Compressed, 1055M Uncompressed, 2.06:1 Ratio
Swap: 2048M Total, 2048M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
8288 root 1 21 0 14M 4208K CPU3 3 0:00 0.12% top
4326 88 59 20 0 2330M 617M select 3 0:34 0.11% mysqld
400 root 30 21 0 378M 268M kqread 3 0:46 0.06% python3.
3042 535 4 20 0 34M 9792K kqread 2 0:03 0.02% redis-se
3011 root 7 20 0 65M 33M select 2 0:03 0.02% python3.
1514 www 1 20 0 38M 10M kqread 2 0:00 0.00% nginx
1427 ntpd 1 20 0 21M 7144K select 1 0:00 0.00% ntpd
1456 root 8 20 0 52M 12M select 3 0:11 0.00% rrdcache
3046 root 1 20 0 616M 32M kqread 2 0:00 0.00% php-fpm

The only test I have done is upgrade the BIOS to the latest and test with windows 11 as a operating system if the CPU will get such interupts also.
Windows 11 was running at 1 to 5 percent and no interupts were seen in the test.

 

[Azsumbre13]

Here is a screenshot of the GUI.

 

[Azsumbre13]

I tried to run top -iom , but still nothing to see there :

Code:

last pid:  8537;  load averages:  0.99,  0.96,  0.92; b up 0+05:33:24  14:04:00
92 processes:  1 running, 91 sleeping
CPU:  0.4% user,  0.0% nice,  0.4% system, 23.7% interrupt, 75.5% idle
Mem: 345M Active, 3623M Inact, 1313M Wired, 10G Free
ARC: 626M Total, 370M MFU, 223M MRU, 4885K Header, 23M Other
     511M Compressed, 1057M Uncompressed, 2.07:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME     VCSW  IVCSW   READ  WRITE  FAULT  TOTAL PERCENT COMMAND
 1523 root        1110     75      0      0      0      0   0.00% collectd
 1456 root         373      0      0      0      0      0   0.00% rrdcached
  400 root          14      1      0      0      0      0   0.00% python3.9
 4326     88       328      1      0      0      0      0   0.00% mysqld
 8521 root           2      0      0      0      0      0   0.00% top
 3042    535        20      0      0      0      0      0   0.00% redis-server
 3011 root           8      0      0      0      0      0   0.00% python3.9
 2774 root           2      0      0      0      0      0   0.00% cron
 3101 root           2      0      0      0      0      0   0.00% cron
 1514 www            6      0      0      0      0      0   0.00% nginx
 1427 ntpd           2      0      0      0      0      0   0.00% ntpd
 1496 root           2      0      0      0      0      0   0.00% cron

 

[joeschmuck]

There are a few threads on this forum discussing high cpu activity.

Try this:
1. Stop any VM's you have running, yes including nextcloud.
2. Disconnect your Ethernet cable.

Does either one of these lower the cpu usage?

Your output for top -mio does not seem complete, only 12 processes?

 

[Azsumbre13]

1.Stoped all VMs
2.Disconnected eth cable.

Still the same issue.

I have read maybe all of the posts ,but nothing helped me.

top -mio , that was all the output i could copy and see

 

[joeschmuck]

How are you running the command? Are you using GUI Shell feature? Are you using SSH? You might be limited to the number of lines on your screen. Also try add the -z to not display idle processes.

The thing is, that PERCENT column should add up to the total usage you are seeing, which is 23.7%. You are not running at 100%, not with the data you posted.

 

[Azsumbre13]

I am running the commands from shell in the GUI . Also the data I am providing is that 1 CPU thread is running at 100 percent all the time , not the whole CPU usage , but only thread 0

 

[Azsumbre13]

This is top -z:

Code:

last pid: 18172;  load averages:  0.87,  0.90,  0.91; b up 0+19:40:06  04:10:42
42 processes:  1 running, 41 sleeping
CPU:  0.0% user,  0.0% nice,  0.1% system, 24.0% interrupt, 76.0% idle
Mem: 157M Active, 1146M Inact, 1235M Wired, 13G Free
ARC: 627M Total, 371M MFU, 223M MRU, 4908K Header, 24M Other
     513M Compressed, 1065M Uncompressed, 2.08:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
  400 root         30  20    0   393M   276M kqread   3   2:05   0.11% python3.
18164 root          1  20    0    14M  4096K CPU3     3   0:00   0.03% top
 1523 root         11  20    0   114M    53M nanslp   3   1:25   0.02% collectd
 1456 root          8  20    0    52M    12M select   3   0:40   0.01% rrdcache
 1514 www           1  20    0    38M    10M kqread   2   0:00   0.00% nginx
 1427 ntpd          1  20    0    21M  7144K select   1   0:01   0.00% ntpd
 1383 root          1 -52   r0    13M    13M nanslp   2   0:00   0.00% watchdog
  684 root          3  20    0   219M   170M piperd   2   1:04   0.00% python3.
  685 root          3  22    0   219M   170M usem     1   0:58   0.00% python3.
  475 root          3  20    0   253M   173M usem     2   0:54   0.00% python3.
  686 root          3  20    0   219M   170M usem     1   0:52   0.00% python3.
  765 root          3  20    0   219M   170M usem     1   0:52   0.00% python3.

Tried a few things already a few things already:
-Reinstalled Truenas
-Updated latest firmware for BIOS for this machine
-Tried with a Windows OS to see if the issue could be from the CPU
-Stopped all VMs: Nextcloud and etc.
-Reviewed if any process is being responsible with triggering top commands , but could not find anything

Even after trying everything still the issue persists

 

[sretalla]

24.0% interrupt

That points to some kind of PCI device misbehaving... can you check your PCI cards are all inserted properly?

 

[Azsumbre13]

After upgrading to TrueNas Scale latest version issue was resolved

 

[joeschmuck]

It is likely FreeBSD did not like one of your devices since the problem went away with Debian (Linux).

 

[Azsumbre13]

Possibly

 

[Azsumbre13]

It is likely FreeBSD did not like one of your devices since the problem went away with Debian (Linux).

Thanks for all you help :)