System Load Vs CPU Usage on report tab

Status
Not open for further replies.

Lucas Rey

Contributor
Joined
Jul 25, 2011
Messages
180
Dear community,
could someone explay which is the differences between System Load and CPU Usage and report tab?

This is System Load on my NAS during FULL BD play (from CIFs) and 8Tb file copy to nas (even using cifs):

immagine001mq.png


And this is the CPU Usage at the same time:

immagine002r.png


Any clarification? :confused:

Thanks
Lucas
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525

Lucas Rey

Contributor
Joined
Jul 25, 2011
Messages
180
Ok thanks cyberjock, got it. Btw, basing on my statistics, is not clear for me if I can use a lower cpu clock.
This because I would like to change my Intel(R) Atom(TM) CPU D525 @ 1.80GHz with AMD APU C-60 1.0GHz Dual-Core
 

JaimieV

Guru
Joined
Oct 12, 2012
Messages
742
If you look at just the CPU usage, that shows that you never get over about 75% (of 200%) CPU used, so you'd think you would be okay going to the slower CPU.

But unfortunately the CIFS sharing software only uses one thread, so if you're using Windows shares then really you're using 75% of 100% (one core). Dropping the speed of the CPU by nearly half would mean that you might get stutters in watching bluray movies. Not much though since your average is way lower than 75%, so you may see any problems.

Try it and see!

(Side question for the experts: the IRQ CPU usage seem really high... is that something to do with the Atom platform in particular? I've never used Atom, and my Core2 and AMD-based FreeNAS IRQ CPU's are more like 0-2% even with very sub-2GHz CPUs)
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
IRQs do nasty things performance-wise to the system. Everything I'm about to explain is hardware-based and not OS specific...

When an IRQ is initiated from a device(NIC, USB, whatever) it halts CPU processing to tend to the particular device. This is great because it ensures that devices are using as much resources as they need and don't use resources when they don't(they don't send IRQ requests when the device isn't being used), but a misbehaving device can cause major performance issues because it continually interrupts the CPU. If a device starts having issues it may start trying to tell the drivers that it is having errors. If its having 100k+ errors per second because of some repeating issue then you have a problem. CPU usage will go up to try to tend to that device which is only telling you that it is having an error condition over and over and over. Having multiple cores helps because an IRQ can only be processed by 1 core at any given time, so having just 2 cores helps alleviate IRQs because you will always have 1 core that isn't dealing with IRQs.

USB is a major pain because your USB devices uses IRQs, and with USB3 you only increased the interrupts when transferring data(that is now up to 5Gb/sec). So now that great 5Gb/sec you want through USB3 is also slowing down your system. Not that good of a trade-off eh? Firewire on the other hand uses DMA so no IRQs are needed. This gives Firewire a distinct advantage in some ways. DMA was requested by many manufacturers to allow USB3 to have higher performance without killing the system performance at the same time. This was rejected by Microsoft and a few of the other big manufacturers because it would require more work to make it backwards compatible with USB2/1.1/1.0. However, a semi-DMA version is implemented in USB3 because initial testing found that even your fastest CPUs from 2011 couldn't actually dish out or receive the total bandwidth of USB3 because IRQs would kill system performance. Whoops!

But all isn't rosy with DMA devices either. DMA devices have direct memory access to your system when plugged in. This is handled in hardware, so no drivers or anything your OS can do can prevent this. So theoretically a "rogue" device could be plugged into your firewire port and the entire contents of RAM(passwords, encryption keys, etc) can be downloaded to the device and you can't do anything about it. You don't like this potential loophole, disable Firewire in your motherboard BIOS. I disable it because I don't use it and I'd rather not deal with the IRQ penalties. But if you use something like Truecrypt or FreeNAS' AES encryption and you don't have complete physical control of the Firewire ports at all times, this is a potential security risk you must accept. Some people are so paranoid that they believe that their government is out to get them and sometimes won't even buy motherboards that have Firewire built-in because they're paranoid that disabling Firewire in the BIOS might not stop someone from trying to get to their precious data. There have been 1 or 2 brands where this was found to be true, so their paranoia isn't completely unfounded.

IRQ savings is one of the ways that you can help increase performance. Disable unused devices and make devices that you do use more efficient. Using things like jumbo packets can save you some performance because instead of potentially lots of smaller IRQs from your NIC you have a few larger ones. Using larger network buffers(hardware) can also help this. Intel NICs also support DMA I believe so you can see performance improvements there too. I've often wondered if Intel supporting DMA was the single biggest reason why their NICs perform better with less than optimal hardware.

In any case, interrupting a slower CPU results in a larger performance penalty. Take an old P4 and put a Gb LAN card in it and try to copy 100MB/sec. You'll see CPU usage go really high(sometimes above 90%!), and getting 100MB/sec is really hard.

Edit: To answer your question(since I gave lots of info but no straight answer) you should look to see where the IRQs are coming from. It could be that some device is trying to connect to the driver 100s of times a second but since FreeBSD doesn't support that device you're losing performance from it. It could also be that the NIC is just making alot of requests and that's completely normal. Unfortunately, I have no clue how you'd find an offending IRQ device. But now that I'm thinking about this I really hope someone posts some info regarding IRQ monitoring in FreeBSD. I have a FreeBSD Atom box and I'm curious now...
 

JaimieV

Guru
Joined
Oct 12, 2012
Messages
742
Thanks CJ. I'm pleased to already know pretty much all of that, but interesting for others too!

Your edit is exactly what I was interested in - perhaps Lucas is doing something obviously IRQ-intensive such as using USB HDDs?
 

Lucas Rey

Contributor
Joined
Jul 25, 2011
Messages
180
perhaps Lucas is doing something obviously IRQ-intensive such as using USB HDDs?
This is my actual config:
Intel(R) Atom(TM) CPU D525 @ 1.80GHz
4 x 1Tb 2,5" in RAIDZ1
1 x 16Gb 2,5" SSD (where I installed freenas) - USB CONNECTED
2 x GigaBit Adapter (configured in link aggregator). One of them is on PCI interface, the other one is onboard
8 Gb DDR2 (2x4)

So, the only USB connection is for FreeNas os disk, could be this one the root cause for high IRQs?

And yes, thanks cyberjock for your great IRQ explanation!
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
This is my actual config:
Intel(R) Atom(TM) CPU D525 @ 1.80GHz
4 x 1Tb 2,5" in RAIDZ1
1 x 16Gb 2,5" SSD (where I installed freenas) - USB CONNECTED
2 x GigaBit Adapter (configured in link aggregator). One of them is on PCI interface, the other one is onboard
8 Gb DDR2 (2x4)

So, the only USB connection is for FreeNas os disk, could be this one the root cause for high IRQs?

And yes, thanks cyberjock for your great IRQ explanation!

There's no way to even take an educated guess in a forum setting unless you have some error message that clues you in. Each system is independent. If your USB stick is having problems it could be. But it could also be completely normal for your hardware. The only way to figure out if anything is wrong is to do your own scientific discovery and research.

I will tell you that using NICs over PCI is a big mistake. A single Gb LAN card can exceed the total bandwidth for PCI, and your PCI bus is likely being shared with other hardware on your motherboard. Not to mention that an Atom with ZFS can't even do half of a single Gb NIC, I have no idea why you think you'd need 2.
 

Lucas Rey

Contributor
Joined
Jul 25, 2011
Messages
180
I will tell you that using NICs over PCI is a big mistake. A single Gb LAN card can exceed the total bandwidth for PCI, and your PCI bus is likely being shared with other hardware on your motherboard. Not to mention that an Atom with ZFS can't even do half of a single Gb NIC, I have no idea why you think you'd need 2.
Oh, just for test. I have some different PC that access the nas (even at the same time), and I'd configured the laggproto as loadbalance to split traffic between interface. That's why I have 2 NICs.
 
Status
Not open for further replies.
Top