IRQs do nasty things performance-wise to the system. Everything I'm about to explain is hardware-based and not OS specific...
When an IRQ is initiated from a device(NIC, USB, whatever) it halts CPU processing to tend to the particular device. This is great because it ensures that devices are using as much resources as they need and don't use resources when they don't(they don't send IRQ requests when the device isn't being used), but a misbehaving device can cause major performance issues because it continually interrupts the CPU. If a device starts having issues it may start trying to tell the drivers that it is having errors. If its having 100k+ errors per second because of some repeating issue then you have a problem. CPU usage will go up to try to tend to that device which is only telling you that it is having an error condition over and over and over. Having multiple cores helps because an IRQ can only be processed by 1 core at any given time, so having just 2 cores helps alleviate IRQs because you will always have 1 core that isn't dealing with IRQs.
USB is a major pain because your USB devices uses IRQs, and with USB3 you only increased the interrupts when transferring data(that is now up to 5Gb/sec). So now that great 5Gb/sec you want through USB3 is also slowing down your system. Not that good of a trade-off eh? Firewire on the other hand uses DMA so no IRQs are needed. This gives Firewire a distinct advantage in some ways. DMA was requested by many manufacturers to allow USB3 to have higher performance without killing the system performance at the same time. This was rejected by Microsoft and a few of the other big manufacturers because it would require more work to make it backwards compatible with USB2/1.1/1.0. However, a semi-DMA version is implemented in USB3 because initial testing found that even your fastest CPUs from 2011 couldn't actually dish out or receive the total bandwidth of USB3 because IRQs would kill system performance. Whoops!
But all isn't rosy with DMA devices either. DMA devices have direct memory access to your system when plugged in. This is handled in hardware, so no drivers or anything your OS can do can prevent this. So theoretically a "rogue" device could be plugged into your firewire port and the entire contents of RAM(passwords, encryption keys, etc) can be downloaded to the device and you can't do anything about it. You don't like this potential loophole, disable Firewire in your motherboard BIOS. I disable it because I don't use it and I'd rather not deal with the IRQ penalties. But if you use something like Truecrypt or FreeNAS' AES encryption and you don't have complete physical control of the Firewire ports at all times, this is a potential security risk you must accept. Some people are so paranoid that they believe that their government is out to get them and sometimes won't even buy motherboards that have Firewire built-in because they're paranoid that disabling Firewire in the BIOS might not stop someone from trying to get to their precious data. There have been 1 or 2 brands where this was found to be true, so their paranoia isn't completely unfounded.
IRQ savings is one of the ways that you can help increase performance. Disable unused devices and make devices that you do use more efficient. Using things like jumbo packets can save you some performance because instead of potentially lots of smaller IRQs from your NIC you have a few larger ones. Using larger network buffers(hardware) can also help this. Intel NICs also support DMA I believe so you can see performance improvements there too. I've often wondered if Intel supporting DMA was the single biggest reason why their NICs perform better with less than optimal hardware.
In any case, interrupting a slower CPU results in a larger performance penalty. Take an old P4 and put a Gb LAN card in it and try to copy 100MB/sec. You'll see CPU usage go really high(sometimes above 90%!), and getting 100MB/sec is really hard.
Edit: To answer your question(since I gave lots of info but no straight answer) you should look to see where the IRQs are coming from. It could be that some device is trying to connect to the driver 100s of times a second but since FreeBSD doesn't support that device you're losing performance from it. It could also be that the NIC is just making alot of requests and that's completely normal. Unfortunately, I have no clue how you'd find an offending IRQ device. But now that I'm thinking about this I really hope someone posts some info regarding IRQ monitoring in FreeBSD. I have a FreeBSD Atom box and I'm curious now...