So let's assume that peak transfer is a fringe case - let's go for real world being 25% of peak - even with the slowest of DDR3 RAM (and what I have is the fastest of DDR3 at 1333), that's still 1,600MB/s, and a single core of a X5670 is supposedly capable of ridiculously higher than that (never mind 12 cores, since the 12 hyperthread cores don't count towards AES-NI). That's where I'm scratching my head. Don't get me wrong - I'm not expecting the disk arrays to be able to pull 20GB/s or anything - that's just not possible (without an array of SSDs, and even then, maybe). I could FULLY accept if the test (which supposedly is fully on the motherboard - totally in the RAM and CPU(s)) said the AES throughput was 2GB/s (just throwing a number out there) but the 'real' performance was 512MB/s - disks & controllers play heavily into the overall real performance - but the test is supposedly 'synthetic' & theoretical, pushing only the fastest components to their limits (CPUs, MB, RAM), leaving the controller(s) and disks out of the picture, so I'm scratching my head as to why the number is so low for this particular test. When I ran the tests on the x5670's, the system had 'just' booted, with none of the VMWare guest load on it at all (the L5630 test was done with all the VMWare guests up and running as normal), so contention should have been non-existent for both X5670 tests. Where could there be a bottleneck for this test? The CPUs are pretty darn fast, the memory is the fastest available for these CPUs... unless the Supermicro motherboards are flaming piles of junk, but I have a hard time seeing that being the case as Supermicro is a fairly big name...
I mean, overall, it's running well - even prior to fixing the clocking issue, it did seem to be running a bit better with the X5670's than it was with the L5630's, as I had several hiccups suspending and shutting down VMs with the L5630's that weren't there with the X5670's - the numbers from the test are just a bit perplexing to me. Based on the benchmarks 'out there', I expected a bigger jump on the test with the swap since it's supposed to be all within the CPU and RAM (I wouldn't expect the same jump with a test that involves the controller and drives as the CPU and RAM should be able to outperform the disks multiple times over unless you're running a P166 or something equally antiquated).