ESXi/FreeNAS AIO bottleneck troubleshooting

[Eds89]

I do have other VMs running, but they are almost completely idle;
pfSense router for low use internet traffic
Domain controller for only a handful of clients
Remote desktop gateway server, only used when I'm working remotely
And a couple of others doing very little

 

[Eds89]

Is there any way to definiteively ascertain if the CPU is causing these bottlenecks?
Looking at the CPU graph in FreeNAS, I'm not sure how accurate it is. I often see spikes up to 150 or 200%, which I can't see as being possible?

Other VMs aren't really doing a lot. I could assign more cores to FreeNAS, as ESXi shows CPU usage as really very low.

I have had my eye on a 2667v2 for a while, but don't want to drop £250 on one if it isn't likely to make much difference performance wise.

Cheers

 

[kdragon75]

Look at CPU Ready on the esxi host. Lower percent is better, like below 5. Search VMware CPU Ready esxtop. You may get better performance with less cores.

 

[HoneyBadger]

Short version:

esxtop -> c -> V -> f -> abe -> q

esxtop to start it, c for CPU view, V (capital V here, use Shift) to show VMs only, f to select fields, abe to pare down the columns, q to get back to the view.

What you want to watch are the values of %RDY and %CSTP

%RDY is "CPU ready" or how often the vCPU is waiting to be scheduled cycles on the pCPU. If this is too high (>5%) there are too many vCPUs for the number of physical threads/CPUs on your host.

%CSTP is "co-stop" or how often a vCPU is waiting on other vCPU threads to complete (or be available simultaneously). If this is too high (>3%) you should consider reducing the vCPU count in that specific machine.

 

[toadman]

I feel like there could be a bus issue somewhere, as I am now trying with the three SSDs in a stripe, with sync disabled, and still only manage around 700MB/s read and 450 writes.

The drives are all connected to a 9207-8i, which will be in a PCIe 8x slot, but the 700 to 750MBs matches the 6Gb/s interface of SATA, so wonder if related?

The 9207-8i has a SAS2308, which is a 6Gb/s controller to the drives (SSDs). That said, I've seen folks bench about 1.2GB/s on a similar card (9200-8e) with three much slower SSDs.

So yes, something else is a bottleneck.

 

[Jessep]

The 9207-8i has a SAS2308, which is a 6Gb/s controller to the drives (SSDs). That said, I've seen folks bench about 1.2GB/s on a similar card (9200-8e) with three much slower SSDs.

So yes, something else is a bottleneck.

Edit:
Never mind, it should be 6Gb per port not total if direct connect to controller.

/off to make coffee

 

[toadman]

More testing, and a 2 SSD stripe is giving me the same read speeds as a single SSD?
The read on each drive drops to 250MB/s. Should I not be expecting close to a GB/s?

That is very odd indeed. Yes, you should expect better performance. For clarity, this was a ZFS stripe of two SSDs attached to the HBA which is passed through to the FreeNAS vm? And the testing was done within the FreeNAS vm (eliminating esxi networking, which may still be an issue to solve later)?