Slow write speed on ESX5.1U1

[Thomymaster]

Hi All


I am pretty new to FreeNAS and set it up on my ESXi environment (5.1U1). I have the problem that within my Windows 7 VM i have veryl low write speeds on 4k read/write approv 30/10MB/s (measured with CrystalDiskMark). Here is my configuration:

Build FreeNAS-8.3.1-RELEASE-p2-x64 (r12686+b770da6_dirty)
Platform Intel(R) Xeon(R) CPU 3040 @ 1.86GHz
Memory 2031MB

HDDs: 4x Seagate Barracuda 7200.11 (ST31500341AS) configured as a RaidZ2 with "force 4K sector"
ZIL: 64GB SSD
L2ARC: none yet

here is my first question:

- how can i configure my SSD (like partitioning) to be used as ZIL and L2ARC ord doesn't this make sense?

My ESXi is connectied via ISCSI to my FreeNAS (JumboFrames enabled to reduce the overhead), using MPIO doesn't improve the read/write speed (i tested already).

In addition, when i do read/write onto the FreeNAS, it gets to a maximum of 100%cpu usage, but as i have 2 CPUs and the total maximum is 200% i assume that it is limited to 1 CPU. Why?

What can i do to improve this situation, add more CPU and/or RAM?


Cheers

Thomy

 

[Thomymaster]

I was told to remove the ZIL and add more RAM (at leaset 8GB) so i'll do this next week and watch for increased performance.

 

[jgreco]

My vote, 16GB of RAM and things will seem much better. We just finished hashing over the effects of sync vs async on one of the other forums.

 

[Thomymaster]

I'm interested in the effects of sync vs. async, where can i find the thread in the forum? :)

 

[jgreco]

Up in the right hand corner, the box next to the magnifying glass, type "sync async" and hit return.

 

[Thomymaster]

I upgraded the RAM to 8GB and used the SSD as L2ARC instead if a ZIL and this greatly improved the performance.

However when i write or read to the disks, the istgt process aquires 50% of CPU load in "top" (there are 2 XEON 5060 CPUs in there with 4x3GHz). So is this then a bottleneck of the istgt because its single-threaded (or does 50% cpu mean it only uses one core out of the 8 at full load)?

The output of gstat looks quite good during read/write, i see approx 60% utilization but nothing red.

Cheers

Thomy

 

[jgreco]

How busy is the pool? Just a single iSCSI connection?

 

[Thomymaster]

It is only a single iSCSI connection to my ESXi server (datastore with all my production VMs)

 

[jgreco]

So is it a Xeon(R) CPU 3040 as you initially reported, or a dual 5060 (which is a dual, not quad, core cpu, so two 5060's is four cores total)?

Either way, it seems to be a poor outlook. The 3040 earns a score of 1100 on this benchmark site. I don't see any easy benchmarks for the 5060 (Dempsey) but the 5160 (Woodcrest) is usually about 20% faster. So a 5160 gets a score of 2189 on the benchmark site; 80% of that is 1751. By way of comparison, the modern Xeon E3-1230, a modest modern Xeon, gets rated at 8163. But that's four cores, so adjusting again that'd be 4081, except that the E3-1230 also has a turbo mode which it'll engage if a single intensive thread is encountered, so really it's a little more.

So my suspicion is you're running out of oomph, and maybe what you ought to try is what happens if you can get the ESXi host and the filer talking over a second connection to see if you can tease more cores into meaningful use.

 

[Thomymaster]

The initial setup was 2 XEON 5060 CPUs. I tried to create a second network connection and enable MPIO (via Round Robin) in ESXi so that there are 2 paths (on different subnets) to my FreeNAS. However this didn't improve the situation.

So my question here ist, can i not get istgt to run in multi-threaded mode?

 

[jgreco]

I'm actually not sure. You appear to have done what I would have done. I believe istgt has various modes but don't know offhand what FreeNAS sets up for.

Maybe see http://lists.freebsd.org/pipermail/freebsd-scsi/2011-February/004815.html which implies that at least it ought to be able to support multiple threads.

 

[pbucher]

I can report that NFS with ESXi balances nicely across all processors. Just make sure to set your NFS server count to match you # of CPU cores.

Also you might want to read the various threads here about NFS & ESXi, but it sounds like you should be good to go with having a SSD for your ZIL.

 

[Thomymaster]

Hi

I changed the 2 XEON 5060 to one E5345 (quad-core) and now i am seeing a quite better performance see attached screenshot

 

[Thomymaster]

I changed the 2 5060 XEONs to one E5345 quad-core XEON and now the performance seems to have increased a lot :)