Poor write speed to ESXi from Freenas

[blaze0808]

Hello everyone.
I am experiencing a problem with my data transfer between my thunder and my esxi.

I had to migrate the information to thunder because I needed to format my esxi server.
At the time of sending the data from my old esxi to my thunder, I had no problem, I had high transfer speeds (I have a 1Gb network).
It copied the machines in just 10-15 minutes (40gb).

The problem has now been when I reinstalled my esxi, connect my thunder again and when transferring the machines from thunder to my esxi, the speed seems to be limited to 10MB / s.
It's very strange. It seems to be trying to speed up but crashes and goes back to 0kb / s and starts going up again ... and goes down again.
Now the machines take me hours to ship again ...
I have reviewed a thousand and one post but I can not find anything that solves my problem.

It would be helpful if you could shed some light on the matter.
Greetings to all!

 

[blaze0808]

Excuse me, I forgot to comment that the connection used was iSCSI

 

[jenksdrummer]

I have same/similar observance as it appears to be an OS threshold for what gets written to a buffer before it chokes it off to allow the RAM buffer to empty. Mainly this is a case of the data is moving faster than it can commit and it will buffer it.

Repeat your test, watch your RAM usage; it will climb until about 20% used; then it will choke off progress while it flushes out the buffer.

Assuming that ESXI is hosting a VM that you're using as your observed issue.

 

[blaze0808]

I have same/similar observance as it appears to be an OS threshold for what gets written to a buffer before it chokes it off to allow the RAM buffer to empty. Mainly this is a case of the data is moving faster than it can commit and it will buffer it.

Repeat your test, watch your RAM usage; it will climb until about 20% used; then it will choke off progress while it flushes out the buffer.

Assuming that ESXI is hosting a VM that you're using as your observed issue.

Thanks for your kind reply.
Currently on my Truenas computer I have 8GB of RAM.
When you turn on a large part of it, it remains in a "free" state, and when it has been on for a while, the system takes it as a zf cache and leaves very little or practically nothing free.

Now a moment ago my system was like this:
7.8GiB total available (ECC)
Free: 2.7 GiB
ZFS Cache: 2.8 GiB
Services: 2.3 GiB

and I have done a process of copying thunder to esxi and the values have practically not been moved.
I am copying a file of about 4-5Gb and I have the problem of drops in speed.

 

[blaze0808]

someone else? thanks

 

[joeschmuck]

I was thinking the same thing as I read the first posting, you have run out of RAM and the system is just reacting normally for a write operation. 8GB RAM, while does meet the minimum specs, it a low amount of RAM. With iSCSI you are doing block writes so it's a different bird and iSCSI likes more RAM as I recall.

the speed seems to be limited to 10MB / s.

That is odd, I would expect to see it max out at +80MB/s and then drop to near zero, or are you saying the average speed is 10MB/s? I would double check the Ethernet connection you have since you are using ESXi and a VM, ensure you have the correct adapter, physical connection, etc... You probably know the drill.

Also you have left out what version of TrueNAS you are using and any other details on your configuration, like your pool is? Are you passing the hard drives through ESXi, RDM, or worse it's a VMDK?

 

[blaze0808]

I was thinking the same thing as I read the first posting, you have run out of RAM and the system is just reacting normally for a write operation. 8GB RAM, while does meet the minimum specs, it a low amount of RAM. With iSCSI you are doing block writes so it's a different bird and iSCSI likes more RAM as I recall.

That is odd, I would expect to see it max out at +80MB/s and then drop to near zero, or are you saying the average speed is 10MB/s? I would double check the Ethernet connection you have since you are using ESXi and a VM, ensure you have the correct adapter, physical connection, etc... You probably know the drill.

Also you have left out what version of TrueNAS you are using and any other details on your configuration, like your pool is? Are you passing the hard drives through ESXi, RDM, or worse it's a VMDK?

Thanks for your answer.
Regarding 10MB / s it is like that, at no time does it upload more than that while I am copying from thunder to esxi.
That's what "misses" me the most. It is not that it goes up faster and does not go fast, but that when it seems that it tries to climb the speed drops. I never see more than 10MB / s.
I have the latest version of Thunder available.

Currently I have a microserver where I host 4 disks in raid 5. Additionally I have another HP ML30 Gen9 server where I have another 4 disks in raid 5 with an esxi version 6.7.

I have tried both with direct connection (without going through the switch) and being in the same switch.
In my esxi computer I have 2 network cards. Create a vswitch with a vm kernel.
The copy from esxi to freenas is fast (exceeding 10MB / s easily), however the copy from thunder to esxi is where I have problems.

I can try to get another 4GB module and expand to get 12GB, at least for now. See if it improves.
Another thing that I have seen that there is no official compatibility by HP towards version 6.7 of ESXi, maybe that causes me some problem with the raid controller and this is giving me the error ...

I have already tried almost everything and I can't improve at all ... it's very frustrating

 

[joeschmuck]

First of all, I'm not familiar with Thunder, maybe you could point me to the link for it.
Second, after rebuilding your ESXi server with FreeNAS/TrueNAS installed, have you been able to transfer from FreeNAS to Thunder faster than 10MB/s? I know you said you did originally but I'm looking to see if you can do it now.
Third, you never really answered my questions above when it comes to the hardware configuration. Do not assume that we understand at all, we do not read minds and we are not there to see it with our own eyes. This makes it difficult to give you good information on how to troubleshoot your problem. I still have no idea what version of software you are running, how your VM hard drives are configured (Pass-Thru, RDM, VMDK), your VM NIC, your motherboard make/model. All kinds of things which may make a difference. I'd hate to see you purchase more RAM for no reason.
Fourth, Do some data transfer tests between FreeNAS and a computer using SMB protocol, report the results. Hopefully you are able to get way above 10MB/s in both directions. And feel free to use "iperf3" as it's included in FreeNAS for troubleshooting throughput issues.