Network Perfomance

[Jan Kare]

Hi,

I am new to FreeNas :)

I have set up an ISCSI link (1gbit) to an vmware host. Have created a ZFS volume which is set as an extent on ISCSI. Not sure this is the best way, but was the only way(except NFS) I could think of.

Anyway, the network perfomance is really slow. It starts out with around 100mb/s and drops to 20mb/s after 30-40 sec.

Is there any tips/tricks/tweaks to figure out whats going on ?

 

[gpsguy]

As per the forum rules in red at the top of the page, please provide detailed information regarding your system and version of FreeNAS used. If you are using 9.3, which build do you have?

In addition to the server specs, tell us about your pool. For example, say you had 10 HDDs. A volume that striped 5 sets of mirors would give you better iSCSI performance than a 10 disks RAIDz2 volume.

How full is the volume? For iSCSI, we recommend that you don't exceed 50%.

 

[Jan Kare]

Hi, sorry for the lack of information :)

It is the 9.3 stable release. (unsure of build since i am at work at the moment, will update later)

Server is an I5-760 with 8gb. It consist of 2 raid5 sets, hence 2tb each. (3x1tb/5x500gb) Both are "new" so completly empty. All are 7200rpm disks. (500gb=wd/1tb=seagate), Realtek 8169 NIC.

 

[pirateghost]

Realtek. Desktop grade hardware. 8gb RAM.

I think I found the problem

 

[Jan Kare]

I have tried both the onboard and additional NIC and it was the same. Starts good but slows down. It is desktop grade hardware and 8gb should be sufficient to write faster than 20mb/s

 

[pirateghost]

You also said raid 5...freenas doesn't do raid 5. Do you mean raidz1?

FYI, you will need more than 8gb RAM to run a good iscsi

 

[HoneyBadger]

Please do a "zpool status" from the command line and post the result in code blocks, I suspect all of your drives are lumped together in one huge pool.

You're also using async writes here which means that initial burst of speed is just "how fast can I stuff my RAM full" and not "how fast can I actually write to disk."

Realtek. Desktop grade hardware. 8gb RAM.

I think I found the problem

You left out "using VMFS on parity RAID."

 

[Jan Kare]

Picture from zpool status in command line.

I noted that the the RAM comment seems accurate. I tested on a vm with 2 and with 4gb ram and both run fine till the vms ram is full.

What can i do to improve the writing speed ?

I am here to learn :)

 

[gpsguy]

I only see one pool, striped with 3 disks (no fault tolerance). Are the other 5 in another pool, which scrolled off the screen?

As I said in my first reply, striped mirrors give better iSCSI performance, while offering fault tolerance.

As pirateghost alluded to, the best solution would be to start over with server grade hardware that supports ECC RAM (i5's don't). Typically, they would include Intel NIC's (while Realtek is okay for Win, it's a bad choice for FreeNAS). And, typically for iSCSI, a starting point for RAM might be 32GB.

 

[HoneyBadger]

I only see one pool, striped with 3 disks (no fault tolerance). Are the other 5 in another pool, which scrolled off the screen?

That should actually make the pool significantly faster than a RAIDZ since he doesn't have to account for parity anywhere.

I think the Realtek NIC is accounting for a lot of this, especially if there's another one in the ESXi server.

@Jan Kare - With regard to the RAM issue, it's actually your FreeNAS server's RAM that's being filled. By default iSCSI writes from ESXi are asynchronous - in that FreeNAS will say "yep, I wrote those to disk" before it actually does. It's just holding them in RAM, which gives the illusion of fast writes.

NFS may be a better choice as it works better in a low-RAM situation, but it defaults to sync writes so you would have to shut those off and forgo safety if you want any semblance of speed from this box.