win8.1 iscsi performance issue

[fisherwei]

hello everyone,

in my build,

Hardware: (FreeNAS is running in ESXi 5.5 u2)
Intel Z87
i5-4570
20G memory (total is 32G, 20G was assigned to freenas(reserved))
FreeNAS-9.3-STABLE-201503120630
5x 2T, RAID Z1
LSI 9211-8i (passthrough in freenas)
Intel X520-SR2 (no passthrough)
extent is device (i created a zvol)
no ZIL, no L2ARC

Network:
FreeNAS used vmxnet3, and connect to a vSwitch(uplink is X520)
MTU is 9000

1/ I tested pool performance by "iozone", it is no problem.
2/ I copied a 16G files from win8.1 via CIFS, it is no problem also

But, there is some wrong in iscsi,
copied one 16G files from win8.1, it likes a "roller coaster":
the max speed is 400mb/s, the min is 0kb/s(very very long time at 0kb/s)

 

[dlavigne]

Which version of FreeNAS? File or device extent?

 

[Fraoch]

Please post your system specs as the forum rules state.

I would guess you don't have enough memory - iSCSI requires a huge amount to function smoothly. (>32 GB)

 

[fisherwei]

Please post your system specs as the forum rules state.

I would guess you don't have enough memory - iSCSI requires a huge amount to function smoothly. (>32 GB)

sorry, updated.

I only have 20GB memory, so you think the root cause is freenas doesn't have enough memory?

 

[fisherwei]

Which version of FreeNAS? File or device extent?

FreeNAS-9.3-STABLE-201503120630
extent is device (i created a zvol)

 

[zambanini]

ditch the stupid jumbo frame. go back to the normal mtu.

vmxnet and freenas is also not smart, use the forum search...find out yourself.


btw...no ecc ram? why do you even use freenas, when your data is not important to you?

 

[Fraoch]

I only have 20GB memory, so you think the root cause is freenas doesn't have enough memory?

Here's where I read this:

https://forums.freenas.org/index.php?threads/arc-hit-ratio-is-at-100.28166/#post-182992

Well, iSCSI is a random-read random-write workload. That means you need gobs of cache. When i say "gobs" I mean at *least* 32GB of RAM to have something resembling reasonable performance, and very likely 64GB of RAM and an L2ARC. You'll also probably need to do mirrored vdevs and lots of them to get good I/O.

With only 12GB of RAM and a RAIDZ1 you are certifiably crazy for even expecting good performance. It starts off going real fast because your writes are being cached in RAM. But the second that cache runs out you are seeing the real performance of the pool. Sucks don't it? Get more powerful hardware that can absorb that suckiness. ;) I've got 32GB of RAM on my main server and I don't even try to do iSCSI. I did once, and it performed so poorly that I quickly realized it was a no-brainer and gave up.

So although 20 GB seems like a lot, it may not be enough for good performance in iSCSI.

 

[fisherwei]

Here's where I read this:

https://forums.freenas.org/index.php?threads/arc-hit-ratio-is-at-100.28166/#post-182992


So although 20 GB seems like a lot, it may not be enough for good performance in iSCSI.

A little doubt,
if ram is not enough, but the speed is 0kb/s, and after many seconds(5-10s), it will back to 200/300/400mb/s.
for my understand, if write cache runs out, the speed should like hdd original speed, such as 30/50/70mb/s, isn't it?

 

[fisherwei]

ditch the stupid jumbo frame. go back to the normal mtu.

vmxnet and freenas is also not smart, use the forum search...find out yourself.


btw...no ecc ram? why do you even use freenas, when your data is not important to you?

just a home lab. ;)