Slow writes on ixsystems hardware

Status
Not open for further replies.

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
Guessing that da0 is the SLOG?

According to this, I should guess so :
upload_2016-9-13_17-2-29.png
 

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
This is the disk performance of da0 during the migrating of the VM. You can see the part where the sync was enabled.
I am guessing that the top performance of 15MB/s might be the issue.
Question is, is it cause or effect?
upload_2016-9-13_17-4-34.png
 
J

jpaetzel

Guest
ESXi artificially sets every NFS write as a sync write, which has to go through the SLOG device. The best solution would be to start using iSCSI (The hardware offload with VAAI is a huge win there) A more appropriate SLOG device would help your current write performance if you continue using NFS + ESXi.
 

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
ESXi artificially sets every NFS write as a sync write, which has to go through the SLOG device. The best solution would be to start using iSCSI (The hardware offload with VAAI is a huge win there) A more appropriate SLOG device would help your current write performance if you continue using NFS + ESXi.
Hi, I understand the NFS sync issue, and as a last resort we will switch to iSCSI.
However I am having a hard time believing that by replacing the slog SSD with another brand of SSD, the write performance will at least quadruple. Mlovelace mentioned that he was getting 240MB/s with an Intel DC S3700 (same price category as our Samsung 850 Pro).
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
This is the complete config :

index.php
Did this config come from ixsystems? Are they the ones who configured the 850's as both a SLOG and L2ARC?
SLOG performance is all about latency. The lower the better.
 

Sakuru

Guru
Joined
Nov 20, 2015
Messages
527
I bought a Samsung 950 PRO and it performed terribly as a SLOG. I don't know why. I guess there's just something about Samsung SSDs that doesn't play nice with a SLOG workload. Oh well, as I mention in my signature I should have gotten an Intel 750 instead.

I am rather concerned that someone at iX Systems sold you a box with an improper SLOG. I'm also concerned they called it a ZIL instead of using the proper term. @jkh Do you know what happened here?
 

zambanini

Patron
Joined
Sep 11, 2013
Messages
479
i had a similar party with an ixsystem offer. they offered a system which would have (regarding the existing data) only 30% free space. vsphere and iscsi would not like that
 

Mlovelace

Guru
Joined
Aug 19, 2014
Messages
1,111
ESXi artificially sets every NFS write as a sync write, which has to go through the SLOG device. The best solution would be to start using iSCSI (The hardware offload with VAAI is a huge win there) A more appropriate SLOG device would help your current write performance if you continue using NFS + ESXi.
This all sounds oddly familiar... ;)
 

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
Did this config come from ixsystems? Are they the ones who configured the 850's as both a SLOG and L2ARC?
SLOG performance is all about latency. The lower the better.
Yes this is what they suggested after we mentioned our budget.
 

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
Okay so everyone agrees the Samsung 850 Pro is the root cause here? This means that we can either replace the samsung with an Intel SSD, or switching to iSCSI.
Can the ZIL-SSD be replaced with another disk without destroying the pool? If we should go with iSCSI, do we leave the samsung ZIL seated?
 

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
J

jkh

Guest
I am rather concerned that someone at iX Systems sold you a box with an improper SLOG. I'm also concerned they called it a ZIL instead of using the proper term. @jkh Do you know what happened here?
I don't know anything about the box in question (at the very least, someone would need to give me a serial number since I otherwise have zilch to go on in this conversation).
 

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
I don't know anything about the box in question (at the very least, someone would need to give me a serial number since I otherwise have zilch to go on in this conversation).
Hi jkh, thank you for joining this topic.
The system has this serial : A1-50397. Quote KL160526.
Thomas.
 

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
Hi,

I was contacted by iXsystems and they were willing to swap the ZIL for Intel type. However they doubt that it will have much effect as the Samsung has better specs in terms of speed than the Intel SSD.
They believe that tuning the parameters is the way to go. I must say that I understand their reasonings.

In the mean time, I have switched from NFS to iSCSI and performance is now at expected level. And I also get VAAI, multiple connection support,...
So me happy.
Because of the asynchronous character of iSCSI, the concerning ZIL is now not used anymore. So no more performance issues there, but is this not as dangerous as using NFS with sync disabled?

Apparently with FreeNAS and ESXi, iSCSI is the way to go.
 

Mlovelace

Guru
Joined
Aug 19, 2014
Messages
1,111
but is this not as dangerous as using NFS with sync disabled?
You can alway force sync writes for iSCSI if/when you get a proper slog device.
 
Last edited:

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
IMHO, I would not feel comfortable at all with "sync=disabled" where VMs are concerned.

*** Side note, I was going to do some tests with NFS and various scenarios (No SLOG, S3710 SLOG, Samsung Evo SLOG, S3500 SLOG), but maybe not needed anymore..
 

Sakuru

Guru
Joined
Nov 20, 2015
Messages
527

Thomas_VDB

Contributor
Joined
Sep 22, 2012
Messages
102
You can alway force sync writes for iSCSI if/when you get a proper slog device.
Okay, but i doubt that iscsi is typically used like that, no? In sync=standard mode, the zil is not used with iscsi. So is iscsi in this mode then not equally unsafer as nfs with sync disabled?
 
Status
Not open for further replies.
Top