sync=always slow on mirrored SSD used as SLOG

Status
Not open for further replies.

mvcad

Contributor
Joined
Feb 25, 2018
Messages
116
Hi All, newbie here. Am I wrong thinking that setting sync=always on a volume with a dedicated ZIL (SLOG) will have faster write performance? because I am getting different results.
I have a FreeNAS mini XL with 4 off 4TB WD red in Raid Z2 with 2 off 256 GB Samsung SSD used as SLOG (mirrored).
I used LAN_Speed_Test program and I have these results:

Sync = Always Write speed is: 521.9 Mbps
upload_2018-9-5_13-39-59.png


Sync = Standard Write speed is 749.4 Mbps
upload_2018-9-5_13-41-8.png


As you can see I am getting 150% faster speeds with sync=standard.
what am i missing?, should I leave it as Sync = standard?
Is the performance better with bigger files being written?
Would i get better performance if I had a sinlge SSD as SLOG (Not mirrored)?
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Nothing fishy.
If you'd remove the SLOG and run the tests again, you'd see why you'd want a SLOG :)
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Sync = Always means that the system returns the "write complete" to the requesting process only when all copies of the data are finished writing to disk.

Normally, with a single log disk or with Sync = standard on a multi-disk setup, the "write complete" is returned as soon as any copy of the data is on disk.

What you are seeing makes perfect sense in line with the above.

The choices you make will determine if your system is skewed toward speed or toward protecting the data (you can't have both).
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Am I wrong thinking that setting sync=always on a volume with a dedicated ZIL (SLOG) will have faster write performance?
Yes. Sync will never be faster than async. ZFS caches writes in RAM, and no SSD is going to be faster than your DRAM.
 

mvcad

Contributor
Joined
Feb 25, 2018
Messages
116
Thank you all guys, I have a better understanding now. I will be transferring a lot of data from a synology nas to our new freenas over the weekend so I am thinking of leaving sync=standards for the moment to get better performance. this to ensure all data will be transferred over the weekend before everyone comes back on Monday to work. after that I'll switch back to sync=always to get better protection. Does this make sense?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
What are you using for the transfer? Sync writes should only help if the server crashes or loses power or something.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
Normally, with a single log disk or with Sync = standard on a multi-disk setup, the "write complete" is returned as soon as any copy of the data is on disk.

I think that’s not quite it.

Sync always and sync standard have the same behavior for sync writes. The write is acknowledged once the transaction is written to the zil. Which if it’s on a slog will be faster.

Sync disabled and sync standard have the same behavior for async writes, which is that the write is acknowledged as soon as it’s received.

Sync always always writes synchronously, even if the write is async.

Sync disabled always writes asynchronously, even if the write is sync.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Happy to hear more precision on the topic.

I don't know how we can explain the performance difference in the OP's tests though in that case.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
When he forces sync it’s slower than when he doesn’t. Nothing to explain
 

mvcad

Contributor
Joined
Feb 25, 2018
Messages
116
@Ericloewe Not sure what you mean, I am just testing sending a single file from my local window 10 PC to our FreeNAS over a Gbit network
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
I suspect Eric wanted to know how you were transferring files. Suspect the answer is via SMB.

I *think* more recent versions of FreeNAS by default enable sync writes on SMB.
 

mvcad

Contributor
Joined
Feb 25, 2018
Messages
116
You are right. I send the files to an SMB share.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
I *think* more recent versions of FreeNAS by default enable sync writes on SMB.
First I've heard of that. And I haven't noticed any performance degradation.
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
I *think* more recent versions of FreeNAS by default enable sync writes on SMB.

Enable as in "will do them if the client asks" possibly, but if it was "treats all SMB traffic as sync" then there'd be a torrent of threads from just-upgraded users asking "why is my SMB performance absolutely garbage?" because without a proper SLOG device you'll decimate your write speeds.

And Windows by default doesn't care about your data, so it'll write async whenever it can. ;)
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
And Windows by default doesn't care about your data, so it'll write async whenever it can. ;)
To be fair, it cares as much as your typical user actually cares. If they want to go check that their stuff was correctly written, they're free to do so.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Status
Not open for further replies.
Top