FreeNAS Mini + Write Cache SSD only 16 GB

[tbol.inq]

Hello,

Too bad that my first entry may be a stupid question. Sorry for that, but...

We bought three FreeNAS Mini by ixsystems with Read and Write Cache. After unboxing and setting up the three devices I figured out, that the SSDs for Read Cache has 480 GB, but Write-Cache SSD only has 16 GB. Both SSDs are the same SSD Model.

Due to the fact that there is no option for sizing of SSD Write Cache for the FreeNAS Mini: Is this the normal setup? 16GB? for Write-Cache of the FreeNAS Mini?


Thank you for your help & best regards,
tbol.inq

 

[anmnz]

Yes. The so-called "write cache" is not really a cache as normally understood. Lots of info available on the forums under its correct name: it is a SLOG ("separate intent log"). It only needs to hold something like 10 seconds worth of writes at the absolute maximum.

 

[sretalla]

I have seen somebody around the forums do a calculation a few years back and determined that 30GB was about the max size that could ever be expected to be held in SLOG (accounting for headroom and free space).

Maybe it was @jgreco or @Spearfoot

 

[jgreco]

I have seen somebody around the forums do a calculation a few years back and determined that 30GB was about the max size that could ever be expected to be held in SLOG (accounting for headroom and free space).

Maybe it was @jgreco or @Spearfoot

Probably not me. You're a relative newbie ;-) I did a lot of explanation of SLOG in the old days, and the calculation is pretty easy, but I've been somewhat busier these last few years so I don't get to the repetitive stuff anymore and USUALLY leave it to those who have more free time. But since you tagged me, here ya go:

It's the maximum size of two transaction groups. So if you could actually saturate dual 10GbE into a FreeNAS and that each of the two transaction groups is 5sec max, you need to figure 20Gbits/sec * 10 seconds * 1byte/8bits = 25000000000 bytes or 25GBytes. That's what's needed for line rate SLOG if you have a system with dual 10GbE.

You would probably need a very fast pool to be able to soak up the required 2.5GBytes-per-second pool write rates, though. I'm pretty sure the FreeNAS Mini would be pretty well served by a 16GByte SLOG. Since the iX product page isn't coming up for me, I'm *guessing* the Mini comes with 2x or 4x 1GbE and may be expandable to 2x 10GbE, so the 16GByte SLOG device seems suitable.

I do wish iXsystems would stop calling the SLOG a "write cache." It isn't. The write cache is in your system RAM (either the pending or in-progress transaction group). The SLOG is there for POSIX compliance.

 

[jgreco]

Also worth noting that I still generally advocate lowering vfs.zfs.txg.timeout from 5 down to 1 second. This will decrease "maximum benchmark" write speeds but even all these years later after bug #1531, it seems to have a nice effect on stabilizing write performance rates (arguably reducing overall throughput to a somewhat reduced rate). I am sometimes still able to induce catatonic states on ZFS before the new write throttle has "learned" the system performance (i.e. especially right after first boot) and reducing txg.timeout makes it a LOT harder to get it to overestimate performance. This also obviously changes the calculus of the maximum transaction group size, reducing it substantially.

 

[jgreco]

Yes. The so-called "write cache" is not really a cache as normally understood. Lots of info available on the forums under its correct name: it is a SLOG ("separate intent log"). It only needs to hold something like 10 seconds worth of writes at the absolute maximum.

And since I'm absolutely worthless this morning, I meant to include

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.ixsystems.com

which is a bit dated but still mostly relevant.

 

[Mlovelace]

Probably not me.

It was more than a couple years ago, so I'll cut you some slack for not remembering.

"If you're on 2 x 10GbE, your clients could potentially be sending you as much as ~~~2.5GBytes/sec, if you're using a 5 second transaction group, that's 12.5GBytes, and sizing for three transaction groups suggests a SLOG partition size of around 40GBytes would be good. You're not truly likely to actually need anywhere near that. I do feel there's value in doing this, because internally it gives the SSD a huge free page pool to pull from when it is under stress, which will keep things flowing nicely."

This was the thread... https://www.ixsystems.com/community...rmance-tuning-advice-needed.54833/post-381427

 

[jgreco]

Well, that's downright impressive searchin' there pardner.