Smallest SLOG drive for 1TB ZFS RAID

[zogthegreat]

Hi everyone!

So I'm working on a project to cram a lot of hardware into an HP T620 thin client. I've worked out how to install 2 SSD's for the OS drives and I will be using 2 x's 1TB HDD for data storage. Based on my progress so far, I think that I can squeeze a SLOG drive in.

The server will be for one person who will be having incremental backups of mostly Word Documents, with occasional backup of photos, so really light usage. What's the smallest SLOG drive that I can get away with for this type of setup?

Thanks!

zog

 

[awasb]

It depends on your network connection speed/bandwidth. As well …

Are you sure, you’ll be using sync writes at all? What share types are you going to use? What type of client(s)?

 

[sretalla]

The server will be for one person who will be having incremental backups of mostly Word Documents, with occasional backup of photos, so really light usage. What's the smallest SLOG drive that I can get away with for this type of setup?

That doesn't sound to me at all like a case where SLOG will help (really only helps with Sync IO requests and where IOPS are highly variable).

You may be better using an SSD with good write endurance as L2ARC or metadata only L2ARC.

 

[jgreco]

What's the smallest SLOG drive that I can get away with for this type of setup?

Great news -- none will work just fine and dandy.

Go read

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com

and scroll down to "Why sync writes matter". They most likely don't for your use case.

 

[zogthegreat]

@awasb

Sorry for the late response, been one of those weeks. The network connection for the TrueNAS machine is 62 Mbps, the client will be connecting via WiFi. For the client backups, I will be using something like Duplicati:

Duplicati

"Are you sure, you’ll be using sync writes at all? What share types are you going to use?"

I'm not sure about the sync writes, I'm new to TrueNAS, usually I just setup a smb server and I'm done, but I decided to try TrueNAS for this project. I'm going with a simple smb/nfs share, no bells and whistles for this server, the user won't need any extra features.

@jgreco

Thanks, that's the info that I was looking for! Reading the link that you provided now.

 

[awasb]

With 62 MBit/s you'll be losing roughly 5(seconds)x8MBytes/s=40MB of data in the worst case (machine lock up/power outage etc.) as default txg sync from RAM to storage pool (or the hopefully faster SLOG device in terms of latency) is 5 secs.

But since you are doing file backups via WiFi, I'd say you don't need anything like a SLOG. The latency you'd (theoretically) see — even if you'll have to use sync writes (with NFS e.g.) — will be largely due to WiFi connection.

 

[zogthegreat]

@sretalla

I read a couple of articles about L2ARC:

L2ARC

Provides information on L2ARC, caches drives, and persistent L2ARC implementations in TrueNAS CORE and SCALE.

www.truenas.com

Use cases for L2ARC

I’m having a lively discussion with @microserf elsewhere in this forum, and we’re hijacking a question about “what should be my first motherboard”. I’m taking this discussion here. The two positions: I can’t think of a home / enthusiast use case that would ever justify an L2ARC. Certainly not...

www.truenas.com

And based on what I read, it looks like an L2ARC drive won't be of much use. But thanks for the advice!

 

[sretalla]

it looks like an L2ARC drive won't be of much use.

What I was suggesting was based on the "metadata only" setting, which can effectively speed up the searching of large numbers of files (often something that happens during backups where checks of existing backup lists are done to find changed or new files)... just a suggestion, but as you say may not help.

Here's an example of where it could help:

L2ARC impact on rsync performance for largely dormant data

I've been doing some more testing and the evidence suggests that using a L2ARC with a metadata only flag has a potential substantial positive impact on rsync performance over AFP. For largely dormant data, the combination of L2ARC dedicated to metadata seems to benefit rsync performance a lot...

www.truenas.com

 

[jgreco]

suggesting was based on the "metadata only" setting, which can effectively speed up the searching of large numbers of files

The HP T620 maxxes out at 16GB RAM. The L2ARC will tend to crush the available ARC with L2ARC pointers, so this may not be the performance win you're thinking...? If you had a very small L2ARC device, perhaps.

 

[sretalla]

The HP T620 maxxes out at 16GB RAM. The L2ARC will tend to crush the available ARC with L2ARC pointers, so this may not be the performance win you're thinking...? If you had a very small L2ARC device, perhaps.

I'm not sure how much metadata will be there, but my understanding says it won't be many GBs, so regardless of the L2ARC device size, if it's set for metadata only, how will it be able to put excessive pressure on ARC?

 

[jgreco]

I'm not sure how much metadata will be there, but my understanding says it won't be many GBs, so regardless of the L2ARC device size, if it's set for metadata only, how will it be able to put excessive pressure on ARC?

By a similar argument, one could say "why wouldn't it all fit in ARC". Just a matter of scale. So I'm going to disregard this for a bit.

The point I'm making here is primarily that the platform has very little memory, at least in ZFS terms. Between the middleware and ZFS base, perhaps 6GB of memory is "consumed" by essential stuff. The Duplicati jail will eat some more. ZFS itself needs ARC to help identify L2ARC eviction candidates. You want to evict "unpopular" blocks, based on MRU/MFU. The problem is that if you don't have sufficient ARC, it's more difficult to build a list of meaningful L2ARC candidates that actually represents the pool dynamics. If your RAM is so low that you only ever have a chance to access a block once or twice while in-core in the ARC, the eviction is somewhat less worthwhile, and it takes the L2ARC a lot longer to warm up, because the process is much more speculative and random.

This ends up being on a variation of the iSCSI and block storage issues.

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.truenas.com

Resource - Why iSCSI often requires more resources for the same result

iSCSI is a SAN protocol. NFS, CIFS, etc., are NAS protocols. For a NAS protocol, the client sends a command to the filer, such as "open this file", or "read ten blocks", or "remove this file." On the filer, the local NAS protocol daemon...

www.truenas.com

So, having thought about this for a number of years, "excessive pressure" is something I understand to mean something along the lines of "a call for ARC eviction of a block before the system has an opportunity to gain accurate characterization of the MRU/MFU properties of that block".

In practice, this seems to imply that you need lots of ARC in order to hold data for substantially longer than you might "guess" is useful.

Trying to explain this simply, think of a histogram of ARC block usage. You have 1000 uses for the root directory block, 500 each for foo and bar subdirectories, random use numbers between 1-499 uses for a hundred additional subdirectory blocks. Now the thing is, if your ARC is only large enough to hold 100 blocks, you're clearly going to be thrashing around a teeny bit. The root, foo, and bar will accrue ARC hits fine due to the sheer number of accesses, and will never be eviction candidates. But what you'll find is that, since you can't quite fit all 103 blocks into the ARC, you're going to have to pick a candidate for L2ARC eviction. You can probably "safely" pick any blocks at the tail end and evict them. This is sorta what people are thinking when they think about L2ARC.

But let's double the number of "additional" subdirectories, so we now have 203 blocks that would like to live in the 100-block ARC. If we go to the "eviction point" 100 blocks in, we would like to find ARC entries with 51, 50, 49, etc., uses. These blocks are more useful for eviction than the ones that would only have accrued a single use. Of course, we can't actually hold onto all the blocks, because our RAM limited us to the 100-block ARC defined by the thought experiment. What is actually happening in the meantime, though? Since ARC fills at 100 blocks, it becomes hard for the MRU/MFU eviction mechanism to properly gather stats useful in identifying those "better" blocks for eviction. You get significant thrashing.

Plus, L2ARC isn't free. It eats away at ARC, because the L2 pointers live in ARC.

Therefore, I maintain that properly identifying L2ARC candidates is a bit tricky, and that you can actually hurt yourself with L2ARC if ARC isn't sized appropriately. We've certainly found this to be true for iSCSI and block storage.

Knowing where the actual pain points are is probably a matter of experimentation.

 

[zogthegreat]

The point I'm making here is primarily that the platform has very little memory, at least in ZFS terms. Between the middleware and ZFS base, perhaps 6GB of memory is "consumed" by essential stuff. The Duplicati jail will eat some more. ZFS itself needs ARC to help identify L2ARC eviction candidates. You want to evict "unpopular" blocks, based on MRU/MFU. The problem is that if you don't have sufficient ARC, it's more difficult to build a list of meaningful L2ARC candidates that actually represents the pool dynamics. If your RAM is so low that you only ever have a chance to access a block once or twice while in-core in the ARC, the eviction is somewhat less worthwhile, and it takes the L2ARC a lot longer to warm up, because the process is much more speculative and random.

@jgreco

A quick question, if I may. The T620 will take up to 16gb DDR3L, however, I was planning to run it on 4gb or 8gb. Will this cause problems in the future or is this fine for a single user?

Also, I would like to point out that, as a general rule, the user is only going to be backing up Word documents with incremental changes, (she's a school teacher, so her data files are small). Is there considerations to be made for small data files as opposed to large data files?

 

[jgreco]

4GB is unsupported and expected to be problematic.

8GB is fine for light usage with no jails. Placing a jail on such a system might be doable as long as it isn't large and piggy.

For light usage, being on the lighter side for memory is fine.