Slog already compressed/deduped?

[briffle]

I am looking at putting in an SLOG device, that has a 150T of writes warranty. however, we send about 1TB a day of data to this backup server. I'm trying to understand, does the SLOG write the data immediately as it comes in, before 'processing' it? or is this after compression and dedupe (which in our case, is actually about 200G/day)

 

[L]

SLOG is an intent log. Data actually remains in memory it just writes log info.

 

[Mlovelace]

I am looking at putting in an SLOG device, that has a 150T of writes warranty. however, we send about 1TB a day of data to this backup server. I'm trying to understand, does the SLOG write the data immediately as it comes in, before 'processing' it? or is this after compression and dedupe (which in our case, is actually about 200G/day)

Does your backup job perform synchronous writes to the zpool? Unless the writes are synchronous the slog device will not be touched. It's not a general write cache device it will only help with sync writes.

 

[briffle]

about 95% of the data is written via NFS, so I think the answer is "yes" to the sync writes. The rest is using scp.

 

[jgreco]

There's a lovely post that describes what the SLOG is. Read it, then come back and tell us what sort of workload you're doing.

https://forums.freenas.org/index.php?threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/

 

[Mlovelace]

about 95% of the data is written via NFS, so I think the answer is "yes" to the sync writes. The rest is using scp.

NFS doesn't automatically equal sync writes, it can be sync or async. What drive are you looking at specifically for your SLOG? Does it have power loss protection?

 

[briffle]

I have read that, a few times. I also understand that the SLOG is NOT a write cache. However, its my understanding for my NFS connections (Linux server's are copying their backup) that at some point, the freenas server will write the data down to either the zil on the pool, or an slog, to verify that yes, the data is not just sitting in ram, to ensure it's there if the system has a problem. I realize it will almost never be read from. In our case, because of the limited iops we have (we have several large raidz3 vdevs, because of the size of disks we had) we think that it is slowing down the writes, also having to copy it to the ZIL on the pool. (20 servers all writing over 10Gb/s NFS).

If that is the case, we would like to move the SLOG to a separate SSD, to both get it off the overloaded pool, and to make it a bit faster in general. What we are not sure of, if is the SLOG is the raw data that comes in (ie, uncompressed) in which case, we can plan on about 6 months of service with this SSD, before we hit the 150TB warranty (we would probably setup a mirror for the SLOG, and every 3 months, replace one of the members). If the SLOG is after compression/dedupe, then I can write to the drives for several years. (really, this just changes the replacement schedule, and planned visits to the datacenter)

 

[briffle]

I use sync defaults in NFS for centos 6, so I'm pretty sure we are using sync. We are looking at a samsung EVO 845 DC Pro, which does have power loss protection.

 

[jgreco]

The SLOG's most important quality is to minimize latency. The system is making a promise that the data has been written to stable storage - which doesn't have to be the pool. That's the SLOG device, then. No compression, no deduplication, nothing that'd cause slowdown. Just "yessir" "yessir" "yessir" as fast as it can go, at which point the data is then also placed in RAM and gets massaged by the normal ZFS write processes.

If you have to consider a SLOG device, then contemplate the S3700's. But if this is for backups, you might want to consider that most of the reason for a SLOG is to guarantee that the disk image is consistent with what you have in-core; that's usually not the case with a backup anyways. Depends on what you're doing for backups, of course. But a filer with a competent SLOG device is much more important (IMHO) for a live SAN appliance, where you are really more afraid that the filer takes a crap while VM's are running, and you want the VM disks to be in a consistent state when the filer returns to service.

 

[briffle]

That is a good point. And since this is just for backups, we could just disable sync with something like: zfs set sync=disabled tank/dataset (knowing that that could break things down the road)