Point of interest though, when you've got a single 300GB file like these, (any suitably large file really, ie anything bigger than your SLOG size) you're really only home once the whole file makes it to disk. A SLOG won't help you if the write fails anywhere in the middle - from the first byte to the last - which is true of all default values (64GB txg trigger or 5 seconds etc) or stuffing the whole thing asynchronously into RAM (with much bigger values than the defaults like 320GB and 300 seconds) and writing it out to disk slowly as the performance of the pool will allow.
I've not seen anywhere in the literature a suggestion that the SLOG should be at minimum the size of the largest file you want to be sure you don't lose to a kernel panic or power out. This should surely mean that a SLOG is only of value to any synchronous write smaller than the 10%/4GB margin in general or at least up to the size of the SLOG. The Jim Salter article notes that the SLOG only needs to be small - to account for a few seconds worth of writes.
It [SLOG] also doesn’t need to be very large – just enough to hold a few seconds’ worth of writes is the quote.
Any single file larger than that will automatically be corrupted with a failure, SLOG or not. And a continuation of my earlier thoughts on "you'll lose what's in a large dirty RAM cache if you get a power fail/kernel panic etc", yes that's true, but i still don't see a difference between losing a complete file in RAM (after a full bore transfer from the file source) on the filer part way through the pool write, and the alternative of having the server panic and stop part way through a file write when the data is still trickling in more slowly over the network with the filer throttling the input to control a slow writing pool. They will both fail to write the file and a verify from the file source will pick up that failure. That's true of both asynchronous and sync/slog behaviour isn't it?
Have I understood that right or not? Is ZFS's SLOG capable for example of picking exactly where you left off, so a fail in the last half of a sync 60GB file write would still succeed on restart with a 32GB SLOG? But a fail in the first half would always produce a corrupted result.
I realise i've wandered in and out of sync/async in my rambling above, hopefully you work out what i mean....
I've not seen anywhere in the literature a suggestion that the SLOG should be at minimum the size of the largest file you want to be sure you don't lose to a kernel panic or power out. This should surely mean that a SLOG is only of value to any synchronous write smaller than the 10%/4GB margin in general or at least up to the size of the SLOG. The Jim Salter article notes that the SLOG only needs to be small - to account for a few seconds worth of writes.
It [SLOG] also doesn’t need to be very large – just enough to hold a few seconds’ worth of writes is the quote.
Any single file larger than that will automatically be corrupted with a failure, SLOG or not. And a continuation of my earlier thoughts on "you'll lose what's in a large dirty RAM cache if you get a power fail/kernel panic etc", yes that's true, but i still don't see a difference between losing a complete file in RAM (after a full bore transfer from the file source) on the filer part way through the pool write, and the alternative of having the server panic and stop part way through a file write when the data is still trickling in more slowly over the network with the filer throttling the input to control a slow writing pool. They will both fail to write the file and a verify from the file source will pick up that failure. That's true of both asynchronous and sync/slog behaviour isn't it?
Have I understood that right or not? Is ZFS's SLOG capable for example of picking exactly where you left off, so a fail in the last half of a sync 60GB file write would still succeed on restart with a 32GB SLOG? But a fail in the first half would always produce a corrupted result.
I realise i've wandered in and out of sync/async in my rambling above, hopefully you work out what i mean....
Last edited: