macOS shows TM backup ~15GBs more than the number in TM dataset

[leonardchee]

Hi everyone

Just got my TrueNAS X+ Mini today, and still wrapping my head around the ZFS file system. I have poked around the forum a bit, and yes definitely quite "shocked"/satisfying to see how TrueNAS is soaking up the 64GB RAM (right now showing Free: 2.6GiB, ZFS Cache: 52.5 GiB, Services: 8.8GiB). RAM not used is RAM wasted indeed!

I have also read about ZIL and SLOG in a few of the threads here. Does all these have something to do with what I'm seeing in my system now? I am backing up my Mac Mini using Time Machine to an SMB Time Machine share on TrueNAS. It is still in the midst of completion, but I can see that consistently macOS reports say 96 GB backed up, but when I look into the dataset in TrueNAS, it reports "Used" of 78 GiB (GB vs GiB yes, but the difference stands).

What is causing the disparity in the reported numbers? Is the difference being held in ZFS Cache right now? So what would happen to this backup if I were to suddenly suffer a power outage now (no UPS for this setup... yet).

 

[sretalla]

What is causing the disparity in the reported numbers?

2 things: first the difference between GB and GiB... probably not enough to make 15GB of difference here though. Second is compression (content dependent, but if you have a lot of content that contains well-compressible structures, your pool will be taking advantage of that).

Is the difference being held in ZFS Cache right now? So what would happen to this backup if I were to suddenly suffer a power outage now (no UPS for this setup... yet).

No, that's not how it works. The ZFS cache (ARC... Adaptive Replacement Cache) is content that was previously read or written from/to ZFS pools and is retained in RAM for quick access if those blocks are requested again. You can check in the reports for your ARC hit ratio to see how many reads are being served directly from RAM instead of from the disks.

All of the data in the ARC is already on the disk and will be safe in a power cut to the system.

Sync writes are another thing that is probably in play here which means that the requesting application (in this case time machine) asks for confirmation from TrueNAS that the write is completed before it is considered ready to move on to the next write. In order to service that requirement, TrueNAS will write a ZIL (ZFS Intent Log) to the pool first, then report back to time machine that it can move on... which is slow if you think about effectively writing it once to the ZIL and then again to the pool for real... which is where a SLOG (Separate LOG) can help (by being both separate from the pool disks, requiring them to write the sync writes only once, but also in the case of a properly chosen SLOG (and the right pool behind it) allowing writes to go much faster, but still be safe.

In total, the position is that your data isn't held only in RAM for more than a few seconds unless it's an async write request (and even in those cases, should not be more than 5 seconds worth of data in that "risky" state).

Read this for a more comprehensive explanation:

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com

 

[leonardchee]

thank you sretalla, yes I will pore over the link you shared in further detail. Cheers and have a great day!

2 things: first the difference between GB and GiB... probably not enough to make 15GB of difference here though. Second is compression (content dependent, but if you have a lot of content that contains well-compressible structures, your pool will be taking advantage of that).


No, that's not how it works. The ZFS cache (ARC... Adaptive Replacement Cache) is content that was previously read or written from/to ZFS pools and is retained in RAM for quick access if those blocks are requested again. You can check in the reports for your ARC hit ratio to see how many reads are being served directly from RAM instead of from the disks.

All of the data in the ARC is already on the disk and will be safe in a power cut to the system.

Sync writes are another thing that is probably in play here which means that the requesting application (in this case time machine) asks for confirmation from TrueNAS that the write is completed before it is considered ready to move on to the next write. In order to service that requirement, TrueNAS will write a ZIL (ZFS Intent Log) to the pool first, then report back to time machine that it can move on... which is slow if you think about effectively writing it once to the ZIL and then again to the pool for real... which is where a SLOG (Separate LOG) can help (by being both separate from the pool disks, requiring them to write the sync writes only once, but also in the case of a properly chosen SLOG (and the right pool behind it) allowing writes to go much faster, but still be safe.

In total, the position is that your data isn't held only in RAM for more than a few seconds unless it's an async write request (and even in those cases, should not be more than 5 seconds worth of data in that "risky" state).

Read this for a more comprehensive explanation:

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com