ZFS Deduplication

ZFS supports deduplication as a feature. Deduplication means that identical data is only stored once, which can significantly reduce storage size. However, deduplication is a compromise and balance between many factors, including cost, speed, and resource needs. Consider and understand the implications of using deduplication before adding it to a pool.

Deduplication on ZFS

Deduplication is one technique ZFS can use to store files and other data in a pool. If several files contain the same pieces (blocks) of data or any other pool data occurs more than once in the pool, ZFS stores just one copy of it. Instead of storing many copies of a book it stores one copy and an arbitrary number of pointers to that one copy. Only when no file uses the data is the data deleted. ZFS keeps a reference table that links files and pool data to the actual storage blocks containing their data. This is the deduplication table (DDT).

Read full post gdoc_arrow_right_alt

ZFS Primer

Zettabyte File system (ZFS) is an advanced, modern file system specifically designed to provide features not available in traditional UNIX file systems. Sun originally developed and intended ZFS to be an open-source file system that can be ported to other operating systems. After the Oracle acquisition of Sun, some of the original ZFS engineers founded OpenZFS to provide continued, collaborative development of the open source version.

ZFS Feature Overview

Here is an overview of the features provided by ZFS.

Read full post gdoc_arrow_right_alt

ZFS dRAID Primer

Introduced in OpenZFS version 2.1.0 and initially supported in TrueNAS 23.10 (Cobia), declustered RAID (dRAID) is an alternate method for creating OpenZFS data virtual devices (vdevs).

Intended for storage arrays with numerous attached disks (>100), the primary benefit of a dRAID vdev layout is to reduce resilver times. It does this by building the dRAID vdev from multiple internal raid groups that have their own data and parity devices, using precomputed permutation maps for the rebuild IO, and using a fixed stripe width that fills storage capacity with zeroes when necessary.

Read full post gdoc_arrow_right_alt

ZFS ZIL and SLOG

The ZIL and SLOG are two frequently misunderstood concepts in ZFS.

ZFS is taking extensive measures to safeguard your data and it should be no surprise that these two terms represent key data safeguards. What is not obvious, however, is that they only come into play under very specific circumstances.

The first thing to understand is that ZFS behaves like any other file system with regard to asynchronous and synchronous writes.

Read full post gdoc_arrow_right_alt

ZFS Capacity Calculator

Determine the usable capacity and other metrics of a ZFS storage pool and compare layouts including stripe, mirror, RAIDZ1, RAIDZ2, RAIDZ3, and dRAID.

Click on the section titles to expand/collapse and view calculated data. Hovering over a table cell loads the relevant data into the Calculation Values section below. You can click table cells to freeze or unfreeze those values.

Read full post gdoc_arrow_right_alt

ZFS Capacity Graph

Visualize and compare different RAID layouts and settings including stripe, mirror, RAIDZ1, RAIDZ2, RAIDZ3, and dRAID.

ZFS Capacity Graph originally created by Jason Rose.

RAIDZ Extension Calculator

RAIDZ extension allows TrueNAS users to add disks one at a time to an existing RAIDZ VDEV, incrementally expanding capacity. Use this calculator to estimate the storage capacity efficiency of a ZFS RAIDZ VDEV after incremental extension.

Existing data blocks retain their original data-to-parity ratio and block width, but are spread across the larger set of disks. New data blocks adopt the new data-to-parity ratio and width, or overhead. Because of this overhead, an extended RAIDZ VDEV can report a lower total capacity than a newly created VDEV with the same number of disks.

Read full post gdoc_arrow_right_alt