J
jpaetzel
Guest
The default ZFS tuning can be too conservative for pools with high speed SSD L2ARC devices.
# cd /usr/local/www/freenasUI/tools
# ./arcstat.py -f l2size,l2read,l2miss,l2hits,l2byte
The run:
# iozone -a -s XXg -r 4096
Pick a size for -g that is a tad below your arc size + l2arc size, so on a system with 16GB of RAM and a 400GB l2arc, you might want a size of 390G
The iozone numbers you are about are the write, read, random write, and random read.
The following two sysctls are read by ZFS when the pool is imported and affect performance when an L2ARC device is attached.
vfs.zfs.l2arc_headroom: 2
vfs.zfs.l2arc_write_max: 8388608
We've discovered that they are too conservative for faster L2ARC devices, and by turning them up you can gain L2ARC performance.
We've found that with Fusion-I/O cards as L2ARC the headroom setting can be 16, and the write_max can be increased by an order of magnitude. Write performance tends to drop 5-10%, but read performance can see a 50-75% improvement. We suspect that devices slower than the Fusion-I/O cards can utilize higher settings than stock, but we haven't had time to gather extensive telemetry on them.
Note: You'll need FreeNAS 8.2.0-RELEASE-p1 to set the sysctls at the proper time during boot, previous releases set them after pool import, which was too late.
# cd /usr/local/www/freenasUI/tools
# ./arcstat.py -f l2size,l2read,l2miss,l2hits,l2byte
The run:
# iozone -a -s XXg -r 4096
Pick a size for -g that is a tad below your arc size + l2arc size, so on a system with 16GB of RAM and a 400GB l2arc, you might want a size of 390G
The iozone numbers you are about are the write, read, random write, and random read.
The following two sysctls are read by ZFS when the pool is imported and affect performance when an L2ARC device is attached.
vfs.zfs.l2arc_headroom: 2
vfs.zfs.l2arc_write_max: 8388608
We've discovered that they are too conservative for faster L2ARC devices, and by turning them up you can gain L2ARC performance.
We've found that with Fusion-I/O cards as L2ARC the headroom setting can be 16, and the write_max can be increased by an order of magnitude. Write performance tends to drop 5-10%, but read performance can see a 50-75% improvement. We suspect that devices slower than the Fusion-I/O cards can utilize higher settings than stock, but we haven't had time to gather extensive telemetry on them.
Note: You'll need FreeNAS 8.2.0-RELEASE-p1 to set the sysctls at the proper time during boot, previous releases set them after pool import, which was too late.