logs are written to the pool, making regular disk activity

[seldo]

Hello,

I just installed TrueNAS 12.0-U2.1 on an SSD.
I created a pool with two rotating drives in a mirror vdev (WD 8TB + ST 8TB)
These two drives are the only rotating devices I have at the moment.

Every 5/10 seconds, there is disk activity on my pool.

I would like to have less activity when I'm not actively working on the pools to reduce the noise.
The NAS is not is a particularly well isolated area and these noises particularly stand out.

Here's my drive configuration:

Code:

root@bob[~]# gpart show
=>         40  15628053088  ada0  GPT  (7.3T)
           40           88        - free -  (44K)
          128  15628053000     1  freebsd-zfs  (7.3T)

=>         40  15628053088  ada1  GPT  (7.3T)
           40           88        - free -  (44K)
          128  15628053000     1  freebsd-zfs  (7.3T)

=>       40  125045344  da0  GPT  (60G)
         40       1024    1  freebsd-boot  (512K)
       1064  125042688    2  freebsd-zfs  (60G)
  125043752       1632       - free -  (816K)

I tried to export my pool and I get:

Code:

root@bob[~]# zpool export ARCHIVE_POOL
cannot unmount '/var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3': unmount failed

Though I cannot find these files in my pool:

Code:

root@bob[~]# ls -al /mnt/ARCHIVE_POOL
total 1
drwxr-xr-x  2 root  wheel    2 Mar 27 21:16 .
drwxr-xr-x  3 root  wheel  128 Mar 27 21:32 ..

Here's my zpool list

Code:

root@bob[~]# zpool list
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
ARCHIVE_POOL  7.27T  7.63M  7.27T        -         -     0%     0%  1.00x    ONLINE  /mnt
boot-pool     59.5G  1.37G  58.1G        -         -     0%     2%  1.00x    ONLINE  -

I don't have swap:

Code:

root@bob[~]# swapctl -l
Device:       1024-blocks     Used:

 

[seldo]

I've made an lsof on the file that is causing issue for the export:

Code:

root@bob[~]# lsof /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3/
lsof: WARNING: compiled for FreeBSD release 12.0-RELEASE; this is 12.2-RELEASE-p3.
lsof: WARNING: access /root/.lsof_bob: No such file or directory
lsof: WARNING: created device cache file: /root/.lsof_bob
COMMAND    PID   USER   FD   TYPE              DEVICE             SIZE/OFF NODE NAME
python3.8  241   root   52w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
python3.8  298   root   22w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
python3.8  299   root   21w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
python3.8  300   root   21w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
python3.8  301   root   21w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
python3.8  302   root   21w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/middlewared.log
syslog-ng 1170   root    5w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/messages
syslog-ng 1170   root   24w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/console.log
syslog-ng 1170   root   25w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/daemon.log
syslog-ng 1170   root   27w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/auth.log
syslog-ng 1170   root   29w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/debug.log
syslog-ng 1170   root   30w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/cron
python3.8 1401 daemon    3w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 -- log/wsdd.log
nginx     1440   root    2w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1440   root    4w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1441    www    2w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1441    www    4w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)

What caught my attention as the last for items:

Code:

nginx     1440   root    2w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1440   root    4w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1441    www    2w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)
nginx     1441    www    4w  VREG 635527490,686022222 18446744071607946523    0 /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3)

Checking the mounts:

Code:

root@bob[~]# mount
boot-pool/ROOT/default on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
tmpfs on /etc (tmpfs, local)
tmpfs on /mnt (tmpfs, local)
tmpfs on /var (tmpfs, local)
fdescfs on /dev/fd (fdescfs)
ARCHIVE_POOL on /mnt/ARCHIVE_POOL (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system on /var/db/system (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/cores on /var/db/system/cores (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/samba4 on /var/db/system/samba4 (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 on /var/db/system/syslog-7bdf26d378f44463a02df9c5bc4d0ea3 (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/rrd-7bdf26d378f44463a02df9c5bc4d0ea3 on /var/db/system/rrd-7bdf26d378f44463a02df9c5bc4d0ea3 (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/configs-7bdf26d378f44463a02df9c5bc4d0ea3 on /var/db/system/configs-7bdf26d378f44463a02df9c5bc4d0ea3 (zfs, local, nfsv4acls)
ARCHIVE_POOL/.system/services on /var/db/system/services (zfs, local, nfsv4acls)

 

[seldo]

So it is clear that there is some data (logs) that is written to my pool.
How can I relocate that permanently?
I'm planning to add a rotating 2.5" HDD for the swap: can I move these data over there too? If so, how?

 

[Redcoat]

Try System>System Dataset. The default location is the first data pool created (it's moved from the boot pool).

 

[seldo]

Try System>System Dataset. The default location is the first data pool created (it's moved from the boot pool).

Thanks! For now, I put another drive just for the logs so that I don't have that noise when I am setting up everything else.
Does it make sense to move these logs to the boot-pool or won't they survive reboot?

 

[Redcoat]

As your boot pool's on an SSD it may not matter to you, but, yes, they will survive a reboot if you move them to the boot pool. The "not survive a reboot" issue is that some changes made on the command line do not get written to the system files and thus don't survive a reboot. Best to make all such changes in the GUI if there's an option to do so.

 

[seldo]

Very good, I will move them there tomorrow then.
That's item less to think off!

 

[floyd71]

One beginners question...
Under the System -> System Dataset is check box.

My understanding is is you uncheck this box, that Syslog will be written in /var/ on the operating system device.

When I make list of the pools, i have only 2 pools
boot-pool
SKLADISTE

and as I checked, boot-pool is physically one SSD drive connected by USB interface, and SKLADISTE is my HDD storage created from 4 x 6TB HDDs.

Up to now, settings was to put the syslog to boot-pool. Is it the same device if I uncheck the "checkbox"?
As I said, I have operating system on this SSD (USB connected) and storage is 4HDDs (pool "SKLADISTE").

The question is related to idea to allow that my disks when not used can spin down.
I tried several scenarios, and once I made that disks stop spinning (if no requests on the "SKLADISTE") for long time, but I cannot repeat it again.
I am setting on the Disks (HDD Standby xx minutes, Force HDD Standby (ON), Advance power management - 1 (Minimum power usage) and Enable S.M.A.R.T (ON), and result is that disks stop spining and after 5 - 6 sec. they again start to rotate.

How to really control them to not rotate if there is not data read / write on "SKLADISTE" (STORAGE) pool.

Thank you!

 

[sretalla]

How to really control them to not rotate if there is not data read / write on "SKLADISTE" (STORAGE) pool.

You should look here:
https://www.truenas.com/community/resources/hdd-spindown-timer.122/ (== https://github.com/ngandrass/freenas-spindown-timer)