Continuous disk activity

[dsouleles]

Hello -

I'm running TrueNAS-13.0-U5.3, I run a Plex Media Server in a Jail and Pi-Hole in a VM. This system has been in continuous use since early Freenas days but recently I have noticed unusual disk activity that I cannot identify. Beginning a few weeks ago I noticed continuous disk activity with an occasional "beep". The system is primarily used as my near-line backup - I run nightly backups to it from a couple of windows machines and that's about it. I've shut down the jail and VM and the disk activity continues. There no errors and nothing obvious in the system log.

Would appreciate any suggestion for how to figure out what is causing the disk activity and why the "beep."


Regards,
Dean

 

[winnielinnie]

"Continuous" disk activity, as in every few minutes?

 

[dsouleles]

"Continuous" disk activity, as in every few minutes?

No - it is continuous - all the time. I can hear the heads moving - seems like on a cycle that runs for 2-3 seconds and then an occasional beep.

 

[winnielinnie]

The obligatory: "Have you tried moving your System Dataset to your boot-pool"?

(Or even to an SSD-only pool.)

 

[dsouleles]

Yes - I have moved the system dataset to the SSD boot pool.

 

[sretalla]

Pi-hole would be writing to logs, so that could be the cause.

Otherwise, check your plex library settings for any of the background jobs that would crawl through the library (video preview thumbnails or chapter thumbnails)

 

[dsouleles]

Thank you. I had the same thought. I already turned off logging in pi hole. And then I shut down both pi hole and Plex and the disk activity continued with both of them turned off.

I suspect it has to be some TrueNAS or BSD process, but I can't think of a way to track it down.

The system seems otherwise very healthy with no alerts and no questionable log entries that I can find.

Dean

 

[sretalla]

If it's CORE we're talking about, the system dataset will be the only thing doing regular writes, but it sounds like we eliminated that already, so we're back to, "my disk is making a weird noise".

That's a whole different thing and needs attention.

smartctl -a for all of your disks would be the next thing to look at

 

[dsouleles]

Thanks - smartctl shows no errors.

After several days of the disk activity that I reported in the original post the activity has mostly subsided. There is still disk activity but is now less frequent - I observe disk activity every 10-15 seconds that lasts for a second or two. Sometimes it takes a little longer. And I still occasionally here a beep - but that is even less frequent now.

I've tried monitoring top for IO activity but it doesn't show any activity even when I can hear the disk activity and the disk activity LED on the case lights up.

Is there any way for me to determine what the system is doing when there is disk activity?

Now I am thinking about scrubs and snapshots. I am at a loss for how to continue to diagnose what is happening wiht the system.

Thanks,
Dean

 

[sretalla]

I am thinking about scrubs and snapshots

The disk access pattern of a scrub is nothing like what you describe, so it's definitively not that.

If you have configured a periodic snapshot task to run every 10-15 seconds, maybe that could be it (if you also selected to allow taking empty snapshots).

If that's the case, reconsider your snapshot frequency (radically).

 

[dsouleles]

No, nothing like snapshots every 10-15 seconds! The behavior and periodicity of the disk activity varies a bit over time but I have not been able to figure out what processes are responsible.