How To Track Down Cause Of Busy Disk

[Vega Alpha]

I have a FreeNAS Mini 2.0 with 32GB ECC RAM, with 4 Western Digital 4TB Red NAS Drives configured as two 4TB Mirrors. I'm running TrueNAS-12.0-U7.
One of my Mirrors is the system data-set and holds the user directories and jails. The other mirror is mapped several times into the jails and has several SMB shares.

In my scenario, all jails are shut down, and no files are open on any SMB shares. I wish to resolve short repetitive disk accesses that occur roughly every 10 seconds to the system data-set.


I expect there to be no access to the mirror at all during this time. Instead, this access continues night and day without rest. I also find it curious that the access is not to both drives (the top two drives) of the mirror in all cases. Sometimes only one drive is accessed.

Why is this happening? How can I resolve this? I care to do so for the longevity of the drives. As I understand it, with this continuous access, the drives of this mirror will never enter their lowest power mode and spin down, which they most often would do for days. Depending on who you are, your argument on whether one should spin down drives may vary. In a busy system, I would never spin them down, but in my case, when most days the drive could be off, I'm on the spin-them-down-side-of-the-fence, so let's not argue that point. I am getting my way on the second mirror. How do I get what I want on the first mirror?

Is the problem my first mirror is a system volume? Maybe an entry is written to a log every 10 seconds? I have an unused 64GB SSD in the system. Is there a way, and would it be wise to make it the system volume somehow?

I've had some recent trouble with my FreeNAS system. It was because I knew and still know too little about what I'm doing. I could use some good advice about what direction to go with my system.

 

[sretalla]

Maybe fstat is what you're looking for? (to see what's open by which user).

I wish to resolve short repetitive disk accesses that occur roughly every 10 seconds to the system data-set.

You're talking about activity that happens by design and can't be avoided, so you should expect to "resolve" anything here. The system dataset is indeed the likely culprit for the "problem" you're wanting resolution for.

Is the problem my first mirror is a system volume? Maybe an entry is written to a log every 10 seconds? I have an unused 64GB SSD in the system. Is there a way, and would it be wise to make it the system volume somehow?

I suspect so.

Make a new pool on that SSD.

Go to System | System Dataset and move it to that pool.

 

[Vega Alpha]

Is it OK if the system dataset is striped? Or is that unrecoverable? My incomplete knowledge says I can rebuild the system data set on a new SSD if it fails. I can do the manual rebuild with no data loss other than the logs because only logs will be on the system data set. Correct?

boot volume - has boot files
system dataset - has logs
dataset 1 - mirror of, jails
dataset 2 - mirror of, maps into jails and network shares.
That's my whole system as far as I know. Unless there is a rule about FreeNAS I don't know. For example, perhaps do Jails always go on system dataset? If so I have not learned that yet.

 

[Patrick M. Hausen]

The system dataset is for log files, state information, ... - nothing in there is essential if you have a config backup. You can always do a clean install and re-import.

So it's ok to put it on a pool without redundancy but you should pick at least an SSD with some write endurance and not a USB drive or similar. That's why by default it goes on the first data pool created. Normally that one is for storage and can stand the write load.

Also note that while you can always recover, momentarily your system will probably halt/crash when the system dataset pool fails. Depending on your availability constraints that might be a point to consider.

 

[Vega Alpha]

It has been 12 days since sretalla suggested I use fstat to track down my mysterious disk accesses. This has been beyond my FreeBSD skillset. I really don't know what to do at this point. On one hand, FreeNAS has been great in serving Samba and Plex, but I've never been able to grow far beyond that. Windows can do those things and since I'm a windows person I don't need new knowledge to take my software development to the next level with it. Then there is Unraid. It runs Linux which I need experience using. But FreeNAS performs well with lower spec hardware than does Unraid.

I think you guys would tell me that fstat is the same as this tool https://docs.microsoft.com/en-us/sysinternals/downloads/procmon on Windows. So it seems the blocker for me is I understand what processes should be accessing what files in Windows and not on FreeBSD. Also, fstat does not make it clear to me, screen update to the next, moment to the next, what happened on the disk. I know on Windows I miss no event. I don't know that on FreeBSD. I know which event is important on Windows. I don't on FreeBSD. Worst of all. I don't know how to ask this community, as great as it is, for help, without abusing your time.

Suggestions?

 

[sretalla]

I would start by guessing that we're only looking to see access (currently open handles) to files that are on the pool in question, so thing #1 to do would be sending it to grep and the path of that pool... something like this:

fstat | grep /mnt/pool/

That should limit the output to only things open on that pool, hence pointing you to the culprit user(s) and those files which are (possibly) keeping your pool active.