TN Scale - are cron jobs working in general in 22.12.3.3 ?

[flashdrive]

Hi,

/ui/system/cron

can someone confirm?

I am asking because of:

How to schedule a shutdown of the system?

Hi, as asked here: Is there another option to shut down the system automatically? https://www.truenas.com/community/threads/tn-scale-shutdown-command-via-cron-job-for-scheduled-shutdown-how.112256/

www.truenas.com

 

[sfatula]

I and endless others have tons of cron jobs on 22.12.3.3 and yes they work fine.

 

[CSylvain]

I also confirm that my crons no longer work since the last update.
According to feedback, this is happening during an update phase, and not a version bug.

I checked the file /etc/cron.d/middlewared
And test a command line cron like:

Code:

midclt call cronjob.run 8 true

And nothing happens.

Same with this command:

Code:

echo "AAAAAA" > /tmp/aaaaa.txt

It doesn't write anything to the /tmp folder

Another thing where I'll spend hours DIYing.

 

[sfatula]

For the first one (maybe even the second one), try the full path to midclt (/usr/bin/midclt), not familiar with the syntax of the call, so can't comment if syntax is right, you may have looked. In general, you always want to use full paths for commands in cron on most platforms. Though I am not clear if that midclt command is in the cron job fields in the Scale UI, or, if you are just trying that to run a cron from the command line?

There could be an update problem, not sure, but try the full paths and see if it makes a difference for either. Assuming you are trying to run the midclt in a cron job, though, that doesn't make sense?

 

[CSylvain]

I just tried it on my test environment, and it works after an update.
We can therefore rule out this possibility.

The midclt command works well because it returns me an id which must be in the SQLite database (jobs), but nothing happens, the same in the logs: /var/db/system/syslog-**** ***********/log/jobs

I have a cron which runs every 5 minutes, I wonder if there is not an ID limit which causes corruption in the SQLite file?

Because each time I run it, I always get the same ID:

Code:

midclt call cronjob.run 8 true
43712

It remains on the ID: 43712

 

[sfatula]

So, I tried the cronjob.run in non Cobia, I am not running Cobia. The syntax I used was:

midclt call cronjob.run 12

Not sure what the trailing "true" is for that you showed but it's not needed. Maybe it messes it up?

I got the cron number from:

midclt call cronjob.query | jq -r

And I took a quickie cron to see if it would run. It does in fact show up in the Scale history log found in the UI via clicking on the jobs icon on top, and then history.

So I ran it again and got a new job number.

I am getting syntax from:

http://192.168.3.13/api/docs/ where the IP is your scale ip.

 

[CSylvain]

The last syntax is included in the cron file: /etc/cron.d/middlewared
Your command shows me all my crons perfectly.

Locally on a VM it works well, but it does not work on a cron every 5 minutes.

So I'm currently exploring my /data/freenas-v1.db file to see if I can reset this id (AUTO_INCREMENT) of the jobs.

Otherwise I'll have to add this to my Webmin.
To anticipate this question on Webmin : https://www.truenas.com/community/t...ras-truenas-13-0-u4.108515/page-2#post-772346

 

[sfatula]

So, you are saying midclt isn't working in a cron job? I vaguely recall a big discussion about midclt system.shutdown (or something like that) not working from a cron. It turned out for them, as weird as this sounds, they had set the BIOS time incorrectly. BIOS clock should be set to UTC. Probably not your issue, but who knows. An every 5 minute cron job shouldn't need correct time.

So, see if I understand correctly, in the cron job, you are using the full path to midclt, and, you've removed the true part? And it runs every 5 minutes but is not working. Care to post the actual cron job? Perhaps it aborts before getting to that part? Perhaps a syntax error, wrong shell, whatever?

 

[CSylvain]

I use Truenas Scale's cron management, which is based on the "/etc/cron.d/middlewared" file: https://pastebin.com/L4P88M0X
Each Cron that you add in Truenas Scale is saved in this file, you can clearly see my 8 crons.
So far everything is OK.

So I refined my search, to understand that commands containing "midclt call cronjob.run * true" no longer work at all.
The one that is every 5 minutes is in 1, but it gives me another ID: 42357

The problem is therefore not partly a problem with the crontab service, but with all the commands: "midclt call cronjob.run * true"
We are therefore sure of the Python exploration, something that I would not want to spend hours looking for in the Python libraries of Truenas.

If I could know how it counts jobs so that I can reset them, that could help me know if the problem comes from there.

 

[sfatula]

Well, all I know is all mine work fine and they also have the "true" so presume something else is up. Since yours do not, instead of spending a bunch of time trying to figure it out, why not just submit a ticket? I've gotten answers that way before.

You've looked in the UI though, at Jobs -> History and don't see your every 5 minute job under cronjob.run, right?

 

[CSylvain]

Haaaa, I think you have put your finger on the problem, they have all been pending since October 31, 2023:

We can clearly see the IDs, which will explain why I keep getting the same ID all the time!

The big question, how can they be canceled?

 

[sfatula]

I don't know about cancelling, but, you can determine why they may have failed (in some cases at least) by expanding the box via that down arrow on the right. I see some for 11/6, though those may have been your manually run ones?

I wonder what they are waiting for!? Does the down arrow on those provide any hint maybe? I've never seen Waiting before. Is perhaps crond not running?

What does this show?

ps -C crond

 

[CSylvain]

No results, the same on my VM which works on the cronjobs side.

For information, I use the Scale version (Debian), the service is called cron:

Code:

ps -C cron
    PID TTY          TIME CMD
1273623 ?        00:00:00 cron

I'm going to move forward with these elements, I'm going to eventually figure out how to remove these expectations.

 

[sfatula]

I am using Scale, here's what I see:

ps waux | grep cron root 7369 0.0 0.0 5740 2288 ? Ss Nov03 0:00 /usr/sbin/cron -f root 69600 0.0 0.0 1616 320 ? Ss Nov03 0:00 crond -f -l 2 root 109034 0.0 0.0 2384 488 ? S Nov03 0:00 busybox crond -f -l 0 -L /dev/stdout

 

[CSylvain]

The update betrayed me, I was doing minor updates, so I am in Bluefin (22.12)!

I just updated to Cobia (23.10), and I now have crond as well as in busybox which did not support it before under Bluefin.
Verdict, everything works!

A big thank you for your patience, I think that this thread of our exchanges will help those who are in the same situation, make sure that you are on a Cobia version and not Bluefin which seems to be buggy on the cronjobs!

 

[sfatula]

I am on 22.12.4.2 but it is working for me as is. I am not going to Cobia until the .2 release as is normal for me. I am not a risk taker!

Anyway, I am glad you got it working!