multi_report.sh version for Core and Scale 3.0

[mistermanko]

Happy new year!
Still works like a charm, even after migrating my existing configuration incl. this script to SCALE. It simply updated itself. Truly magnificent work this is.
I'm also flattered to still have an existent custom config option inside the script - which will soon be removed as I understand - it's also not much needed anymore, I uninstalled the crooked drives from my system that made Joe hack-up the custom stuff in the first place.

Many thanks to @joeschmuck and @TooMuchData and all the others that contributed, for the hard work that was put into this!

 

[Davvo]

I agree with @mistermanko , we are spoiled so much! Thank you!

 

[joeschmuck]

@mistermanko and @Davvo, thanks for the kind words. Glad to hear the script is still working without any trouble. Of course, as always, if you see anything fishy, let me know and we will figure it out. I've already started on version 2.0 and yes, the custom hacks have been removed.

I found another irk that I want to fix for my own peace of mind, and have reformatted all the -config and -help/-h screens to fit a 80x24 screen (something I wanted to do a long time ago), and I have one more thing I wanted to make work a little better. I'm hoping that anyone who has any issue with the script will reach out before late February when I plan to release version 2.0.

 

[Deeda]

Heya @joeschmuck, bit of a strange one. Have been running version v1.6e on TrueNAS Core 13.0-U3.1 for a while now. Was working fine. I shut down the server today to check some fans, etc. Once I restarted and ran the script again, I am seeing the error:

line 2458: ((27957 - -) / 24): syntax error: operand expected (error token is ") / 24)")

So I upgraded to version v1.6f, created a new config file, and am receiving the same error. The line it appears to be referring to:

if [[ $lastTestHours != "" ]]; then let testAge=$(((($onHours - $lastTestHours) / 24))); fi

Cheers

 

[ArchatParks]

I get this error on one machine:
./multi_report.sh: line 2460: ( / 8760): syntax error: operand expected (error token is "/ 8760)")

 

[joeschmuck]

Heya @joeschmuck, bit of a strange one. Have been running version v1.6e on TrueNAS Core 13.0-U3.1 for a while now. Was working fine. I shut down the server today to check some fans, etc. Once I restarted and ran the script again, I am seeing the error:

line 2458: ((27957 - -) / 24): syntax error: operand expected (error token is ") / 24)")

So I upgraded to version v1.6f, created a new config file, and am receiving the same error. The line it appears to be referring to:

if [[ $lastTestHours != "" ]]; then let testAge=$(((($onHours - $lastTestHours) / 24))); fi

Cheers

@Deeda and @ArchatParks

Attached is a quick and dirty update to provide me some additional data, I added the word "test" to the title so you would know it's not a fully vetted script, but it should run just fine normally. When you run the -dump parameter it will not include a copy of the html file, well sort of. You can't attach a file while it's still being generated so I picked a good place to use what has been created so far, it should include all the information I need to help out. The new attached file is called "output.html". If you look at it you can tell the file was not terminated properly. I will fix that later but for now I'm just looking to get the data from you.

I need you to run the script using the -dump parameter. This will generate an email with a lot of attachments. It consists of all the drive data, your multi-report configuration file, and now the HTML file. It's easier on you than trying to post an email output.

Next open up a conversation with me and attach all the email attachments.

In the conversation, please describe the error message (cut/paste from this thread if you desire) because what I can't capture it. It's outside the running script. Also, was there anything that had changed form before if the script was running fine? Think hard.

I'm back at work so my time is limited and I'm taking care of my wife so I will jump on this when I have the time.

 

[Deeda]

You won't believe it, it's not showing the error now. I must of ran the script a dozen times yesterday as I was closely monitoring temps on the hard drives. I ran it again today and the error isn't occurring. I haven't restarted the server or changed any settings do with the script.

Sorry if I have wasted your time. Did you still want me to do the testing? Or no point?

 

[joeschmuck]

Sorry if I have wasted your time.

Odds are one of your drives reported the last SMART Test in a slightly different format. It happens and I can't explain why. But if you have a drive which has a SMART Test with a hour value of 27957 (or maybe power on hour), if you could perform the dump and send it to me, I could still see what might be going on. Also, for the email that was generated when the problem was occurring, a cut/paste of that hard drive data. I need to examine for formatting differences, even though it won't be exactly proper it's the best I think we will get. So two pieces of information please, if you have a drive with the reported SMART Test hours, or even Power On Hours that matches or is close to the value you posted.

If you would just like to do that -dump command and send it all to me in an email, feel free, just as before. You should still have my email address.

But off to work I go now, so much fun. I can't wait for retirement to happen.

 

[joeschmuck]

Odds are one of your drives reported the last SMART Test in a slightly different format. It happens and I can't explain why. But if you have a drive which has a SMART Test with a hour value of 27957 (or maybe power on hour), if you could perform the dump and send it to me, I could still see what might be going on. Also, for the email that was generated when the problem was occurring, a cut/paste of that hard drive data. I need to examine for formatting differences, even though it won't be exactly proper it's the best I think we will get. So two pieces of information please, if you have a drive with the reported SMART Test hours, or even Power On Hours that matches or is close to the value you posted.

If you would just like to do that -dump command and send it all to me in an email, feel free, just as before. You should still have my email address.

But off to work I go now, so much fun. I can't wait for retirement to happen.

Problem fixed a few days ago, forgot to mention it here. This was a SMART Test recognition issue. I never checked for "Offline" SMART Test results before, this was the first reported to me. A simple update and all is working good. It will be included in the next posted update.

 

[lx05]

Hi together,

first post on the forum so...
Thanks a lot for the scripts, they are great. However the email report says that the "last test age" for my boot ssd is "96", which obviously causes a warning. I have a short smart test configured for my 5 drives (2pools with 2 drives + Boot ssd) and it runs every 2 days. The result for the HDDs are fine ("last test age"=0). Do I need to configure something differently or is my smart task not configured properly. I checked the "scan all disk" box in the smart settings.

Also I always wondered if it is possible to add the date to the subject of the report email. I'm not good at scripting so if it is, please tell me how it's done. Otherwise all my report emails always have the same subject/header.

Thanks a lot in advance!

 

[joeschmuck]

However the email report says that the "last test age" for my boot ssd is "96"

First if you can post the output of the SMART data for the boot drive in question so I can see what the drive is reporting. Someone else might see the issue and be able to help you before I can get around to it as I will be at work in a few hours so I will not be able to do anything today.

Next I will need you to run the script using the -dump parameter which will generate an email with lots of attachments. Send me a private message (Start Conversation) and attach all the files from the email so I can examine the data to determine what is going on and provide you the proper solution. Do not post this data from the -dump in this forum, it will include your actual email address and it's best to not make it wide known, unless you don't care about it but most people do care. I will not share your email with anyone.

So depending on the data there are a few things you can do but I need to see the data so I provide you the correct answer.

Also I always wondered if it is possible to add the date to the subject of the report email. I'm not good at scripting so if it is, please tell me how it's done. Otherwise all my report emails always have the same subject/header.

Doesn't your email client sort your emails by date?

And welcome to the forums! I hope you enjoy your time here.

 

[ChrisRJ]

Also I always wondered if it is possible to add the date to the subject of the report email. I'm not good at scripting so if it is, please tell me how it's done. Otherwise all my report emails always have the same subject/header.

As an optional behavior that may make sense. However, I personally prefer to have a fixed subject here.

 

[lx05]

I attached the email as a PDF and I send you my "-dump email".

I use Gmail, and it automatically groups the report Emails into this Gmail "Conversation view" (which is great for actual conversation) and then it's a bit unorganized. The grouping is based on the email's subject, so I thought I could make this individual for each report by adding the date, I just don't know how to use the variables in the script.

Thanks a lot for quick response!

 

[joeschmuck]

@lx05 In looking briefly at the drive data, it doesn't make any sense to me. The dump files indicate something different from what you posted here. I will run it through the simulator and see what happens. If that does not work well then I will send you a newer script to run a -dump on. I'm collecting a little more data in the newer version to aid in troubleshooting problems. It's much easier than asking someone to do a dozen things and it also speeds things up for everyone involved. The discrepancy I see is the drive dump data shows the Short Test data hours are recycling at every 255 hours. This isn't the first time I've seen weird stuff like this. But what is weird is, smartctl -a and smartctl -x have different results. The -x actually shows all the SMART tests properly, not rolling over at 255 hours. This means that I need to figure out how to combat this type of discrepancy. You could just ignore the Last Test Age for this one drive, the feature is already built into the script you are running. Use -config and go into Advanced, then S for Custom Configuration. Near the end of the questions it will ask you about Last Test Age, answer yes and then when the drive serial number in question comes up, tell it to ignore it. This is NOT my preferred way to handle things but I will never be able to address every oddity which is why I developed this override feature. I will try to fix the script to recognize your drive but it will take time and I need to ensure I do not break other parts. That is what actually take all the time, testing to ensure I didn't break something else. Making to work for you is actually not that hard, making sure I didn't break it, that is the hard part.

I'll contact you in the other PM you sent me.

 

[lx05]

I figured out how to edit the email's subject and thought I post it here if somebody ever runs into the same issue and has similarly little understanding of bash.
In the "Auto-generated Parameters" section (line 626) I added a new variable

###### Auto-generated Parameters
softver=$(uname -s)
host=$(hostname -s)
truenas_ver=$(cat /etc/version)
testdata_path="data"
date_today=$( date '+%F_%H:%M:%S' )

This new variable then contains date and time (the format can be changed, mine will look e.g. "2023-01-10_23:32:09".

Then in the "create_email" function (line 2321), I added this variable to every scenario's respective email subject

create_email () {
### Create New Email Header - Set Subject Line

## Test if there is a Warning Message and Setup Subject Line
if test -e "$logfile_critical"; then
subject="*${date_today}_*CRITICAL ERROR* SMART Testing Results for ${host} *CRITICAL ERROR*"
elif test -e "$logfile_warning"; then
subject="${date_today}_*WARNING* SMART Testing Results for ${host} *WARNING*"
elif [[ $disableWarranty == "false" ]]; then
if [[ ! $logfile_warranty == "" ]]; then
subject="${date_today}_*Drive Warranty Expired* - SMART Testing Results for ${host}"
else
subject="${date_today}_SMART Testing Results for ${host} - All is Good"
fi
else
subject="${date_today}_SMART Testing Results for ${host} - All is Good"
fi

### Set email headers ###
(
echo "From: ${from}"
echo "To: ${email}"
echo "Subject: ${subject}"
) > ${logfile_header}

cat $logfile >> $logfile_header
force_delay
### Send report
sendmail -t -oi < "$logfile_header"
}

It works fine for me.

And just wanted to give a huge shoutout to this forum. Only been here for 2 days and already received lots of help and everybody is super friendly. Way better than many other forums I've been to!!

 

[joeschmuck]

I figured out how to edit the email's subject and thought I post it here if somebody ever runs into the same issue and has similarly little understanding of bash.

Great job! You have done what unfortunately many would not do, that is put your foot forward and jump in to solve the problem yourself. This is much better than just asking for the answer.

and has similarly little understanding of bash.

That is exactly how I started. I had no clue about BASH, and at times I still don't. Syntax is everything.

Only been here for 2 days and already received lots of help and everybody is super friendly. Way better than many other forums I've been to!!

Glad to hear that. As a community we try to be helpful but once in a while you get a bad apple. You just need to ignore those trolls.

 

[lx05]

Hi together,

I had an idea, but somehow it didn't work.
I wanted to change the Emails From_Name. I wanted it, so that The "From Name" is taken from TrueNas WebGUI or can be changed via the config file. But how can I change the emails from_name?
I tried adding it in the emails header in the create_email() function like this

### Set email headers ###
(
echo "From: ${fromName} <${from}>"
echo "To: ${email}"
echo "Subject: ${subject}"
) > ${logfile_header}

and added the variable in the upper "Auto-generated Parameters" section. The "from" variable i left since it works perfectly.

Obviously I didn't come up with this (as I said, very little knowledge, but I'm trying). I found all of this in another script I use (here).

They used what I suppose is a search in the database

fromEmail="$(sqlite3 /data/freenas-v1.db "select em_fromemail from system_email;")"
fromName="$(sqlite3 /data/freenas-v1.db "select em_fromname from system_email;")"

and then this in their "send email" section

ee <<- EOF
From: ${fromName:="${host}"} <${fromEmail:="root@$(hostname)"}>
To: ${email}
Subject: ${subject}
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="${boundary}"
Date: $(date -Rr "${runDate}")
Message-Id: <${messageid}@${host}>
EOF
} > "${logfile}"

(the .= stuff is just there I think to still send an email (have a value in the variable) if the database request gone wrong.)
The script works completely fine and if I change the From Name in the WebGUI it changes in the emails.
Couldn't this be added to the script? I just wanted to copy it over.

How do i have to edit the email headers? I manually checked if the Database request works, and it gives me the correct reading from the value saved in the WebGUI.
I think I understand how the "-config" program works and could add a section that asks for a From_Name value different to the WebGUI (just like the from email adress can be changed) but that won't help if I won't get the email headers right.

I'm actually quite happy I came this far but no I don't know what to do.

 

[lx05]

I just figured it out. Here's an overview.
I changed the following things (if all of those changes are fine, this could maybe be added to a further version)
Changes printed bold

Added a variable in the "Auto-generated Parameters" section

softver=$(uname -s)
host=$(hostname -s)
fromName="$(sqlite3 /data/freenas-v1.db "select em_fromname from system_email;")"
truenas_ver=$(cat /etc/version)
testdata_path="data"

edited the "create_email()" function

### Set email headers ###
(
echo "From: ${fromName:="truenas"} <${from}>"
echo "To: ${email}"
echo "Subject: ${subject}"
) > ${logfile_header}

cat $logfile >> $logfile_header
force_delay
### Send report
sendmail -t -oi < "$logfile_header"

put in stuff into the -config program

C)
clear
echo "Email Settings"
echo " "
echo "Current email address(s): "$email" "
echo "separate multiple email addresses with a comma "
echo 'Enter nothing to accept the default or change it to something new.'
echo -n ": "
read Keyboard_email
if [[ ! $Keyboard_email == "" ]]; then email=$Keyboard_email; fi
echo "Set Value: "$email
echo " "
echo " "
echo "Current from address: "$from" "
echo 'While most people are able to use the default "from" address,'
echo 'Some email servers will not work unless you use the email address'
echo 'the email address the server is associated with.'
echo 'Enter nothing to accept the default or change it.'
echo -n ": "
read Keyboard_email
if [[ ! $Keyboard_email == "" ]]; then from=$Keyboard_email; fi
echo "Set Value: "$from
echo " "
echo " "
echo "Current from Name: "$fromName" "
echo 'Use this setting to customize the sender`s name'
echo '(e.g. "TrueNas_SMART_Reporter"). By doing so, you'
echo 'can give the E-Mails from this script a custom'
echo 'sender`s name to filter or sort those in your'
echo 'E-Mail client/programm'
echo 'Enter nothing to accept the default or change it.'
echo -n ": "
read Keyboard_email
if [[ ! $Keyboard_email == "" ]]; then fromName=$Keyboard_email; fi
echo "Set Value: "$fromName
echo " "
echo "returning..."
sleep 2
;;

added same thing in the N) routine (create new file) part (roughly line 5425)

changed "This sets the default values up for the external config file."

(
echo "#" $programver
echo "#"
echo "# This file is used exclusively to configure the multi_report version 1.6c or later."
echo "#"
echo "# The configuration file will be created in the same directory as the script."
echo "#"
echo "# The configuration file will override the default values coded into the script."
echo " "
echo "###### Email Address ######"
echo "# Enter your email address to send the report to. The from address does not need to be changed unless you experience"
echo "# an error sending the email. Some email servers only use the email address associated with the email server."
echo " "
echo 'email="'$email'"'
echo 'from="'$from'"'
echo 'fromName="'$fromName'"'
echo " "

Feedback is very welcomed!

Questions:
I don't remember how the initial first setup of the config file works and if I have to add something there (is it just the "N)" routine?), so I don't know if it only works for me, but I can't do a complete reinstall right now just to check it?

Couldn't the database pull command be used for the sender's email address as well (and stored in the "from" variable). I tried adding it where I added the "fromName" variable (in the Auto-generated Parameters" section") but it didn't work.

 

[lx05]

Okay quite stupid: I didn't add the fromName variable to the config.txt. If I do, that nothing is automatically updated.

So even more questions:
How do I save the value of the fromName variable in the config file?
How can it be updated from the WebGUI value if untouched and otherwise not. Maybe a use_custom_from_name boolean and the same for use_custom_from_address. If set to no then it will use the WebGUI values if set to yes the custom ones.

If I don't put the fromName variable in the config.txt it will always display the WebGUIs value when i run the -config command.


So honestly, all the editing of the -config file is useless because if it won't save. And if it saves (as with the "from" variable) then it won't update.

 

[joeschmuck]

I found all of this in another script I use (here).

Wow, that is very similar to my script, even some of my newer changes seem to have migrated into it. I didn't even know it was out there in this version, I recall the original version.

Feedback is very welcomed!

It looks like you are learning quite quickly. But you should understand what the commands are doing. Remember that you are running commands on someone's NAS, should there be anything done that does any damage, shit will hit the fan. I'm not saying do not make changes to your copy of the script but be careful of what you recommend someone else to do. Also, have you tested it well, in all situations, and on both Core and Scale?

I will have to test this out to see how it works and what it looks like, but maybe this weekend.

So honestly, all the editing of the -config file is useless because if it won't save. And if it saves (as with the "from" variable) then it won't update.

You can't just go changing things without understanding the script and how everything is intertwined. Read it over, you will figure it out.