Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Never heard that, I don't see how putting scripts on a pool can cause issues with SMART or scrubs.

I'm doing exactly that (before I was putting the scripts in the /root on the system drive but I got tired of copying the scripts each time I needed to re-install (yeah, USB sticks are crappy...)) and I don't have any problem ;)
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
python version of the drive identification script: http://pastebin.com/jYWMcHwn

python version of the CPU and drive temperatures script: http://pastebin.com/ZAtzs18r

The drives and CPUs are enumerated automatically, so there's no need to edit the script to match your system.

Enjoy!

EDIT: simplified drive enumeration
EDIT: added temperatures script
 
Last edited:

SavageCore

Dabbler
Joined
Jun 24, 2015
Messages
17
Hi

Small error in the SMART report script I believe.

When a drive is failing the output of
Code:
smartctl -l selftest /dev/ada4 | grep "# 1"


will go onto 2 lines with the following:

Code:
# 1  Extended offline    Completed without error       00%      3285         -
1 of 1 failed self-tests are outdated by newer successful extended offline self-test # 1


That results in the following error:

Code:
awk: newline in string 3285
newer... at source line 1


So in order to fix it I changed line 44 to

Code:
-v lastTestHours="$(smartctl -l selftest /dev/"$drive" | grep "# 1" | awk '{ printf "%s", $0 }' | awk '{print $9}')" '\
 

Bhoot

Patron
Joined
Mar 28, 2015
Messages
241
@Bidule0hm can you please modify the temperature script to send emails of the temperature to me? I would like to see how my disks/cpu temps are doing when my house is shut and the freenas is running.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Yep, when I saw the thread about the script who set the fans speed I got the idea to do a script who emails you if the temp go over a certain threshold, I just need time to do it ;)
 

Marcet

Contributor
Joined
May 31, 2013
Messages
193
@Bidule0hm Thanks for those very useful scripts : adopted !
 
Joined
Dec 2, 2015
Messages
730
Yep, when I saw the thread about the script who set the fans speed I got the idea to do a script who emails you if the temp go over a certain threshold, I just need time to do it ;)
FreeNAS can already email if the temperatures are too high. Services=>SMART. Set the "Critical" temperature. I tested it, and it works for me.
 

Marcet

Contributor
Joined
May 31, 2013
Messages
193
FreeNAS can already email if the temperatures are too high. Services=>SMART. Set the "Critical" temperature. I tested it, and it works for me.
You're right, but a custom script could report also on high cpu temperature or even low fan speed.
That would be a plus.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
And it doesn't work for me, and for some other people too. We don't know why, it's just like that and the devs doesn't seem to bother too much with this bug.
 
Joined
Dec 2, 2015
Messages
730
And it doesn't work for me, and for some other people too. We don't know why, it's just like that and the devs doesn't seem to bother too much with this bug.
Interesting. It worked for me when I tested it, but I'd like to learn more about the issue. Is it an intermittent failure, or is it a case that it works perfectly fine for some installations, and not at all on others? I searched the bug tracker, and all the issues I can find that seem to be for this function are claimed to be fixed. Did I miss the bug in the tracker?
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Yep, works fine forme some people but it never worked for me. I don't know if there's a bug currently open but it doesn't change anything much anyway.
 

ethereal

Guru
Joined
Sep 10, 2012
Messages
762
In the smart testing script i edited

smartctl -l selftest /dev/"$drive" | grep "# 1 \|Num" | cut -c6-

to

smartctl -l selftest /dev/"$drive"

and now the email lists not just the last smart test run on the drives it lists the smart test history. I wanted the history so i could easily see that my short tests were running daily and the long tests weekly on all my drives.
 

Krowvin

Explorer
Joined
Jul 24, 2014
Messages
60
This is fantastic!

Not only did I setup my own schedule based on all your scripts, but I was able to put together my first script using your code as well as code from Vivek Gite.

I've been wanting for a while now to have something that could email me when one of the local ips went down. It's also great if I forget to turn a plugin back on after updating.

I think I've got a working solution, but please feel free to look at it and build upon it! (If you do let us know haha)
Hopeful what this will help someone else trying to do the same!

Ping Email Report


In the hosts section i'm able to add both local and public IPs (Few wordpress sites that I manage), and when they go down i'll get an email! What's even more cool, is that I think i've set it up in such a fashion that it wont email me if the site isn't down! (Thanks to Vivek's post mentioned previously)

Add whichever addresses/domains you would like to monitor in the HOSTS=" " section
Change the email address to the one you want this sent to.

The output: http://pastebin.com/a5XS7SVj

The script: http://pastebin.com/1g5Vv6fK Don't forget to change the hosts and email values


P.S. @cyberjock I wasn't really sure where to post this so I thought i'd add it to the origin!

Edit: Looks like this only works with reporting 1 host in the list, if more than one is down it spits an error and only reports 1 of the hosts as being down via email.
 
Last edited:

wah00kid

Dabbler
Joined
Nov 24, 2015
Messages
37
Thought I would post this here as I was able to use this thread to get some temperature reporting graph set up and I know others had asked for it so I thought I would share my setup as it might benefit others. Show no mercy I can take it but know this was my first attempt at javascript.

The idea is that I use a modified version of the SMART reporting script in the OP.
The email that is generated then gets handled by a google script and dumped into a google spreadsheet.
When I want to see how my drives have been doing I just open the spreadsheet and look at the graph.

Here is what I see when I open my spreadsheet:

hddtemps.PNG


You might notice the data in my spreadsheet is a bit janky... this is purely due to my crap coding maybe someone who likes this can help me out.

Okay here is the script I modified from the OP put on my server and run every 15 mins with a cron job:

Code:
#!/bin/sh

### Parameters ###
drives="ada0 ada1 ada2 ada3 ada4 ada5"
email="******@gmail.com"
subject="hddtemps"
logfile="hddtempreport.tmp"

### Set email headers ###
(
    echo "To: ${email}"
    echo "Subject: ${subject}"
    echo "Content-Type: text/html"
    echo "MIME-Version: 1.0"
    echo -e "\r\n"
) > ${logfile}

### Set email body ###
(
    date "+%m-%d %H:%M:%S"
for drive in $drives
do
    temp="$(smartctl -A /dev/${drive} | grep "Temperature_Celsius" | awk '{print $10}')"
    echo -n ", "
    echo -n "$temp"
done
    echo ""
) >> ${logfile}

### Send report ###
sendmail -t < ${logfile}
### rm ${logfile} ###


This is an example of the logfile that is generated and emailed:

To:******@gmail.com
Subject: hddtemps
Content-Type: text/html
MIME-Version: 1.0


03-16 21:03:34, 36, 35, 37, 32, 33, 31

With my gmail I set up a label based on that subject line. The label will hold all the emails for me and I could turn the notifications of those emails down to once a week. This way I don't even see that I am getting these emails every 15 minutes. It might be possible to turn the notifications of that label all the way off but I haven't figured out how yet. These emails stay unread in that label which is important for the next piece.

Here is my script that I have on google scripts. I set it to automatically run every 6 hours or so:

Code:
function hddtempsmailtosheet() {
  var ss = SpreadsheetApp.openByUrl('https://docs.google.com/spreadsheets/d/*************************************/edit');
  var label = GmailApp.getUserLabelByName("hddtemps");
  var unread = label.getUnreadCount();
  if (unread > 0)
  {
  var threads = label.getThreads(0,unread);
    for (var i = threads.length -1; i>=0; i--) {
     var messages = threads.getMessages();
      for (var j = 0; j < messages.length; j++) {
      var temps = messages[j].getBody();
      var array = temps.split(',');
      var sheet = ss.getSheets()[0];
      sheet.appendRow([array[0], array[1], array[2], array[3], array[4], array[5], array[6]]);
      messages[j].moveToTrash();
    }}}
  
}


Okay so yeah that is my first javascript using a bunch of the examples in the google help docs. This took forever even though for most of you all this is probably pretty straightforward. As is it though it still doesn't work quite right.

The idea is that every 6 hours the script runs and goes through my email with the hddtemps label and sucks up all the email data and puts it in an array and parses it and then feeds it to that spreadsheet (see image at top of post) and deletes the emails. I had to manually create the spreadsheet and copy the link over and grant all the permissions for the script to see my email and my spreadsheet. Obviously the script didn't generate the graph although that does seem possible with the google scripting. I made the graph once initially and then all the new data gets added automatically.

So yeah a semi-automatic setup to create a drive temperature graph!

The problems are numerous and include:

That my email script is built only for my particular set up of drives.
The google script has the same problem where the appendrow command is tied to my setup.
My Google script does this weird thing where is also sends the un-parsed emails to the spreadsheet(See the bottom of the screenshot). These end up being duplicated and I am not losing any data however this is super annoying and could lead to another problem if I wasn't going in every other day or so and deleting out all these unneeded lines from the spreadsheet.

Another potential problem is that google will only let spreadsheets get so big. I am not certain if it is based on total number of rows or total number of cells. It seems like it would be easy to create a script that would delete out a certain number of rows from the spreadsheet before adding new data in.

I initially built this for every 15 minutes because I was really trying to track the temps because I kept creeping over 40C. on a couple of my drives. This is just the graph from the restart of my server with the new airflow I have a huge spreadsheet I just made a copy of that has about 2 weeks of data. I just did a different airflow layout in my case and so I will continue to monitor this at that frequency and then hopefully I can reduce the frequency.

Hope this helps someone as I spent a lot of time on it. And if you have any tips on my scripts then would be much appreciated.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
I wanted to graph the temps using pure text since I made the script but I just don't havethe time right now... Actually I have a dozen of tabs related to my threads here I keep open in my browser just to remember me of what I need to do... I'm super busy at the moment so I find some time to answer the posts but not really to make my scripts/apps/tutos/whatever better but one day or another I will do what is left to do.

So, anyway, thanks for the sharing ;)
 

lazybones

Dabbler
Joined
Jan 16, 2016
Messages
45
Hey there, first of all I want say your work is awesome, but i noticed this morning that the script who's supposed to display, GPTID's and Serials, the "Serial" part doesn't show any serials.

Any idea what might be causing this?

Thanks for awesome scripts again!
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Can you post the version of FreeNAS you use and the output of smartctl -i /dev/your_drive for one of the drives please?

Thanks ;)
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
@biduleOhm,

Have you thought about putting your scripts in github so there is a logical version control on changes that everyone can see? I REALLY despise pastebin, for just about everything....
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
And I REALLY despise github (well, git actually) :)

I usually use SVN for my projects but these scripts aren't a project big enough so I don't think the advantages/disadvantages ratio would be high enough to justify it.

Plus there's a changelog and each time I do a modification I post here what has changed, so if you "watch" this thread you'll be notified ;)
 

lazybones

Dabbler
Joined
Jan 16, 2016
Messages
45
Can you post the version of FreeNAS you use and the output of smartctl -i /dev/your_drive for one of the drives please?

Thanks ;)


I figured out what was wrong, I didn't run the script as root, once i did it works. I am sorry for wasting your time.
 
Top