Github repository for FreeNAS scripts, including disk burnin and rsync support

Github repository for FreeNAS scripts, including disk burnin and rsync support

Muzza

Dabbler
Joined
Jun 17, 2015
Messages
31
Thank you for your help....

I used a "fresh copy" of the script with your fix and now I'm getting

Time: 2020-08-04 10:56:20

=== , model , serial number ===

Name: upslist=ups50@ASrock:161
Status: Output Load: %
Battery Runtime: s Battery Charge: %
Battery Voltage: V

Getting Closer?
 
Last edited:

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
My burnin testing got interrupted when reading back the final test 0x00. My spare computer I'm using to do the burnin wasn't connected to a UPS. I have no bad block on the 4 drives. Is it possible to resume or should I just call it a day and call them good?
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
My burnin testing got interrupted when reading back the final test 0x00. My spare computer I'm using to do the burnin wasn't connected to a UPS. I have no bad block on the 4 drives. Is it possible to resume or should I just call it a day and call them good?
I don't know of any way to resume from where badblocks stopped.

If it were me and I was in a hurry to use the disks, I'd just run the long smart tests on the disks and be done with it. If I wasn't in a hurry, I'd just run the full burn-in again.

But they're your drives, so of course it's your decision to make. :smile:
 

JohnKnee

Dabbler
Joined
Aug 24, 2011
Messages
43
Hi,

Just need a little help or guidance. I recently upgraded from 11.3-U3.2 to 11.3-U41. I have been using the older smart-report.sh script without any problems with the script emailing me the results on a weekly basis. However ever since the 11.3-U4.1 update the script fails to complete and I get a partial list of my drives sent to me including a separate error email.

I thought maybe the newer script would fix that and so I created a new cron task but I get the same error and the script does not get to my sata drives before it gives up.

Below is some info on my drives. The da drives are my data drives and the two ada drives are my mirrored boot drives. The da drives are attached to H310 cards flashed to IT mode on latest lsi 2008 firmware version. There has been no change to my system hardware.

Code:
root@freenas:~ # camcontrol devlist
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 0 lun 0 (pass0,da0)
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 3 lun 0 (pass1,da1)
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 4 lun 0 (pass2,da2)
<ATA ST2000DL003-9VT1 CC32>        at scbus1 target 2 lun 0 (pass3,da3)
<ATA ST2000DL003-9VT1 CC32>        at scbus1 target 3 lun 0 (pass4,da4)
<ST380815AS 3.AAD>                 at scbus2 target 0 lun 0 (pass5,ada0)
<ST380811AS 3.AAE>                 at scbus3 target 0 lun 0 (pass6,ada1)


root@freenas:~ # smartctl --scan
/dev/da0 -d scsi # /dev/da0, SCSI device
/dev/da1 -d scsi # /dev/da1, SCSI device
/dev/da2 -d scsi # /dev/da2, SCSI device
/dev/da3 -d scsi # /dev/da3, SCSI device
/dev/da4 -d scsi # /dev/da4, SCSI device
/dev/ada0 -d atacam # /dev/ada0, ATA device
/dev/ada1 -d atacam # /dev/ada1, ATA device
root@freenas:~ #


This is the output I now get with the old script. Note da0 is missing

Code:
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
|Device|Serial                  |Temp|Power|Start|Spin |ReAlloc|Current|Offline |Seek  |Total     |High  |Command|Last|
|      |Number                  |    |On   |Stop |Retry|Sectors|Pending|Uncorrec|Errors|Seeks     |Fly   |Timeout|Test|
|      |                        |    |Hours|Count|Count|       |Sectors|Sectors |      |          |Writes|Count  |Age |
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
|da1   |5YD4W***                | 29 |30387|  725|    0|      0|      0|       0|     3|  53499232|     0|      3|   1|
|da2   |6YD0X***                | 28 |30387|  721|    0|      0|      0|       0|     3|  51146581|     1|5      |   1|
|da3   |6YD0X***                | 28 |30535|  808|    0|      0|      0|       0|     3|  72882392|     0|      8|   1|
|da4   |5YD4Z***                | 28 |30381|  728|    0|      0|      0|       0|     0|  51640032|     0|      4|   1|
|ada0  |9QZ0W***                | 34 | 2102|  212|    0|      0|      0|       0|     0|   3476734|     0|    N/A|   1|
|ada1  |6PT0Y***                | 35 | 1875|  367|    0|      0|      0|       0|     0|   6422976|     0|    N/A|   1|
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+


This is the error email I am getting

Code:
awk: newline in string 30388
newer... at source line 1


This is the email I get with the "new" latest script. Note it is still missing info on da0, and I no longer get info on the ada0 or ada1 drives.

Code:
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+
|Device|Serial                  |Temp| Power|Start|Spin |ReAlloc|Current|Offline |Seek  |Total     |High  |    Command|Last|
|      |Number                  |    | On   |Stop |Retry|Sectors|Pending|Uncorrec|Errors|Seeks     |Fly   |    Timeout|Test|
|      |                        |    | Hours|Count|Count|       |Sectors|Sectors |      |          |Writes|    Count  |Age |
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+
|da1   |5YD4W***                |28  | 30361|  725|    0|      0|      0|       0|     3|  53283073|     0|          3|   0|
|da2   |6YD0X***                |27  | 30361|  721|    0|      0|      0|       0|     3|  50937775|     0|          2|   0|
|da3   |6YD0X***                |27  | 30509|  808|    0|      0|      0|       0|     3|  72652536|     0|          8|   0|
|da4   |5YD4Z***                |28  | 30355|  728|    0|      0|      0|       0|     0|  51432218|     0|          4|   0|
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+


And the error email message is the same.

Code:
awk: newline in string 30388
newer... at source line 1


Any help on how to fix this or troubleshoot this script for my system is greatly appreciated.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Hi,

Just need a little help or guidance. I recently upgraded from 11.3-U3.2 to 11.3-U41. I have been using the older smart-report.sh script without any problems with the script emailing me the results on a weekly basis. However ever since the 11.3-U4.1 update the script fails to complete and I get a partial list of my drives sent to me including a separate error email.

I thought maybe the newer script would fix that and so I created a new cron task but I get the same error and the script does not get to my sata drives before it gives up.

Below is some info on my drives. The da drives are my data drives and the two ada drives are my mirrored boot drives. The da drives are attached to H310 cards flashed to IT mode on latest lsi 2008 firmware version. There has been no change to my system hardware.

Code:
root@freenas:~ # camcontrol devlist
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 0 lun 0 (pass0,da0)
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 3 lun 0 (pass1,da1)
<ATA ST2000DL003-9VT1 CC32>        at scbus0 target 4 lun 0 (pass2,da2)
<ATA ST2000DL003-9VT1 CC32>        at scbus1 target 2 lun 0 (pass3,da3)
<ATA ST2000DL003-9VT1 CC32>        at scbus1 target 3 lun 0 (pass4,da4)
<ST380815AS 3.AAD>                 at scbus2 target 0 lun 0 (pass5,ada0)
<ST380811AS 3.AAE>                 at scbus3 target 0 lun 0 (pass6,ada1)


root@freenas:~ # smartctl --scan
/dev/da0 -d scsi # /dev/da0, SCSI device
/dev/da1 -d scsi # /dev/da1, SCSI device
/dev/da2 -d scsi # /dev/da2, SCSI device
/dev/da3 -d scsi # /dev/da3, SCSI device
/dev/da4 -d scsi # /dev/da4, SCSI device
/dev/ada0 -d atacam # /dev/ada0, ATA device
/dev/ada1 -d atacam # /dev/ada1, ATA device
root@freenas:~ #


This is the output I now get with the old script. Note da0 is missing

Code:
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
|Device|Serial                  |Temp|Power|Start|Spin |ReAlloc|Current|Offline |Seek  |Total     |High  |Command|Last|
|      |Number                  |    |On   |Stop |Retry|Sectors|Pending|Uncorrec|Errors|Seeks     |Fly   |Timeout|Test|
|      |                        |    |Hours|Count|Count|       |Sectors|Sectors |      |          |Writes|Count  |Age |
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+
|da1   |5YD4W***                | 29 |30387|  725|    0|      0|      0|       0|     3|  53499232|     0|      3|   1|
|da2   |6YD0X***                | 28 |30387|  721|    0|      0|      0|       0|     3|  51146581|     1|5      |   1|
|da3   |6YD0X***                | 28 |30535|  808|    0|      0|      0|       0|     3|  72882392|     0|      8|   1|
|da4   |5YD4Z***                | 28 |30381|  728|    0|      0|      0|       0|     0|  51640032|     0|      4|   1|
|ada0  |9QZ0W***                | 34 | 2102|  212|    0|      0|      0|       0|     0|   3476734|     0|    N/A|   1|
|ada1  |6PT0Y***                | 35 | 1875|  367|    0|      0|      0|       0|     0|   6422976|     0|    N/A|   1|
+------+------------------------+----+-----+-----+-----+-------+-------+--------+------+----------+------+-------+----+


This is the error email I am getting

Code:
awk: newline in string 30388
newer... at source line 1


This is the email I get with the "new" latest script. Note it is still missing info on da0, and I no longer get info on the ada0 or ada1 drives.

Code:
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+
|Device|Serial                  |Temp| Power|Start|Spin |ReAlloc|Current|Offline |Seek  |Total     |High  |    Command|Last|
|      |Number                  |    | On   |Stop |Retry|Sectors|Pending|Uncorrec|Errors|Seeks     |Fly   |    Timeout|Test|
|      |                        |    | Hours|Count|Count|       |Sectors|Sectors |      |          |Writes|    Count  |Age |
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+
|da1   |5YD4W***                |28  | 30361|  725|    0|      0|      0|       0|     3|  53283073|     0|          3|   0|
|da2   |6YD0X***                |27  | 30361|  721|    0|      0|      0|       0|     3|  50937775|     0|          2|   0|
|da3   |6YD0X***                |27  | 30509|  808|    0|      0|      0|       0|     3|  72652536|     0|          8|   0|
|da4   |5YD4Z***                |28  | 30355|  728|    0|      0|      0|       0|     0|  51432218|     0|          4|   0|
+------+------------------------+----+------+-----+-----+-------+-------+--------+------+----------+------+-----------+----+


And the error email message is the same.

Code:
awk: newline in string 30388
newer... at source line 1


Any help on how to fix this or troubleshoot this script for my system is greatly appreciated.
Yikes!
Would you please run these commands and send me the output (disk_info.txt) via PM?
Code:
smartctl -i /dev/da0 > disk_info.txt
smartctl -A -i -v 7,hex48 /dev/da0 >> disk_info.txt
smartctl -i /dev/ada0 >> disk_info.txt
smartctl -A -i -v 7,hex48 /dev/ada0 >> disk_info.txt
smartctl --scan | awk '{printf $1}' >> disk_info.txt
This will give me something to analyze. Thanks
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Spearfoot updated Github repository for FreeNAS scripts, including disk burnin and rsync support with a new update entry:

Changes to disk burn-in script

  • Improved the GitHub 'Read Me' file
  • Dry runs are no longer the default setting
  • Changed badblocks call to exit immediately on first error
  • Set logging directory to current working directory using pwd command
  • Reduced default tests from 5 to 3, so that we run:
    • Short SMART test
    • badblocks
    • Extended SMART test

Read the rest of this update entry...
 

moraga695

Dabbler
Joined
Jan 19, 2012
Messages
21
Hello, I've been using your smart-report.sh script for some time now. However, I just updated to TrueNas Core 12.0 Release version and the script won't run. I suspect the script needs to be modified to be TrueNas compatible. Can you look into this to see what needs to be done?
Thanks very much
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Do these updated scripts work on older (11.2_u8 / 11.3-u5) versions as well or just on the new version?
I will always maintain backward compatibility with older versions of FreeNAS, at least back to 11.2-U8 - which is the version I run in production both here at home and at work.

This update is just for zpool_report.sh; I'll work on the others as time permits.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Hello, I've been using your smart-report.sh script for some time now. However, I just updated to TrueNas Core 12.0 Release version and the script won't run. I suspect the script needs to be modified to be TrueNas compatible. Can you look into this to see what needs to be done?
Thanks very much
I installed TrueNAS 12.0 RELEASE on my dev system just now and the SMART report script (smart_report.sh) works just fine.

What error message(s) do you see when you run the script?

Are you using the latest version from the GitHub repository?
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
I will always maintain backward compatibility with older versions of FreeNAS, at least back to 11.2-U8 - which is the version I run in production both here at home and at work.
Thank you very much for the work you do on this.
 

AdamL.

Cadet
Joined
Oct 27, 2020
Messages
1
Hello,

Please pardon me if this is the wrong location to post this as well as my lack of knowledge when it comes to NAS config's.


I'm wanting to know what are the benefits are to using rsync-invoke.sh to run rsync "directly" and or using rsync-module.sh ? The description of rsync-module.sh mentions running rsync "...with the target being an rsync module on the remote server" does this have any tie to remote access of your NAS? The reason I'm asking is that I've spent a fair amount of time trying to follow a guide for FreeNAS to allow me to access my storage remotely with no avail. It highly suggested I have a stable server prior to attempting remote access, which I thought I did, but now it gives me trouble locally since trying to gain remote access. I just found out about rsync and was wondering if it would be beneficial for both local and remote access. Any explanation for the confusions discussed above would be greatly appreciated.

-AdamL.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Hello,

Please pardon me if this is the wrong location to post this as well as my lack of knowledge when it comes to NAS config's.


I'm wanting to know what are the benefits are to using rsync-invoke.sh to run rsync "directly" and or using rsync-module.sh ? The description of rsync-module.sh mentions running rsync "...with the target being an rsync module on the remote server" does this have any tie to remote access of your NAS? The reason I'm asking is that I've spent a fair amount of time trying to follow a guide for FreeNAS to allow me to access my storage remotely with no avail. It highly suggested I have a stable server prior to attempting remote access, which I thought I did, but now it gives me trouble locally since trying to gain remote access. I just found out about rsync and was wondering if it would be beneficial for both local and remote access. Any explanation for the confusions discussed above would be greatly appreciated.

-AdamL.
Welcome! My scripts you're referring to use rsync to copy filesets between systems, or between datasets on the same system. In a sense, the former is a kind of 'remote access', though somehow I doubt it's what you're looking for.

You'll probably get a more useful response if you post your question on one of the the "Help and Support" subforums listed here:


EDIT: This thread may be helpful, too:
 
Last edited:

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Thank you very much for the work you do on this.
You're very welcome; I'm glad to contribute in a small way to the FreeNAS/TrueNAS project.
 

Jatrabari

Contributor
Joined
Sep 23, 2017
Messages
100
I updated my server to 12.0 U1 yesterday. I am using save_config_enc.sh script to backup my config to email. After the update I got this error in the email:

hex string is too short, padding with zero bytes to length
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
I updated my server to 12.0 U1 yesterday. I am using save_config_enc.sh script to backup my config to email. After the update I got this error in the email:

hex string is too short, padding with zero bytes to length
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
I've posted an update to the repository. Give it a try:
 

Jatrabari

Contributor
Joined
Sep 23, 2017
Messages
100
Finally got the time to get into this.

Thanks for the fix. I didn't test the new script yet because I started to go thru your script to understand it and I tried manually to make a tarball, encrypt it and then decrypt it.

Encryption went fine but when I tried to decrypt I got this error:

Code:
*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
bad decrypt
34371117056:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:/truenas-releng/freenas/_BE/os/crypto/openssl/crypto/evp/evp_enc.c:583:


... and no decryption. I tried to troubleshoot the command on the Github page for decryption but couldn't get it to work.
 
Top