Github repository for FreeNAS scripts, including disk burnin and rsync support

Github repository for FreeNAS scripts, including disk burnin and rsync support

MrToddsFriends

Documentation Browser
Joined
Jan 12, 2015
Messages
1,338
Using FreeNAS 11.1

I downloaded and tried the disk-burnin.sh script and am running into an error. When I run the script, I get: Syntax error: newline unexpected (expecting word)

From the documentation in
https://github.com/Spearfoot/disk-burnin-and-testing
Code:
The only required command-line argument is the device specifier, e.g.:

./disk-burnin.sh sda

...will run the burn-in test on device /dev/sda
 

VintageGold

Cadet
Joined
Mar 12, 2018
Messages
6
From the documentation in
https://github.com/Spearfoot/disk-burnin-and-testing
Code:
The only required command-line argument is the device specifier, e.g.:

./disk-burnin.sh sda

...will run the burn-in test on device /dev/sda
Thanks for pointing that out. I get the same error when I tried again and substituted "sda" for "ada0" based on the disk name I can see when I look in the FreeNAS console or run the command
Code:
camcontrol devlist
Am I looking in the wrong place?
 
Last edited:

MrToddsFriends

Documentation Browser
Joined
Jan 12, 2015
Messages
1,338
Thanks for pointing that out. I get the same error when I tried again and substituted "sda" for "ada0" based on the disk name I can see when I look in the FreeNAS console or run the command
Code:
camcontrol devlist
Am I looking in the wrong place?

If you are still getting the same error ("Syntax error: newline unexpected (expecting word)") when running ./disk-burnin.sh ada0 with the first argument ada0 being the device to be chosen for the burnin test then there's something weird going on. camcontrol devlist is of course not a bad way to get access to device names. Hopefully @Spearfoot has some time to look at it.

While I'm running 11.1-U2 right now, it has been a while since I ran Spearfoot's disk burnin script for the last time, so I can't say for sure that it is supposed to run as-is on 11.1.
 
Last edited:

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
If you are still getting the same error ("Syntax error: newline unexpected (expecting word)") when running ./disk-burnin.sh ada0 with the first argument ada0 being the device to be chosen for the burnin test then there's something weird going on. camcontrol devlist is of course not a bad way to get access to device names. Hopefully @Spearfoot has some time to look at it.

While I'm running 11.1-U2 right now, it has been a while since I ran Spearfoot's disk burnin script for the last time, so I can't say for sure that it is supposed to run as-is on 11.1.
@VintageGold : I suspect you may have used a Windows text editor to edit disk-burnin.sh at some point... DOS or Windows text editors will often replace UNIX newlines with DOS/OS2/Windows newlines (Carriage Return/Line Feed pairs), which makes UNIX shell scripts fail with errors much like you're seeing.

Try downloading a fresh, unmodified copy of the script and then edit it in a shell session on your FreeNAS system, using either vi or nano -- both text editors are available on FreeNAS. If you must use a Windows text editor, use one that lets you select how newlines are encoded and use the UNIX setting when saving the file.

Regarding the devices on your particular server... you can use the smartctl --scan command to scan it and show you a list. Here's the result from one of my servers:
Code:
root@bandit:~ # smartctl --scan
/dev/cd0 -d atacam # /dev/cd0, ATA device
/dev/da0 -d scsi # /dev/da0, SCSI device
/dev/da1 -d scsi # /dev/da1, SCSI device
/dev/da2 -d scsi # /dev/da2, SCSI device
/dev/da3 -d scsi # /dev/da3, SCSI device
/dev/da4 -d scsi # /dev/da4, SCSI device
/dev/da5 -d scsi # /dev/da5, SCSI device
/dev/da6 -d scsi # /dev/da6, SCSI device
/dev/da7 -d scsi # /dev/da7, SCSI device
/dev/da8 -d scsi # /dev/da8, SCSI device
/dev/da9 -d scsi # /dev/da9, SCSI device
/dev/da10 -d scsi # /dev/da10, SCSI device
/dev/da11 -d scsi # /dev/da11, SCSI device
/dev/da12 -d scsi # /dev/da12, SCSI device
/dev/da13 -d scsi # /dev/da13, SCSI device
/dev/da14 -d scsi # /dev/da14, SCSI device
/dev/da15 -d scsi # /dev/da15, SCSI device
Note that all of my HDDs are attached to an HBA, so they have dax filenames. If you have HDDs attached to SATA ports they'll typically show up with adax filenames.

Good luck!
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Using FreeNAS 11.1

I downloaded and tried the disk-burnin.sh script and am running into an error. When I run the script, I get: Syntax error: newline unexpected (expecting word)

Here are my steps. I based my commands on the ones jgreco included here for his HDD burn-in script: https://forums.freenas.org/index.php?threads/building-burn-in-and-testing-your-freenas-system.17750/
Code:
cd /tmp
fetch https://github.com/Spearfoot/disk-burnin-and-testing/blob/master/disk-burnin.sh
chmod +x disk-burnin.sh
./disk-burnin.sh


Is this because I'm using 11.1 and the code needs an update?
You are fetching the wrong file. I used your command to fetch it and I ended up with the html document.
 
Last edited:

VintageGold

Cadet
Joined
Mar 12, 2018
Messages
6
You are fetching the wrong file. I used your command to fetch it and I ended up with an html document.
Hmm, I tried it again with my original code. Here's what I entered and the output. I modified the last line to include the target device, as instructed by MrToddsFriends.
Code:
root@freenas:~ # cd /tmp
root@freenas:/tmp # fetch https://github.com/Spearfoot/disk-burnin-and-testing/blob/master/disk-burnin.sh
fetch: https://github.com/Spearfoot/disk-burnin-and-testing/blob/master/disk-burnin.sh: size of remote file is not known
disk-burnin.sh										 132 kB  465 kBps 00m00s
root@freenas:/tmp # chmod +x disk-burnin.sh
root@freenas:/tmp # ./disk-burnin.sh ada0
./disk-burnin.sh: 7: Syntax error: newline unexpected (expecting word)
 
Joined
Jul 10, 2016
Messages
521
As @Chris Moore stated, you're fetching the wrong file; you need to get the raw file instead. Use this fetch statement:
Code:
fetch https://raw.githubusercontent.com/Spearfoot/disk-burnin-and-testing/master/disk-burnin.sh
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Don't forget that it is in dry run mode until you change the variable from 1 to 0.

Code:
# Set Dry_Run to a non-zero value to test out the script without actually
# running any tests: set it to zero when you are ready to burn-in disks.

Dry_Run=1
 
Last edited:

VintageGold

Cadet
Joined
Mar 12, 2018
Messages
6
@Jurgen Segaert, thank you! That's exactly what I needed. I looked back what @Chris Moore said and opened it with nano and saw it was the wrong file. Also, thank you Chris for mentioning the variable about the dry run so I didn't have to post back here that it didn't run right! @danb35, thanks for teaching me a new way to see the file w/out using nano.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
One last thing. Now that I have this working, is the disk-burnin.sh script considered a replacement for jgreco's solnet-array-test-v2.sh script provided as part of his Building, Burn-In, and Testing your FreeNAS system guide, or will they effectively serve the same purpose, which is to find issues with a HDD that would warrant a replacement?
They have different purposes.
solnet-array-test-v2.sh is a non destructive test, as I understand it, that is designed to determine the performance of all the disks in a group. The burn-in utility is designed to write to all locations of the drive in a data destructive test that is intended to stress the drive and uncover defects.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
yes. you can run solnet over a live zfs array.
 

droeders

Contributor
Joined
Mar 21, 2016
Messages
179
First off - thanks for creating and posting these scripts. I use the SMART report script daily on several servers.

I noticed a bug in the SMART test age checks within smart_report.sh, but it's under some weird circumstances. It's very repeatable, so I wanted to report it.

I have an Intel SSD which doesn't support SMART test logging. It returns the following for "smartctl -l selftest":


smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]


I have the testAgeWarn variable set to 7 days. When my drive age went over 1000 days (24k+ hours), the test age warning no longer adds the "?" symbol for the drive name. I've tested it some, and it's definitely on the 1000 day boundary. For some reason, you also need testAgeWarn set to a value greater than 1 in order to see the error.

I patched it on my systems with the following change, but I clearly don't fully understand why it's broken. Some sort of string vs integer comparison problem?


-- smart_report.sh 2018-04-08 12:48:32.447970000 -0500
+++ /tmp/smart_report.sh 2018-04-08 13:49:50.265942000 -0500
@@ -92,7 +92,7 @@
/High_Fly_Writes/{hiFlyWr=$10}
/Command_Timeout/{cmdTimeout=$10}
END {
- testAge=sprintf("%.0f", (onHours - lastTestHours) / 24);
+ testAge=(onHours - lastTestHours) / 24;
if (temp > tempCrit || reAlloc > sectorsCrit || pending > sectorsCrit || offlineUnc > sectorsCrit)
device=device " " critSymbol;
else if (temp > tempWarn || reAlloc > 0 || pending > 0 || offlineUnc > 0 || testAge > testAgeWarn)
@@ -105,7 +105,7 @@
}
if (hiFlyWr == "") hiFlyWr="N/A";
if (cmdTimeout == "") cmdTimeout="N/A";
- printf "|%-6s|%-18s| %s |%5s|%5s|%5s|%7s|%7s|%8s|%6s|%10s|%6s|%7s|%4s|\n",
+ printf "|%-6s|%-18s| %s |%5s|%5s|%5s|%7s|%7s|%8s|%6s|%10s|%6s|%7s|%4d|\n",
device, serial, temp, onHours, startStop, spinRetry, reAlloc, pending, offlineUnc,
seekErrors, totalSeeks, hiFlyWr, cmdTimeout, testAge;
}'



Let me know if you have any questions.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
First off - thanks for creating and posting these scripts. I use the SMART report script daily on several servers.

I noticed a bug in the SMART test age checks within smart_report.sh, but it's under some weird circumstances. It's very repeatable, so I wanted to report it.

I have an Intel SSD which doesn't support SMART test logging. It returns the following for "smartctl -l selftest":


smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]


I have the testAgeWarn variable set to 7 days. When my drive age went over 1000 days (24k+ hours), the test age warning no longer adds the "?" symbol for the drive name. I've tested it some, and it's definitely on the 1000 day boundary. For some reason, you also need testAgeWarn set to a value greater than 1 in order to see the error.

I patched it on my systems with the following change, but I clearly don't fully understand why it's broken. Some sort of string vs integer comparison problem?


-- smart_report.sh 2018-04-08 12:48:32.447970000 -0500
+++ /tmp/smart_report.sh 2018-04-08 13:49:50.265942000 -0500
@@ -92,7 +92,7 @@
/High_Fly_Writes/{hiFlyWr=$10}
/Command_Timeout/{cmdTimeout=$10}
END {
- testAge=sprintf("%.0f", (onHours - lastTestHours) / 24);
+ testAge=(onHours - lastTestHours) / 24;
if (temp > tempCrit || reAlloc > sectorsCrit || pending > sectorsCrit || offlineUnc > sectorsCrit)
device=device " " critSymbol;
else if (temp > tempWarn || reAlloc > 0 || pending > 0 || offlineUnc > 0 || testAge > testAgeWarn)
@@ -105,7 +105,7 @@
}
if (hiFlyWr == "") hiFlyWr="N/A";
if (cmdTimeout == "") cmdTimeout="N/A";
- printf "|%-6s|%-18s| %s |%5s|%5s|%5s|%7s|%7s|%8s|%6s|%10s|%6s|%7s|%4s|\n",
+ printf "|%-6s|%-18s| %s |%5s|%5s|%5s|%7s|%7s|%8s|%6s|%10s|%6s|%7s|%4d|\n",
device, serial, temp, onHours, startStop, spinRetry, reAlloc, pending, offlineUnc,
seekErrors, totalSeeks, hiFlyWr, cmdTimeout, testAge;
}'



Let me know if you have any questions.
Are you certain that your Intel SSD doesn't support self-test logging? I'll be surprised if it doesn't.

Have you ever run any self-tests on the SSD? If not, please run a short test -- smartctl -t short /dev/??? -- specifying your SSD's device ID where I've put question marks. Then post the output of smartctl -l selftest /dev/??? in code tags.
 

droeders

Contributor
Joined
Mar 21, 2016
Messages
179
Are you certain that your Intel SSD doesn't support self-test logging? I'll be surprised if it doesn't.

I was surprised too, but this model definitely does not support logging. I've run both short and long tests in the past on two of these drives. On both drives, they show the percentage remaining as it's being tested, but never a record of the test in the logs. I just ran a short test again and got the same results.

Here's a link to the SSD:

https://ark.intel.com/products/56576/Intel-SSD-510-Series-120GB-2_5in-SATA-6Gbs-34nm-MLC
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
I was surprised too, but this model definitely does not support logging. I've run both short and long tests in the past on two of these drives. On both drives, they show the percentage remaining as it's being tested, but never a record of the test in the logs. I just ran a short test again and got the same results.

Here's a link to the SSD:

https://ark.intel.com/products/56576/Intel-SSD-510-Series-120GB-2_5in-SATA-6Gbs-34nm-MLC
Astounding!
This means that the script simply won't be able to track the length of time since the last SMART test for this particular SSD, and thus won't be able to correctly display the 'warning' indicator character in the output.
 
Last edited:

devnullius

Patron
Joined
Dec 9, 2015
Messages
289
I tried to do:
git clone https://github.com/Spearfoot/FreeNAS-scripts.git, but I get the following error:
Code:
Failed to connect to github.com port 443: Operation timed out
.

When I google this, I see that it has to do with proxies - which I don't use? How to easily get those scripts to a folder on my NAS? :)
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
You can just download them to your client and upload them to the server in your preferred way.
 
Top