Register for the iXsystems Community to get an ad-free experience
Resource icon

Hard Drive Burn-In Testing - Discussion Thread

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,085
I started the burnin on Sept 4 th at 10PM and it's still churning away.
That does sound like it is slightly long. I can't recall exactly how long the last burn-in I did took, but I think it was around six days. I have done as many as 12 drives at a time without impacting other functions on the NAS.
 

NasKar

Guru
Joined
Jan 8, 2016
Messages
733
That does sound like it is slightly long. I can't recall exactly how long the last burn-in I did took, but I think it was around six days. I have done as many as 12 drives at a time without impacting other functions on the NAS.
I can't tell if it's running the same tests over again. Here is the output
Code:
+ Run SMART extended test on drive /dev/da8: Sun Sep  8 01:25:34 EDT 2019
+-----------------------------------------------------------------------------
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Extended self-test routine immediately in off-line mode".
Drive command "Execute SMART Extended self-test routine immediately in off-line mode" successful.
Testing has begun.
Please wait 457 minutes for test to complete.
Test will complete after Sun Sep  8 09:02:34 2019

Use smartctl -X to abort test.
Extended test started, sleeping 27420 seconds until it finishes
SMART self-test complete
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%        83         -
# 2  Short offline       Completed without error       00%        76         -
# 3  Extended offline    Completed without error       00%        75         -
# 4  Short offline       Completed without error       00%        68         -
# 5  Extended offline    Completed without error       00%         7         -
# 6  Short offline       Completed without error       00%         0         -

Finished SMART extended test on drive /dev/da8: Sun Sep  8 09:04:20 EDT 2019
+-----------------------------------------------------------------------------
+ Run badblocks test on drive /dev/da8: Sun Sep  8 09:04:20 EDT 2019
+-----------------------------------------------------------------------------
Checking for bad blocks in read-write mode
From block 0 to 976754645
Testing with pattern 0xaa: set_o_direct: Inappropriate ioctl for device

Broadcast Message from root@freenasSuper.local
        (no tty) at 13:02 EDT...

UPS ups on battery


Broadcast Message from root@freenasSuper.local
        (no tty) at 13:02 EDT...

UPS ups on line power

done
Reading and comparing:  59.80% done, 11:24:21 elapsed. (0/0/0 errors)                      done
Testing with pattern 0x55: done
Reading and comparing: odondone
Reading and comparing: done
Testing with pattern 0x 79.60% done, 58:22:37 elapsed. (0/0/0 errors)ors)                      done
79.61% done, 58:22:39 elapsed. (0/0/0 errors)elapsed. (0/0/0 errors)
80.94% done, 58:29:51 elapsed. (0/0/0 errors)


Last time I did a 4 TB drive it took ~ 3 days and 5hrs
 
Last edited:

Meyers

Patron
Joined
Nov 16, 2016
Messages
210
I see that this drive had two short and two long tests run on it recently. Is it possible you accidentally ran the script more than once for each drive? Check how many badblocks processes are running per drive. I just burned in 8TB drives and it didn't take this long.
 

NasKar

Guru
Joined
Jan 8, 2016
Messages
733
I see that this drive had two short and two long tests run on it recently. Is it possible you accidentally ran the script more than once for each drive? Check how many badblocks processes are running per drive. I just burned in 8TB drives and it didn't take this long.
How do you check how many badblock processes are running? The disk-burning directory only has one log file and one bb file (bad blocks).
The burnin finally stop at 5am today. About 7 days. I'll reboot before running the burnin on the remaining drives.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
18,101
ps aux | grep badblocks would be one way.
 

Ruff.Hi

Patron
Joined
Apr 21, 2015
Messages
259
I am looking at this bash script to test some new HDDs coming my way. It runs and creates a log file that I can see when logged into the command shell via putty (logged in as root) ... but I can not see the log files in file explorer (windows). Googling suggests that I need to run file explore as admin ... that didn't help.

I have run this script on a different NAS with none of these problems? Any suggestions?
 
Last edited:

NasKar

Guru
Joined
Jan 8, 2016
Messages
733
I am looking at this bash script to test some new HDDs coming my way. It runs and creates a log file that I can see when logged into the command shell via putty (logged in as root) ... but I can not see the log files in file explorer (windows). Googling suggests that I need to run file explore is admin ... that didn't help.

I have run this script on a different NAS with none of these problems? Any suggestions?
It worked great on all the other drives I ran it on. Don't know why it took so long on the 1st one. Make sure you run it in a tmux session.
 

Ruff.Hi

Patron
Joined
Apr 21, 2015
Messages
259
I am looking at this bash script to test some new HDDs coming my way. It runs and creates a log file that I can see when logged into the command shell via putty (logged in as root) ... but I can not see the log files in file explorer (windows). Googling suggests that I need to run file explore as admin ... that didn't help.

I have run this script on a different NAS with none of these problems? Any suggestions?
I added an 's' to the end of my share ... so that I knew it was a share. That made the windows path different from the NAS path. I just deleted the share (the one with the 's') and recreated it. Bingo ... log files are visible in Windows Explorer.
 

Ruff.Hi

Patron
Joined
Apr 21, 2015
Messages
259
The file in ssh but not in explorer is also a ownership issue. I chown'd them back to me and now I can see them in both situations.

I got 3 x 4TB WD Reds on Thursday. They are working their way through the burn-in script (short test, long test, bad blocks, short test, long test). I have noticed that one of the drives is taking much longer than expected on bad blocks.

Drive 1 ... currently at 20% done on a read / compare with pattern 0x55 (the 2nd pattern) after 24:34 hrs
Drive 2 ... currently at 30% done on a read / compare with pattern 0x55 (the 2nd pattern) after 24:43 hrs
Drive 3 ... currently at 2% done on a read / compare with pattern 0xaa (the 1st pattern) after 20:28 hrs

Drives 1 and 2 are over 3/8 way through and on their way to finish at around the 65hr mark (about normal). Drive 3 is only 1/8 of the way through and is looking to finish at about the 160hr mark.

Edit: I just watched all drives for a minute ... Drive 1 moved on 0.26%, Drive 2 moved 0.26% and Drive 3 moved 0.10%.

How do I test through put?
 
Last edited:

Ruff.Hi

Patron
Joined
Apr 21, 2015
Messages
259
Turns out I had a bad / fault data cable or seat. I am rerunning the burn-in again. Back in about 80 hrs :(
 

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
650
FYI ... Just finished burn-in of three of my new WD "White" 12TB

badblocks took 166 hours per disk (running parallel)
badblocks -sw -b 4096 -c 32768 /dev/daX
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
18,101
I'm commissioning a shiny new NAS for work and left the disks (10 TB WD Reds) being subjected to badblocks over the weekend, when nobody's around to complain about the noise. I guess I could've used a weekend twice as long as normal...
 

S1RC

Dabbler
Joined
Jul 28, 2016
Messages
28
I've gone through this process before I built my first array. I've got some new drives to burn in, can this process be run in a jail?
 

hezohab

Cadet
Joined
Aug 3, 2019
Messages
9
FYI ... Just finished burn-in of three of my new WD "White" 12TB

badblocks took 166 hours per disk (running parallel)
badblocks -sw -b 4096 -c 32768 /dev/daX

Wow, 7 days to complete 1 drive?

I am using the same model of drives.
Would you mind describing which tests your and what settings you used for the test?
Is it smartctl short, long and badblocks?

I know very little, I am just getting started.

Thanks!
 

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
650
@hezohab Hi, yes. I ran SMART short test on all. Then LONG test on all. After that i ran badblock with -c 65535 on all 6 disks in parallel (via TMUX) but system got stuck and i got UDMA CRC Error on one of the disks. Not sure if i've just hammered the SAS or the stuck was a just consequence of that error. Anyway i've replaced the SATA cable and started badblocks with -c 32768 and on three disks in parallel. Once finished i did the other three. After that i ran the long SMART again.

Code:
smartctl -t short /dev/daX
smartctl -t long /dev/daX
badblocks -sw -b 4096 -c 32768 /dev/daX   //!! Data Destructive test !! Run only on new/empty disks!
smartctl -t long /dev/daX
 

yourmate

Contributor
Joined
Apr 4, 2016
Messages
105
Having lost all my data it's time to update the 4TB WD Reds with 8TB ones. [Just a note on 8TB WD Reds - while the "normal" ones are running at 49-55 C (in contrast 4TB ~ 32-34 C) the helium ones are running at the same temperature as 4TBs at 32-36 C]

Is this still the "FreeNAS recommended" procedure to burn in disks or is there an updated procedure?
Many thanks
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Having lost all my data it's time to update the 4TB WD Reds with 8TB ones. [Just a note on 8TB WD Reds - while the "normal" ones are running at 49-55 C (in contrast 4TB ~ 32-34 C) the helium ones are running at the same temperature as 4TBs at 32-36 C]

Is this still the "FreeNAS recommended" procedure to burn in disks or is there an updated procedure?
Many thanks
Yikes! Sorry to hear about you losing all of your data!

Don't know if there's an official recommended procedure, but I always burn-in new drives, and I've created a resource to help people do that:

 

yourmate

Contributor
Joined
Apr 4, 2016
Messages
105
Yikes! Sorry to hear about you losing all of your data!

Don't know if there's an official recommended procedure, but I always burn-in new drives, and I've created a resource to help people do that:

Thanks a lot!
Unfortunately, I already started hammering them by the time I read your message but I bookmarked your script. There is always a next time :)
 

Mastakilla

Contributor
Joined
Jul 18, 2019
Messages
179
Trying these tests as well now...

A useful command in case you want to see the progress of the smartctl long test:
Code:
echo /dev/da[0-7] | tr " " "\n" | xargs -L 1 smartctl -a | grep "progress" -i -A 1

Replace 'da[0-7]' with whatever your disks are...

You can use a similar command to launch the smartctl tests too

Btw:
Is it normal that FreeNAS 11.3U2 Reporting does not see any Disk I/O during the long SMART test?

Edit:
When running badblocks, it does report I/O.
 
Last edited:

Teeps

Dabbler
Joined
Sep 13, 2015
Messages
36
FYI ... Just finished burn-in of three of my new WD "White" 12TB

badblocks took 166 hours per disk (running parallel)
badblocks -sw -b 4096 -c 32768 /dev/daX

Thanks for the tip on the boosted -c option, kicking off a 10TB Toshiba MG06ACA10TE presently.

I'm curious if anyone knows what this message below means:

Checking for bad blocks in read-write mode
From block 0 to 2441609215
Testing with pattern 0xaa: set_o_direct: Inappropriate ioctl for device
0.23% done, 1:32 elapsed. (0/0/0 errors)


It only shows up in a few posts elsewhere and it's unclear if there's an issue
 
Top