Boot drive degrades

Terry Pounds

Dabbler
Joined
Jan 21, 2017
Messages
21
Running FreeNAS 11 and added dual boot usb drive after the initial install and now every few days the extra boot drive degrades and I have to set up the usb drive up again. I have tried a different usb drive each time I had to re-set it up again. Anyone else heard of the happening? Suggestions of possible causes? The USB drives are all same size and brand 8GB each. Also this server is always running and is not shut down and rebooted for this to happen. Usually happens over night and I wake up with the email stating boot drive has errors and is degraded.

System Information
Hostname freenas-11.local Edit
Build FreeNAS-11.0-U2 (e417d8aa5)
Platform Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Memory 15271MB
System Time Sun Sep 03 11:43:14 CDT 2017
Uptime 11:43AM up 4 days, 2:30, 0 users
Load Average 0.19, 0.28, 0.26
 
Last edited by a moderator:

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
Suggest you check that the syslog and database are being stored on your pool and not on the boot drive. Go System, System Dataset, and see that the boxes are checked and the pool name is in the dropdown. This will minimize writes to the boot drive.

Short USB boot stick life is not uncommon - many have moved to SSD's to eliminate the hassle - see https://forums.freenas.org/index.ph...slower-gui-boot-time.51489/page-2#post-358319 for some background.
 

Terry Pounds

Dabbler
Joined
Jan 21, 2017
Messages
21
Suggest you check that the syslog and database are being stored on your pool and not on the boot drive. Go System, System Dataset, and see that the boxes are checked and the pool name is in the dropdown. This will minimize writes to the boot drive.

Short USB boot stick life is not uncommon - many have moved to SSD's to eliminate the hassle - see https://forums.freenas.org/index.ph...slower-gui-boot-time.51489/page-2#post-358319 for some background.

Thank you for your response. Just checked where syslog and database is being stored and they are not being store on boot drive, they are being stored on my main data set. So that all looks proper.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
I share(d) your pain - my secondary system consumed Sandisk Cruzer Fits (the most-recommended model here) at an alarming rate until I installed an old laptop HDD to replace them (subsequently itself replaced by a used SSD from Ebay).
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
Running FreeNAS 11 and added dual boot usb drive after the initial install and now every few days the extra boot drive degrades and I have to set up the usb drive up again. I have tried a different usb drive each time I had to re-set it up again. Anyone else heard of the happening? Suggestions of possible causes? The USB drives are all same size and brand 8GB each. Also this server is always running and is not shut down and rebooted for this to happen. Usually happens over night and I wake up with the email stating boot drive has errors and is degraded.

System Information
Hostname freenas-11.local Edit
Build FreeNAS-11.0-U2 (e417d8aa5)
Platform Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Memory 15271MB
System Time Sun Sep 03 11:43:14 CDT 2017
Uptime 11:43AM up 4 days, 2:30, 0 users
Load Average 0.19, 0.28, 0.26
How long has your FreeNAS been in operation?
What are the specific make/model of the USB flash drives?
Is it always the second boot device which fails or do you see a pattern?

So my concern is you say this has only happened since you upgraded to FreeNAS 11-U2 and only the "extra" (or second mirrored) drive is failing. I wouldn't expect you to have two failures in a row so quickly, assuming the replacement device was new.

USB 3.0 Flash drives can overheat and fail prematurely which is why I recommend using a USB 2.0 version of a USB Flash drive or buty a high quality USB 3.0 flash drive, or just use a cheap SSD.

What can you do? Well save the config file, rebuild the USB Flash boot drive as a single device, stay away from the mirrored thing, it's crap. Just maintain a copy of your config file and you will be good.

If you have frequent failures of the USB flash drive, you may want to submit a bug report. You shouldn't have failures every few days.
 
Last edited by a moderator:

Terry Pounds

Dabbler
Joined
Jan 21, 2017
Messages
21
How long has your FreeNAS been in operation?
What are the specific make/model of the USB flash drives?
Is it always the second boot device which fails or do you see a pattern?

So my consern is you say this has only happened since you upgraded to FreeNAS 11-U2 and only the "extra" (or second mirrored) drive is failing. I wouldn't expect you to have two failures in a row so quickley, assuming the replacement device was new.

USB 3.0 Flash drives can overheat and fail prematurely which is why I recommend using a USB 2.0 version of a USB Flash drive or buty a high quality USB 3.0 flash drive, or just use a cheap SSD.

What can you do? Well save the config file, rebuild the USB Flash boot drive as a single device, stay away from the mirrored thing, it's crap. Just maintain a copy of your config file and you will be good.

If you have frequent failures of the USB flash drive, you may want to submit a bug report. You shouldn't have failures every few days.

My FreeNAS server has been up for about 4 or 5 months. I did not use dual boot the first 3 or 4 months of setup and only recently started using it and added it through the boot tab under system. The flash drives are Emtec 8GB that were cheap I bought from 5 Below Store. I have been using these types of drives for the last few years with my old FreeNAS 9 Server with no issues. I did not use dual boot on my old FreeNAS server though.

The last two times this has happened has been with the same USB Port but each time I would swap with a different Emtec 8GB drive.

I have setup a new dual boot drive once again today. This time I formatted the drive with a windows machine first (disabling quick format option) to to rule out any corrupted formatting. I usually do the (quick format) and one thing I did notice is if I format the USB first with a windows machine. It will resilver the drive within a hour. The first time the boot drive went down I re-set it back up and did not format the drive and it took about 18 or 20 hours to resilver the 8 GB drive. I have seen other talking about long resilvering times so I figured I would mention pre-formatting is a way around the long resilvering time.

I will give an update if the boot drive fails again letting everyone know if the full format option corrected it. If it happens again I will be trying a different USB port and update if the corrects it or not as well.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
Maybe the problem is with the drives?

Maybe it's the port they're plugged in?

I use San disk Cruiser Fit 3.0.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
I use San disk Cruiser Fit 3.0.
Those are the ones that my Dell FS12-TY ate up...
So maybe it's the ports.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
Those are the ones that my Dell FS12-TY ate up...
So maybe it's the ports.
I just looked back through my notes on my Dell experience - it's clear from the lack of granularity of them on the boot device issue that I was not really focused on it (probably because I already had use of an SSD in mind as a solution). But, interestingly enough I can see that I only had a USB stick crap out when I had two mirrored installed. There were no failures when only one was installed. I don't have the respective durations of solo or dual operation of particular drives noted. I started out with 4 sticks and ran though three of them quite quickly, ordered 4 more (running with one until they arrived), then ran through them and operated with one for a day or two until I had a brainwave and installed a used SATA laptop drive.
Out of the 8 sticks, all but one took a Rufus repartition and reformat and pronounced themselves windowze-ready. I have those 7 - unused, as untrusted - in a baggie in my rogues' gallery.
Of course I have no idea if the "apparent failure only in mirrored install" is in any way significant to the issue but it seems similar to the OP's report here.
In context this thread makes interesting (re)reading ...
 

Terry Pounds

Dabbler
Joined
Jan 21, 2017
Messages
21
I just looked back through my notes on my Dell experience - it's clear from the lack of granularity of them on the boot device issue that I was not really focused on it (probably because I already had use of an SSD in mind as a solution). But, interestingly enough I can see that I only had a USB stick crap out when I had two mirrored installed. There were no failures when only one was installed. I don't have the respective durations of solo or dual operation of particular drives noted. I started out with 4 sticks and ran though three of them quite quickly, ordered 4 more (running with one until they arrived), then ran through them and operated with one for a day or two until I had a brainwave and installed a used SATA laptop drive.
Out of the 8 sticks, all but one took a Rufus repartition and reformat and pronounced themselves windowze-ready. I have those 7 - unused, as untrusted - in a baggie in my rogues' gallery.
Of course I have no idea if the "apparent failure only in mirrored install" is in any way significant to the issue but it seems similar to the OP's report here.
In context this thread makes interesting (re)reading ...

I just want to say that once the drive fails as a FreeNAS boot drive I use a windows utility called SD Formatter to reformat the drive and the drive works fine again on all of them. They also scan out fine with no errors. I really do not think it is a drive hardware issue. I will try a different brand if it does happen again in the the next couple of days just to rule that out. Thank you for you help.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
I think that you should try @joeschmuck 's suggestion of one USB drive only, no mirror. Based on my experience maybe the single will be less subject to failure.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
I just want to say that once the drive fails as a FreeNAS boot drive I use a windows utility called SD Formatter to reformat the drive and the drive works fine again on all of them. They also scan out fine with no errors. I really do not think it is a drive hardware issue. I will try a different brand if it does happen again in the the next couple of days just to rule that out. Thank you for you help.
If this is true then I would submit a bug report on the issue providing every little detail you can, including how long is had been since the drive was replaced, if any scrubs had been performed, you know, everything you can think of even if it's a remote thing. I can only speculate as the the cause but I will try...

Maybe the Flash drives you have say that they have committed to writing a block of data when they really didn't before something happens to cause an issue, and if they are very slow to write the data, well maybe??? It's just odd that the data becomes corrupt on the second device yet the device can be reformatted and used again.

I use SD Formatter as well, it's a very good tool.

If you must use USB Flash drives I would purchase a SanDisk Cruiser FIT 2.0 (not the FIT 3.0) as these have been reported to work for a very long time without premature write failure. Some people have had good luck with the FIT 3.0 and many have not, the cause is thought to be heat generated and not dissipated. USB Flash drives are not all designed to run 24/7.

Good luck and if you can swap over to a single SSD as a boot device, I'd recommend it.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
Some people have had good luck with the FIT 3.0 and many have not, the cause is thought to be heat generated and not dissipated.

@joeschmuck - here's (another unqualified) datapoint: My first set of three mirrored Fit 3.0 USB failures were when the sticks were mounted in two rear server case ports immediately adjacent to the fan discharges from the two redundant power supplies. To reduce potential of a thermal problem I put the next set of two stick pairs in a powered USB hub in a cool location - without any outcome change as far as stick corruption was concerned.
 

Terry Pounds

Dabbler
Joined
Jan 21, 2017
Messages
21
Hi all! I just wanted to update that my system has been up for 5 days now without mirrored boot drive failure. I believe what fixed it was full format of USB drive with quick format unchecked using Windows 10. One thing I also did after boot mirror was up and running was also a boot scrub under the boot section tab.

Since then I have also updated from FreeNAS 11-U2 to FreeNAS 11-U3.

Finally, I also want to point out that the USB drive I am using was one of the drives that failed as boot drive about a week ago. I simply re-formatted it with a utility called SDFormatter and then as I said format with quick format unchecked using Windows 10 to format once again and then I added it as mirrored boot drive. Hope this helps someone having the same issue.

Thank you to everyone that helped with input.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
I hope it hold up.
 

LMB

Dabbler
Joined
May 16, 2016
Messages
16
Running FreeNAS 11 and added dual boot usb drive after the initial install and now every few days the extra boot drive degrades and I have to set up the usb drive up again. I have tried a different usb drive each time I had to re-set it up again. Anyone else heard of the happening? Suggestions of possible causes? The USB drives are all same size and brand 8GB each. Also this server is always running and is not shut down and rebooted for this to happen. Usually happens over night and I wake up with the email stating boot drive has errors and is degraded.

System Information
Hostname freenas-11.local Edit
Build FreeNAS-11.0-U2 (e417d8aa5)
Platform Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Memory 15271MB
System Time Sun Sep 03 11:43:14 CDT 2017
Uptime 11:43AM up 4 days, 2:30, 0 users
Load Average 0.19, 0.28, 0.26

I got exactly same problem with my Dell T20 and 2x SanDisk Ultrafit. Brand new sticks installed in USB 3.0 ports. Everything installs fine but next time I fire up the NAS, I got error message about degraded pool. I then got a brand new replacement and same brand new USB stick is unavailable (pool degraded) again, next time I get my NAS rebooted. So it must be either hardware problem or FreeNAS. I am on FreenAS 11.3 U1.
 
Joined
Jan 4, 2014
Messages
1,644
It's worth trying a different brand or type of USB stick (in pairs of course).

I used Sandisk Cruzer Facets for mirrored boot drives successfully with earlier model HP microservers with no problems that I can recall. I'd get a degraded boot pool when I attempted to use those same sticks in a later generation HP microserver. I tried Verbatim Store and Go sticks on the newer server and have not had a problem since.

Similarly, I tried brand new Sandisk Cruzer Facets in a high-end Pioneer DJ mixer and experienced intermittent issues. The moment I changed to Sandisk Ultra sticks, all my problems disappeared. Go figure.
 
  • Like
Reactions: LMB

LMB

Dabbler
Joined
May 16, 2016
Messages
16
It's worth trying a different brand or type of USB stick (in pairs of course).

I used Sandisk Cruzer Facets for mirrored boot drives successfully with earlier model HP microservers with no problems that I can recall. I'd get a degraded boot pool when I attempted to use those same sticks in a later generation HP microserver. I tried Verbatim Store and Go sticks on the newer server and have not had a problem since.

Similarly, I tried brand new Sandisk Cruzer Facets in a high-end Pioneer DJ mixer and experienced intermittent issues. The moment I changed to Sandisk Ultra sticks, all my problems disappeared. Go figure.

I have been a loyal Sandisk user for last 15 years, when it comes to usb sticks/memory cards :). I have different models of USB 3.0 sticks from Sandisk and I will try swapping them.

But reading other similar threads, I am tempted to get those sticks in USB 2.0.

At the moment I have 4 FreeNAS machines, all have OS running on mirrored USB sticks, no problem until now...

For the time being, I formatted USB stick in question. If it fails again, I will try using different model/brands and different USB ports.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
But reading other similar threads, I am tempted to get those sticks in USB 2.0
Or just plug the USB3 sticks into USB2 ports...
 

LMB

Dabbler
Joined
May 16, 2016
Messages
16
A quick update.

It seems to be hardware issue. I tried:

1) formatting "corrupted" usb stick (ultra fit), re-silvering, and it did not work,
2) moving ultra-fit sticks to USB2.0, this did not work as well,
3) different sticks, i.e. Sandisk Ultra Flair (2x 16GB USB3.0), and this did not work as well.

Whereas it will reboot fine, every time after turning machine off for 10-15 minutes and booting it up I was getting this message:

CRITICAL
Boot pool status is DEGRADED: One or more devices could not be used because the label is missing or invalid. Sufficient replicas exist for the pool to continue functioning in a degraded state...


On the other hand, there is no issue with OS when there is only a single stick installed.

I checked my other machine (Dell Optiplex 3020) and it works fine with mirrored Sandisk ultra fit sticks.

So all I have to do now is to get a PCI controller for additional drives (all four populated by IronWolfs Pro) for my OS. How frustrating :/
 
Top