New build and warning about usb boot drive

[Mynorx]

Just wanted to share my new build and give a little warning about booting from usb drives.

Build
As per forum recommended build guide I tried to follow as close as possible. found Here https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/

1.super micro mobo
https://www.amazon.com/Supermicro-M...ro+ATX+Server+Motherboard+LGA+1151+Intel+C236
2. Intel Xeon E3-1220 V6 Kaby Lake 3.0 GHz
https://www.amazon.com/Intel-Xeon-E...words=Intel+Xeon+E3-1220+V6+Kaby+Lake+3.0+GHz
3. WD RED Hard drives x4
https://www.amazon.com/Red-3TB-Hard...r=1-3&keywords=WD+Red+3TB+NAS+Hard+Disk+Drive
4.APC 1500VA Compact UPS Battery Backup & Surge Protector
https://www.amazon.com/APC-Compact-...Watts+10+Outlets+Uninterruptible+Power+Supply
5. Seasonic FOCUS Plus Series SSR-650FX 650W 80+ Gold
https://www.amazon.com/Seasonic-SSR-650FX-ATX12V-Modular-Compact/dp/B073H33X7R/ref=sr_1_1?s=electronics&ie=UTF8&qid=1524509784&sr=1-1&keywords=Seasonic+FOCUS+Plus+Series+SSR-650FX+650W+80++Gold
6. SanDisk Ultra Fit 32GB USB 3.0 Flash Drive Mirror (CANNOT RECOMMEND)

The server was build for an electrical contractor. They have been in business for a couple of years but just got an office and are starting to network computers. So the system was running for about a week I went back to configure some settings and I see there is a problem with one of the Mirrored SanDisk usb boot drives

( Alert System - Critical The boot volume state is DEGRADED: One or more drives could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state). .

Not being and expert I researched and I figure out it was one of the mirrored usb drives I figured out how to properly remove the bad drive and decide to instead back up the configuration file and reinstall freenas on an SSD (good thing there was a best buy close by). Everything seems to be working fine now. There is a an error I noticed it after reinstalling freenas on the ssd it came up after I restored the configuration file (the configuration file was backed up and restored from the usb drive that was ok). error reads:

SYSTEM ALERT CRITICAL: The boot volume state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.

Im using snapshots and synchronizing to a backblaze bucket. I think USB 3.0 drives get pretty hot and that is why it failed. If you insist on using usb boot drives I think you should be mirroring them but in reality should be booting off an SSD.

When I built the companies nas I decided to update mine. I was still runing v9.10. Version 11.4u is beautiful. I love the new features specially the cloud sync.

 

[kdragon75]

USB drives are not ideal for server boot drives.
But there are a few things you can do to make it more reliable. As you pointed out, mirroring will save your butt. Also ensuring the system dataset is on a storage pool (not the boot pool) will minimise the writes to the USB drive and keep a lot of important system data safe in the case a drive(s) fail. Using quality drives and testing them just like any other data drive. Ensuring adequate ventilation. The tiny drives that also run hot are probably not the best as noted.

With that all said, there are a huge number of enterprise grade systems that boot from USB without issues for years. Granted more and more are moving to sataDOMs it's still the same flash memory.

Personally I have used Kingston Digital DataTravelers for years on several servers and never had an issue. There slow as heck but they always work.

 

[Mynorx]

USB drives are not ideal for server boot drives.
But there are a few things you can do to make it more reliable. As you pointed out, mirroring will save your butt. Also ensuring the system dataset is on a storage pool (not the boot pool) will minimise the writes to the USB drive and keep a lot of important system data safe in the case a drive(s) fail. Using quality drives and testing them just like any other data drive. Ensuring adequate ventilation. The tiny drives that also run hot are probably not the best as noted.

With that all said, there are a huge number of enterprise grade systems that boot from USB without issues for years. Granted more and more are moving to sataDOMs it's still the same flash memory.

Personally I have used Kingston Digital DataTravelers for years on several servers and never had an issue. There slow as heck but they always work.

on my first build I used a various models of Kingston drives. They've all worked fine. Thanks

 

[Inxsible]

and I see there is a problem with one of the Mirrored SanDisk usb boot drives.

You don't mention what that problem is. So one can only assume. What exactly happened? Did the drive just fail? Was the pool degraded? Had the drive melted where it was affixed?

I have been using a 8GB Sandisk Cruzer since day 1 without issues.

 

[Mynorx]

You don't mention what that problem is. So one can only assume. What exactly happened? Did the drive just fail? Was the pool degraded? Had the drive melted where it was affixed?

I have been using a 8GB Sandisk Cruzer since day 1 without issues.

good point I made revisions to my post. Original error was Alert System - Critical The boot volume state is DEGRADED: One or more drives could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state). .

I disconnected the usb drives first the internal , system booted fine, then removed the external reconnected internal usb and figured the internal usb drive was bad.

 

[Inxsible]

first the internal

internal meaning you had put in a usb drive on the type A port on the board? That is usually not a good place because of the heat that CPU gives out + the heat that the HDDs give out. You didn't mention the case you are using, but that could be a factor in terms of air flow over the board/cpu and the HDDs.

 

[Mynorx]

internal meaning you had put in a usb drive on the type A port on the board? That is usually not a good place because of the heat that CPU gives out + the heat that the HDDs give out. You didn't mention the case you are using, but that could be a factor in terms of air flow over the board/cpu and the HDDs.

The case being used is NZXT H440 it has very good air flow. 3x 120mm front fans for the hard drives, 1x 120mm top fan, and 1x 120mm Rear fan. Yes I did use the internal usb port.

 

[Inxsible]

I would use external USB ports..but since you've already moved to a SSD, I'd keep it as is.

 

[ech1965]

I also had more or less the same issue...

Bought two new " Clé USB 3.0 Sandisk Ultra Fit 16 Go Allant jusqu'à 130 Mo/s" on amazon

Could not create a mirror... they both report the same size but one was missing the last 10 mega

( I user f3write tools to test it and it failed writing the last couple of sectors...)

I rushed to the shop an bought a third one... and I could create my mirror

 

[Stux]

Only ever had issues with the Cruzer Fits when mounted internally...

Good luck with the new build

 

[Mirfster]

If you insist on using usb boot drives I think you should be mirroring them but in reality should be booting off an SSD.

This +10000; AFAIK FreeNAS doesn't simply load into memory only. USBs are really not designed for lots of Read/Write (compared to a SSD or even HDD); so IMHO avoid USBs.

Not knocking anyone's choice for boot device(s); just stating my personal preference. ;)

 

[Inxsible]

This +10000; AFAIK FreeNAS doesn't simply load into memory only. USBs are really not designed for lots of Read/Write (compared to a SSD or even HDD); so IMHO avoid USBs.

Not knocking anyone's choice for boot device(s); just stating my personal preference. ;)

It does depend on how mission critical your system is. Personally I can afford to take down my system down for a few days. I'd just have to find youtube alternatives for my 2 year old who watches Moana or Frozen almost every other day. But other than that, I don't need my system to be running 24/7. I keep it running 24/7 because I don't want to kill my disks sooner but that's a different topic.

btw, I have a boot usb -- no mirrors.

 

[Mirfster]

It does depend on how mission critical your system is

A 2 year old without their favorite shows can rival most CTO's ideas of "mission critical"... ;)

btw, I have a boot usb -- no mirrors.

As long as you are keeping config backups, you should be alright.

 

[Inxsible]

As long as you are keeping config backups, you should be all right.

Emailed to me every 15 days.

 

[Mynorx]

Came in to the office and I still get

Code:

 CRITICAL: April 26, 2018, 12:48 p.m. - The boot volume state is DEGRADED: One or more devices has experienced an error resulting in data corruption. Applications may be affected.

Boot scrub message

Boot Volume Condition: DEGRADED One or more devices has experienced an error resulting in data corruption. Applications may be affected.
Last Scrub Run on: Thu Apr 26 03:45:02 2018	

Size: 111.0 GiB
Used: 850.7 MiB (0%)


 Automatic scrub interval (in days) 

booting from San Disk SSD Plus 120gb I wonder if this message is from the old installation. I restored the configuration from the good mirror.