USB Mirror, drive failed, resilvered, cant boot

[Shifty One]

Title says most of it. I have a pair of 8GB USB sticks in a Mirror for booting. All was well until one of them started going bad and was degraded. No problem, I have more! Pop in a new drive, let it resilver. All appears well. Until I had to reboot... Upon reboot, the system with hang at "Divide by Zero" red screen with a few error codes. I tried booting a few different ways and narrowed it down to my new USB stick. When I removed it from the system and used just one drive, it booted fine (but complaining about the boot volume of course).

Well, I deleted all partitions and cleaned the stick, let it resilver again. Still happens. Something about my Proliant ML10v2 server does not seem to like those sticks. However, I have used the same exact brand to load Linux on my desktop machine (bought a 4 pack).

Is there something that the resilver process is not doing? The only think that I can see from "Disks" in Ubuntu is that it is not flagged as a system partition / Bootable. Would that be the cause and/or fix?

Code:

Disk /dev/sdg: 7.5 GiB, 8002732032 bytes, 15630336 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 9FF67DD5-51A4-11E9-871F-3CA82A4B5028

Device     Start      End  Sectors  Size Type
/dev/sdg1     40     1063     1024  512K FreeBSD boot
/dev/sdg2   1064 15630295 15629232  7.5G FreeBSD ZFS

Code:

                 *-usb:3
                      description: Mass storage device
                      product: Cruzer Fit
                      vendor: SanDisk
                      physical id: 8
                      bus info: usb@1:8
                      logical name: scsi10
                      version: 1.00
                      serial: 4C530001091104113044
                      capabilities: usb-2.00 scsi emulated scsi-host
                      configuration: driver=usb-storage maxpower=200mA speed=480Mbit/s
                    *-disk
                         description: SCSI Disk
                         product: Cruzer Fit
                         vendor: SanDisk
                         physical id: 0.0.0
                         bus info: scsi@10:0.0.0
                         logical name: /dev/sdg
                         version: 1.00
                         serial: 4C530001091104113044
                         size: 7632MiB (8002MB)
                         capabilities: removable
                         configuration: ansiversion=6 logicalsectorsize=512 sectorsize=512
                       *-medium
                            physical id: 0
                            logical name: /dev/sdg
                            size: 7632MiB (8002MB)
                            capabilities: gpt-1.00 partitioned partitioned:gpt
                            configuration: guid=9ff67dd5-51a4-11e9-871f-3ca82a4b5028
                          *-volume:0
                               description: boot partition
                               vendor: FreeBSD
                               physical id: 1
                               logical name: /dev/sdg1
                               serial: 9ff866f2-51a4-11e9-871f-3ca82a4b5028
                               capacity: 511KiB
                               capabilities: boot
                          *-volume:1
                               description: ZFS partition
                               vendor: FreeBSD
                               physical id: 2
                               logical name: /dev/sdg2
                               serial: a0344772-51a4-11e9-871f-3ca82a4b5028
                               capacity: 7631MiB

Thanks!

 

[tfran1990]

I had the same problem a while back. Make sure you BACKUP your CONFIG while you still can. best thing to do it reinstall the same version of freenas you are currently running.

 

[Meyers]

Can you boot off the second USB stick? I'm wondering if you have to manually install the boot loader on that one.

USB stick issues have come up a lot in the past few months. I'm not sure how much help you will get with this. Is there any way you can move over to an SSD? You could also put in a PCI-E to m.2 adapter if you have one of the two x8 slots available. You can get dual m.2 slots in a single PCE-E card and mirror them. Definitely back up your config either way.

 

[rvassar]

I'll second the "backup your config"... and add "Immediately".

In general USB is a poor disk interface. I got burned on USB boot devices. I went thru 3 USB "thumb drive" sticks in less than 6 weeks. It appears to come down to the individual system's USB controller, as my previous test build had no issues with them. I ended up replacing a failed stick with a cheap 60Gb SSD in a USB enclosure, resilvered that in, and eventually moved to a new build booting off two 60Gb SSD's attached to the motherboard SATA controller.

There seems to be a plentiful supply of tiny 60Gb SATA SSD's. They go for anywhere from $18 to $35 here in the US. Not much more expensive than a good flash thumb drive, but you have to have SATA ports available.

 

[Shifty One]

Nope, the suspect stick will not boot at all. Same red Divide by zero error if that stick is plugged in to any USB port (have some USB2 and USB3, tried all). I also suspected the boot loader, but you can see the boot partition at least. As for the reliability, I have seen mixed results on reddit and elsewhere. In my experience (maybe I got lucky), I was running on the same 2 sticks for 2 years before one went out. I have seen people say that they loose sticks every couple months.

I do have a spare SSD that I was considering for other uses, but would be willing to use it for booting. Looks like the internal SAS/SATA controller can handle that no problem. However, I am finding very mixed to negative results on PCIe SSD booting on this server.

And yes, I have quite a few copies of my config. I am not worried there.

 

[Meyers]

I'd definitely move to SSD then. That's been the general consensus lately. You can use SATA if you have the slots of course.

Several of us started having problems with USB starting with 11.2. One system I have is completely unusable on 11.2. I have no control of the hardware so I had to install FreeBSD on the data drives just to get it going.

 

[Shifty One]

Well, I picked up some 120gb PNY SSDs and will just mirror them instead. Sad that we couldn't track this down, but I this is for the best at least.

 

[tfran1990]

I had a problem 1 time. i was able to boot from just the one. then when i went to replace the "failed" usb, i had more problems because it was a tad bit smaller then the one it was replacing. I went and got 3 exact same usb drives and i did the swap wrong. i used attach and not replace.( ithink)