USB Boot Devices

[rvassar]

In case anyone is wondering... This pool was created just before the system was left up.

Code:

  pool: freenas-boot
 state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
   Sufficient replicas exist for the pool to continue functioning in a
   degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
   repaired.
  scan: scrub repaired 0 in 0 days 00:00:44 with 0 errors on Sat Jun  2 03:45:44 2018
config:

   NAME		STATE	 READ WRITE CKSUM
   freenas-boot  DEGRADED	 0	 0	 0
	 mirror-0  DEGRADED	 0	 0	 0
	   da0p2   ONLINE	   0	 0	 0
	   da1p2   FAULTED	  0   227	 0  too many errors
 # uptime
 8:20PM  up 28 days,  4:16, 1 user, load averages: 0.29, 0.27, 0.25

Since I'm out of SATA ports on my motherboard, and don't have a proper HBA available at the moment, I am going to try the USB->SATA enclosure experiment, with a couple 60Gb Patriot SSD's.

https://www.newegg.com/Product/Prod...m_re=Patriot_Flare-_-0D9-003W-00031-_-Product

Those should at least get me wear levelling. I suspect I'll be working on Build 2.0 by fall.

 

[Chris Moore]

I am going to try the USB->SATA enclosure experiment

The USB to SATA enclosure is probably a bigger problem than the SSDs. I wouldn't expect that to work as a boot device.

 

[Chris Moore]

I can't remember what hardware you are working with. Do you have any available expansion slots?

 

[rvassar]

The USB to SATA enclosure is probably a bigger problem than the SSDs. I wouldn't expect that to work as a boot device.

I don't disagree. But you're 8 minutes 38 seconds too late. :)

Code:

  pool: freenas-boot
 state: ONLINE
  scan: resilvered 855M in 0 days 00:00:38 with 0 errors on Tue Jun  5 21:14:34 2018
config:

   NAME		STATE	 READ WRITE CKSUM
   freenas-boot  ONLINE	   0	 0	 0
	 mirror-0  ONLINE	   0	 0	 0
	   da0p2   ONLINE	   0	 0	 0
	   da1p2   ONLINE	   0	 0	 0

errors: No known data errors
# uptime
 9:21PM  up 28 days,  5:16, 1 user, load averages: 0.24, 0.22, 0.22

I put the USB enclosure of a different HCI than the thumb drive. I have a PCIe USB 3.0 HCI installed. So at least in theory it can't lock the motherboard's HCI bus that the remaining thumb drive is on. (I know... I know... I was laughing as I typed it... Honest!)

I'm not sure it will boot from a PCIe HCI at all. So the remaining good thumb drive has to stay for the moment. But for now the pool is redundant, and I'm "up".

I can't remember what hardware you are working with. Do you have any available expansion slots?

Optiplex 790 / i3-2120 / FreeNAS 11.1-U4

I have a PCIe slot I was saving... But it's kind of a moot point. I'm either swapping chassis to my ESXi box (Dell T3500) and converting the RAM to ECC, or scratch building something. But I need to consider some things, and procure some parts, etc... Running ESXi on the 790 isn't really ideal. It can't take 8Gb sticks of memory, and would be underpowered without a CPU swap.

 

[rvassar]

BTW - I suspect the root cause is access time stamps. Thumb drives in Raspberry Pi service are somewhat notorious for this. It's pretty much a given that you have to mount the flash devices "noatime", or they burn out in a few weeks.

 

[Chris Moore]

Dell T3500

I built my second FreeNAS system using a Dell Precision T3500. Decent system. I upgraded to a socket 1155 Xeon with 16GB of memory, but it could take 32GB.
If you get a SAS HBA, you can always take it forward to the next system. I used one of these and it worked very well.

Dell H310 6Gbps SAS HBA w/ LSI 9211-8i P20 IT Mode for ZFS FreeNAS
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601

With a pair of these cables.
Lot of 2 Mini SAS to 4-SATA SFF-8087 Multi-Lane Forward Breakout Internal Cable
https://www.ebay.com/itm/Lot-of-2-M...-Forward-Breakout-Internal-Cable/371681252206

 

[rvassar]

I built my second FreeNAS system using a Dell Precision T3500. Decent system. I upgraded to a socket 1155 Xeon with 16GB of memory, but it could take 32GB.

They are built like tanks. T3500's are LGA1366, with triple channel memory, but limited to 24Gb. The ECC RAM for it is "unbuffered" and kind of difficult to get a hold of. This is that era where the Intel chipsets didn't support the DDR3 8Gb sticks. There's a T5500 and a T7500 of similar vintage. I've scouted around for a T7500 with the CPU mezzanine board, as those can use registered server memory, but they're hard to find, and expensive to ship. Unfortunately the Bloomfield & Gulftown Xeon's didn't make the cut for the speculative execution fixes. There's no microcode updates from Intel, and no BIOS updates from Dell.

If you get a SAS HBA, you can always take it forward to the next system. I used one of these and it worked very well.

Dell H310 6Gbps SAS HBA w/ LSI 9211-8i P20 IT Mode for ZFS FreeNAS
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601

With a pair of these cables.
Lot of 2 Mini SAS to 4-SATA SFF-8087 Multi-Lane Forward Breakout Internal Cable
https://www.ebay.com/itm/Lot-of-2-M...-Forward-Breakout-Internal-Cable/371681252206

Yes, I look at those on eBay from time to time. Probably complete overkill for my needs. I doubt I'm going to grow past 6 drives for home use. But I haven't got all my iSCSI stuff configured/running yet, so it's probably too soon to think like that.

 

[Chris Moore]

The reason I went to the SAS HBA to begin with was the limited number of SATA connections on-board. Then, I decided to give up on boot from USB after having two USB sticks failed inside of 12 months.
I put all my storage drives on the SAS controller and booted the system from a mirrored pair of SATA drives.
I have had great success with the configuration.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[rvassar]

And now the other USB thumb drive went belly up. Total time on it, 35 days. The USB->SATA enclosure is holding up OK. But here's the interesting bit... Both failed thumb drives format and test OK on Linux. These have to be HCI related failures of some sort.

I'll have a real SAS HBA, hopefully by the weekend. Then the SATA SSD will go on the motherboard bus, and the storage drives will go on the HBA.