What happens if lose connection to expansion unit

[SwisherSweet]

Hi,

I have have a zfs pool with one vdev with 5 drives in RAID Z1. I want to expand the pool by adding a second vdev with 5 drives in RAID Z1, but I will be adding them via external expansion unit.

My concern is if there is any hardware issue (data or power cable, psu failure) where the entire expansion unit becomes inaccessible for some reason, I assume the pool will crash. However, if I correct the problem (such as PSU failure), would I be able to recover the pool and if so, how would I do that?

Note, I am using latest TrueNAS Core. I'm running a Mac Pro with 5 drives connecting to the SATA ports. I'll be using an external esata enclosure. This is a home backup server. I have 3 other copies of the data.

Thank you.

 

[sretalla]

an external esata enclosure

That's a VERY terrible idea. Those will need to have SATA port multipliers to go from 1 eSATA to multiple SATA ports for the disks.

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

This resource was originally created by user: jgreco on the TrueNAS Community Forums Archive. Please DM this account or comment in this thread to claim it. In the last year or two, we’ve had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use...

www.truenas.com

If you do that, it will not end well for your pool.

If you had a SAS/SATA HBA with an external connection and an external SAS enclosure, a disconnection would present little issue, provided it isn't too frequent. Simple re-connection of the disks would be enough to recover things.

 

[SwisherSweet]

If you do that, it will not end well for your pool.

Thanks for this warning. I read through the linked article. I understand the points the post is making. The port multiplier and external drive bay is by Sonnet Tech and is essentially designed for the Mac Pro. It's was sold as Pro level equipment. I've used it for several years with no issues and I used to run FreeNAS on this same hardware with no issues, but I had separate pools (one internal and one per drive unit). I'm using this as temporary storage and I don't care about performance with this.

For reference, here's the card and enclosures I'm using:

Tempo SATA E4P 4-Port SATA PCIe Computer Card - Sonnet

Fusion D500P 5-Drive SATA Storage System with Port Multiplier - Sonnet

Synology actually uses this same architecture for many of their NAS units. Many of them have two eSATA connectors and they sell an eSATA expansion unit:

DX517 | Synology Inc.

Effortless capacity expansion for Synology DiskStation solutions on-the-fly.

www.synology.com

They even have a higher end NAS that connects to this expansion unit:

Download Center - download | Synology Inc.

Centralize data storage and backup, streamline file collaboration, optimize video management, and secure network deployment to facilitate data management.

www.synology.com

They've sold this architecture from at least 2017 and still selling this today. I realize Synology DSM and TrueNAS are not the same, however they are similar enough that I would expect the use of a quality port multiplier and 5 bay eSATA expansion should function similarly and as reliably as the Synology solution.

If you had a SAS/SATA HBA with an external connection and an external SAS enclosure, a disconnection would present little issue, provided it isn't too frequent. Simple re-connection of the disks would be enough to recover things.

I appreciate this alternative suggestion. Do you happen to know of an affordable desktop SAS enclosure that will work with these SAS/SATA HBAs?

Appreciate your help.

 

[Arwen]

One reason why we discourage some hardware, is that ZFS during a disk replacement or scrub, can hit the drive(s) or enclosure really hard. If their is a firmware or hardware bug in the eSATA port multiplier, this heavy data transfer can trigger such bugs. Then you might loose data. Or the entire pool. Thus, we prefer known to work solutions. Plus, some lower end or cheaper external enclosure solutions might not have good cooling.

Other vendors don't do data scrubbing because the technology they use for their NASes does not support it. And during disk replacements, their method of replacement is to walk the disk. Meaning, start at the beginning of the disks. Then perform very regular reads on a disk, (or set of disks in RAID-5/6 case), and write to the replacement disk.


All that said, we can't stop you from using some hardware.


I did run across this external disk enclosures that should work better, (on HBA with external ports). I have no knowledge if it is good and no connection to the company;

RS6414S/6418S Overview | | HighPoint Technologies, Inc.

The HighPoint RocketStor 6414S/6418S external 4- and 8-bay 6Gb/s SAS/SATA storage tower enclosures are designed for use with RocketRAID hardware.

www.highpoint-tech.com

Now these are in multiples of 4, because that is the most common SAS HBA external port width.

Just don't use their hardware RAID cards. Use a LSI based one.

 

[SwisherSweet]

Thank you @Arwen and @sretalla for your comments and reply. FYI, I connected everything today to perform some testing on this setup and performance was awful and nearly unusable. I honestly don't remember it being this bad. Just using the Mac Pro with the internal drives, it works great, but as soon as you connect the eSATA drives, things gets very, very slow.

I think I'll sell of this stuff on Ebay and use the money to buy one of those pre-built IXsystems TrueNAS Mini systems.

Thanks again!