Setting up 2 nodes with failover

[plastichairball]

Hi all, complete noob here.

I have set up True NAS Scale on two servers (let's call them TN1 and TN2) and created storage on TN1. I'd love to set up TN2 as a failover for TN1 and I cannot seem to figure out how to do it. I saw somewhere that Minio was an option, and when I went down that path I couldn't get it to work.

Is there a way get TN2 to see the same storage as TN1 (they're all plugged into the same JBOD) and then act as a failover if TN1 dies?

Thank you!

 

[HoneyBadger]

Hello!

Performing reliable failover while maintaining the data integrity promises of ZFS is challenging; due to the integration with the hardware and the customization required, the "shared-storage clustering" you're looking to implement is only supported with TrueNAS Enterprise on the M-series platform.

Other forms of clustering are available with SCALE including a mirrored cluster, but this is designed to have each system carry its own storage in a "Scale-Out" fashion.

TrueNAS SCALE Clustering Overview

Discover the power of TrueNAS SCALE Clustering, enabling scalable and reliable unified storage solutions for modern data environments.

www.truenas.com

 

[NWMrTIm]

I've been trying to figure out the same thing for quite some time, and I read through the article Honeybadger posted that shows that graphic but never really got solid answers.

So looking at this, would the "federated storage" be "mirrored storage at two locations" (two locations that each have a storage device) and the "mirrored storage" be like "mirrored storage at one location (two storage devices at one location)?

I think OP is going for the: "I have two NAS devices and one goes offline, loses network, reboots, or something, but everything keeps running because TN1 and TN2 are redundant" type of setup. I think I'll try to set that up.

But I want to ask HoneyBadger:

When you say: the "shared-storage clustering" you're looking to implement is only supported with TrueNAS Enterprise on the M-series platform.

does that mean I can't set that up on a couple of VMs?? or some old hardware - even if the old hardware is Xeon E3 12xx series with LSI 7211 controller in IT mode to act as an HBA? (BTW I set up TNAS Core on it and it works great) I guess my question is: should it work if the hardware supports it, or will the software prevent it because it will detect I'm not running specifically a TrueNAS Enterprise on the M-series platform?

Thanks

 

[HoneyBadger]

Hey @NWMrTIm

The page has actually since been updated - the "mirrored cluster" option has been dropped in favor of the erasure-coding option, but the same general principle remains. "Federated Storage" is "each server has its own storage, but can be accessed through a single name-space" - but if Server A goes offline, the data on it is temporarily inaccessible. Think of a single server named \\TRUENAS, and then you have FOLDER A and FOLDER B - two separate folders, two separate servers.

The mirrored or erasure-coded cluster distributes the data across all of the member servers in the cluster - you still have a single namespace of \\TRUENAS, but because the data is distributed, the loss of a single server means you can reconstruct the data similar to a RAIDZ configuration.

But I want to ask HoneyBadger:

When you say: the "shared-storage clustering" you're looking to implement is only supported with TrueNAS Enterprise on the M-series platform.

does that mean I can't set that up on a couple of VMs?? or some old hardware - even if the old hardware is Xeon E3 12xx series with LSI 7211 controller in IT mode to act as an HBA? (BTW I set up TNAS Core on it and it works great) I guess my question is: should it work if the hardware supports it, or will the software prevent it because it will detect I'm not running specifically a TrueNAS Enterprise on the M-series platform?

The shared-storage clustering identified as the "HA System" above does require a TrueNAS Enterprise license and iXsystems hardware that supports it (TrueNAS X/M/F-series) - based on your other thread of wanting hypervisors to have failover/uninterrupted access, that's also what you'd require.

You can certainly set up two TrueNAS systems and use some manner of replication/sync between them in order to keep them in near-real-time sync for files using something like Syncthing, but to have it fail over in an "uninterrupted" fashion when a single system is rebooted requires both systems to have access to the drives over a primary storage bus like a dual-ported SAS/NVMe backplane, as well as arbitration to decide which head unit should be mounting the pool at a given time. Having two controllers attempt to forcibly mount the same ZFS pool at once is a recipe for data corruption.

 

[NWMrTIm]

OK thank you. In my research I had a hunch about what you spelled out, but never found something that really spelled it out the way you just did. I really appreciate that! And I'm finding out more about ZFS too. I think you're right in that I could probably do a warm or hot standby setup in near real-time sync, but not "direct mirroring" per se.

Again, thanks!