SOLVED fencing drives in a repurposed HA system

[jdratlif]

We have an old TrueNAS HA device that we're turning into two FreeNAS systems. We're going to use top shelf for one side, and the bottom for the other. The bottom shelf is attached via an external SAS cable. The top shelf is in the same case as the systems themselves. So both sides are connected to the SAS backplane of the top shelf.

I want to ensure one of the sides doesn't see the drives in the top shelf. I know the TrueNAS version had some method of fencing the drives, but I'm not sure how it worked.

Any suggestions on how I might tell one of the FreeNAS devices to never touch these drives, or even remove them from its sight? I'm not sure there's a hardware method of terminating the connection, but I know there is a software one, because TrueNAS did it. I don't need an HA failover; I just need one side to see the drives, and the other side not to.

Thanks.

 

[Ericloewe]

Just disconnect the cables from the backplane to the host you don't wish to provide the drives to.

 

[jdratlif]

There are no cables. The systems themselves connect to the backplane directly. This same backplane supplies power.

However, I did find out we can move one of the systems to the other chassis and this works.

 

[Ericloewe]

No SAS cables at all, even inside the nodes? Is that system really that integrated?

 

[jdratlif]

I'll try to remember to take some pictures and post them next time I'm at that office, but if there are cables, we couldn't find any. It's a Super Micro JBOD chassis; I don't know what model. I thought at first the two chassis were different, one being specific to the HA integration and the other being specific to the JBOD. Upon inspection, that doesn't seem to be the case. Both chassis seem to be identical. The backplane inside the chassis provides a connection to the SAS expander, the power, and the BMC/IPMI and heartbeat NIC. There are no cables that couple the controllers to the backplane inside the JBOD; there are pins that connect when the controller is fully inserted into the JBOD chassis.

The method for coupling the main shelf to the auxiliary shelf when it was an HA appliance was an external SAS cable that coupled each controller to the auxiliary JBOD chassis. But since I don't want the HA function anymore, my goal is to remove the main shelf drives from one of the controllers. Since we learned the chassis appear to be identical, we simply put the secondary controller in the main slot of the secondary chassis and moved the SAS controller from that slot into the auxiliary slot of the primary chassis. So now there is only one controller connected to the set of drives inside the chassis.

Since the only apparent downside of this was to lose the HA communication, which I didn't want anyways, this is a good solution to my original problem. We did learn that the SAS backplane cannot communicate directly with SATA drives, but using the old SAS cable to connect the controller to the the SAS expander in the secondary slot works just fine. I'm not sure what would happen if you mixed SAS and SATA drives, but since we just ordered some SAS drives to add to the second shelf, I will know soon enough.