BUILD SAS HBA that enumerates in the same order as FreeNAS

[Benoire]

Hello, first post here so be gentle

I'm looking at creating a new FreeNAS server using my Xeon L5630 based server which currently has LSI 9211-8i's to feed the 12 hotswap bays. As per other operating systems, the LSI cards do not enumerate according to the order the drives are in the bays; i.e. the first physical slot is occupied by a WD Red 3TB but that is enumerating as da1 or 3 and not da0 (this is a 500GB drive which is in fact drive 4 out of 5.

I appreciate that this makes little difference to the operation of FreeNAS but it makes it more cumbersome to establish which drive has failed in the actual bays.

What I am after is a list of 6G SAS HBAs that do enumerate properly; i.e. SAS port 0 = da0, SAS port 1 = da1 etc. I understand that the 12G Adaptec HBA 1000-16i appears to do this, but I am after a list of adapters that work fine in FreeNAS.

Any guidance and thoughts is appreciated.

Many thanks,

Chris

 

[Mirfster]

Not going to find it. Initially, I thought along the same lines. I even tried to match up to the actual SAS connectors on the backplane. Then I realized that is not the way to think of things. FreeNas is not tied into a particular controller, SAS connector, etc.; all it cares about is being able to access the drive(s) and basically the Serial Number.

That is what makes it so robust. Switch drives around or even controllers. Heck even switch motherboards. As long as FreeNas can get to the drives directly you are good. Identifiers like da0, da1, etc. can change and are not guaranteed to always be the same (even between reboots).

My advice is to not worry about that so much. Do however note the drive Serial Numbers and where they are physically. If FreeNas tells you a drive has failed, simply cross-reference the serial number to your records and replace the drive. There are other tricks you can do as well, just search the forums.

Here is an example of an older record I created. Even though I added "da0" type info, that was latter found out to not hold true; just the Serial Numbers are all that matters.

 

[nightshade00013]

Yep you're going to be in for a whole lot more headaches if you try and force this to work the way you think it should. Put the serial number on the outside of the drive bay or make a cheat sheet like Mirfster said.

And like he said switch drives or controllers and everything is good, you can swap them around to your hearts content. Try doing that with a hardware raid card, LOL. Just google "LinusTechTips all our data is gone
youtube" and watch the ensuing mayhem then be thankful FreeNAS doesn't work that way.

 

[Robert Trevellyan]

What I am after is a list of 6G SAS HBAs that do enumerate properly; i.e. SAS port 0 = da0, SAS port 1 = da1 etc.

It's not up to the HBA to determine this. All else being equal, the order in which the drives respond at boot time determines their device numbers. With staggered spin-up, you could be more confident that it would come out the same every time.

 

[Benoire]

thanks for the reply people! So why does SATA respond in a logical order, including addon cards yet a SAS HBA won't? I know I can use the method mirfster described above, I guess I've got OCD drive layout problems that I NEED to resolve haha

 

[Mirfster]

So why does SATA respond in a logical order, including addon cards yet a SAS HBA won't?

TBH, I am not sure if SATA would with FreeNas. I use HBAs, but really have not looked into it from strictly SATA connections...

 

[Benoire]

thanks for the reply people! So why does SATA respond in a logical order, including addon cards yet a SAS HBA won't? I know I can use the method mirfster described above, I guess I've got OCD drive layout problems that I NEED to resolve haha

So this was an assumption with respect to FreeNAS, but it is certainly the case in Linux, Windows and even Xpenology (Unofficial way of booting Synology Disk Station Manager software without a synology device). In all those other OS's LSI cards do the same as per FreeNAS, but SATA is ordered sequentially, based on the SATA port number.

 

[jgreco]

thanks for the reply people! So why does SATA respond in a logical order, including addon cards yet a SAS HBA won't? I know I can use the method mirfster described above, I guess I've got OCD drive layout problems that I NEED to resolve haha

Best advice, go get some meds for the OCD and move on with life ;-) ;-)

No but seriously, I think the answer is that the SATA devices may seem a little more orderly because there may be a slight delay in the probe sequence. SCSI drives definitely register in the order they present to the system, and I believe the same is true for SATA ports nowadays.

It *is* possible to wire down devices to a particular mapping, but there's no compelling reason to do this. In the old days, before we had GPT and nice labels, this was the default method for managing large fleets of drives, such as the full-rack 72 drive array I created back in the early 2000's.

 

[Benoire]

Thanks Jgreco, I've always wondered this since I got my numerous 9211-8i cards; if the SAS topology detects the drives on the right port in the 'CTRL-C' menus, why does this not get passed to the OS on boot? That's all I want to occur, but it appears that once booted the driver does something different. Is there anyway to set a spin up delay for each drive so that they spin up and detect in the right order?

 

[jgreco]

Thanks Jgreco, I've always wondered this since I got my numerous 9211-8i cards; if the SAS topology detects the drives on the right port in the 'CTRL-C' menus, why does this not get passed to the OS on boot? That's all I want to occur, but it appears that once booted the driver does something different. Is there anyway to set a spin up delay for each drive so that they spin up and detect in the right order?

It isn't a matter of spinup. It's a matter of the order in which they happen to "check in".

Try this from the CLI:

# grep " target " /var/run/dmesg.boot

I'm guessing you'll see some sort of mapping between the target numbers and the actual devices.