Drawbacks of using sas hba?

[Toydoll]

Hi

TL;DR:

• Can my sas controller card ruin my drives and thus destroying my pool?
• Is there any other drawbacks from using it instead of the sata connections on my motherboard?
• Is there any drawback on having my boot drive on the card instead of the motherboard?

I recently bought a Dell H310 controller card that I managed to flash to IT mode.
I will have a total of 14 drives, I initially planned to have eight of them connected to the motherboard (=maximum) and six connected to the HBA.

The case is however a small-ish tower and thus making the cable management a living hell. It's not a problem once it's set up but I fear that my fat fingers will destroy something when I will need to replace my drives. It would be a tad bit better if I didn't use all sata ports on the MB but I'm not sure if that's a wise move.

I haven't actually found someone who says it happens but some forum threads implies that it's not impossible for the HBA card to wreck the data. If that were to happen now my pool would still be intact but that would not be the case if I filled the HBA´s ports instead of the motherboards. Does anyone have a clear answer if the HBA actually can destroy my data?



(By the way, guess who managed to buy wrong RAM even though I clearly remember doing a double check. I got dimm instead of udimm. So now I have a completely new Freenas system that wont work until I get the right RAM which will take ageeeeeees. At least that's how it will feel. Not my proudest moment...)

 

[HoneyBadger]

Can my sas controller card ruin my drives and thus destroying my pool?
In the same way that your computer "could" be struck by lightning and explode, yes. I've not heard any reports of an actual HBA wrecking data; only RAID cards pretending to be HBAs or people doing silly things like individual RAID0 drives.

Is there any other drawbacks from using it instead of the sata connections on my motherboard?
A couple microseconds more latency since your data has to traverse the PCIe bus to hit a SAS controller, but the transactional nature of ZFS means you'll never see this unless you're planning to attach an SLOG device behind the SAS controller.

Is there any drawback on having my boot drive on the card instead of the motherboard?
Nope, not really.

 

[SweetAndLow]

Hi

TL;DR:

• Can my sas controller card ruin my drives and thus destroying my pool?
• Is there any other drawbacks from using it instead of the sata connections on my motherboard?
• Is there any drawback on having my boot drive on the card instead of the motherboard?

I recently bought a Dell H310 controller card that I managed to flash to IT mode.
I will have a total of 14 drives, I initially planned to have eight of them connected to the motherboard (=maximum) and six connected to the HBA.

The case is however a small-ish tower and thus making the cable management a living hell. It's not a problem once it's set up but I fear that my fat fingers will destroy something when I will need to replace my drives. It would be a tad bit better if I didn't use all sata ports on the MB but I'm not sure if that's a wise move.

I haven't actually found someone who says it happens but some forum threads implies that it's not impossible for the HBA card to wreck the data. If that were to happen now my pool would still be intact but that would not be the case if I filled the HBA´s ports instead of the motherboards. Does anyone have a clear answer if the HBA actually can destroy my data?



(By the way, guess who managed to buy wrong RAM even though I clearly remember doing a double check. I got dimm instead of udimm. So now I have a completely new Freenas system that won't work until I get the right RAM which will take ageeeeeees. At least that's how it will feel. Not my proudest moment...)

Using a hba is pretty much the same as using the sata ports. Not sure what you are reading but it's not correct.

 

[2nd-in-charge]

• Is there any drawback on having my boot drive on the card instead of the motherboard?

Did you keep the bios on your H310 when you flashed it to IT mode? If not you probably won't be able to boot from an HBA port. IMHO it's just much easier to boot from motherboard SATA ports.

six connected to the HBA.

If that were to happen now my pool would still be intact but that would not be the case if I filled the HBA´s ports instead of the motherboards.

You can lose 6 drives and have your pool still intact? Are you running 6 mirrored Vdevs?

 

[jgreco]

Lots of things can ruin your drives. I would put bad selection of power supply way out in front of SAS HBA though. SAS and SATA are designed to be interoperable. While SAS does use significantly higher voltages for signalling in SAS mode, the standards are designed to make sure that the right thing happens and that SATA-compatible signalling is used with SATA disks. I've literally never heard of a SAS controller blowing out a SATA drive.

The big red flag with a SAS HBA is if you don't keep it cool. This CAN cause problems. Make sure there is airflow around the HBA and that it isn't just sitting there cooking. If it gets way too hot, I've actually seen cases where data starts getting corrupted. These things are designed to sit in a server chassis with redundant fans blowing air. Arranging this in a typical PC chassis can be problematic.

FreeNAS will treat disks attached via SAS the same as via SATA. It doesn't care.

SAS HBA ports tend to be somewhat slower than mainboard ports. If you are hooking things like SSD's up where you want max performance, prefer the mainboard ports for those. This suggests that if you're port-limited, there might be some value in booting from the HBA. I wouldn't get too uptight about it though.

 

[Ericloewe]

Is there any drawback on having my boot drive on the card instead of the motherboard?
Nope, not really.

Actually, there is. You have more hoops to jump through and more things that can fail in one of the more failure-prone parts of any system.

It should still work, but it's definitely lacking advantages in the vast majority of scenarios.

 

[Toydoll]

Thanks for the repy, now I can sleep better ;)

Did you keep the bios on your H310 when you flashed it to IT mode? If not you probably won't be able to boot from an HBA port. IMHO it's just much easier to boot from motherboard SATA ports.

I must admit I'm one of those that sometimes follows guides without really knowing what I'm doing. This was one of these times. I guess I'll find out if it works or not.

You can lose 6 drives and have your pool still intact? Are you running 6 mirrored Vdevs?

If these six drives includes my two boot drives and if I connect the other drives in the right way then it will survive, barely. I have two vdevs in raidz2 (6x2TB and 6x6TB).

The big red flag with a SAS HBA is if you don't keep it cool. This CAN cause problems. Make sure there is airflow around the HBA and that it isn't just sitting there cooking. If it gets way too hot, I've actually seen cases where data starts getting corrupted. These things are designed to sit in a server chassis with redundant fans blowing air. Arranging this in a typical PC chassis can be problematic.

Yeah, I've read that especially the H310 can get a bit hot. I have four chassi fans, three of them or more or less blocked by hard drives but I hope that they will be enough. The CPU fan also blows a tiny bit of air on it.
If it's not enough then I'll borrow a fan from my main PC and ghetto mod it until I can find a better solution.

 

[2nd-in-charge]

I'm one of those that sometimes follows guides without really knowing what I'm doing.

I was in the same boat when I flashed an H200 a couple of years ago. The guide I followed didn't think it was important to preserve the bios. I still have no bios on that card...

I have two vdevs in raidz2 (6x2TB and 6x6TB).

Got it...

 

[Evertb1]

I must admit I'm one of those that sometimes follows guides without really knowing what I'm doing. This was one of these times. I guess I'll find out if it works or not.

No worries. If you need to boot from your HBA you can flash the bios right back. I have done that with an H310 I previously flashed to IT mode without keeping the bios. At some point I needed it enabled for booting to temporary "save" a motherboard with a faulty Sata controller. I followed this guide to flash the H310 to IT mode and followed up on the comment of "SleepingDeamon" on that guide to flash the LSI Bios. It worked great and I could boot from it without a problem.

 

[jgreco]

I was in the same boat when I flashed an H200 a couple of years ago. The guide I followed didn't think it was important to preserve the bios. I still have no bios on that card...

The BIOS is handy for debugging, because it's better not to let your NAS boot with a broken pool. It's also necessary if you use it as a boot device. Enthusiasts generally like their systems to "boot fast" so they leave it off, but the better solution is to include it anyways and just turn off the option ROM probe within the BIOS setup, which means you can flick it back on in an instant if there's an issue.

 

[Toydoll]

. I followed this guide to flash the H310 to IT mode and followed up on the comment of "SleepingDeamon" on that guide to flash the LSI Bios. It worked great and I could boot from it without a problem.

I used another guide but it looked very similar to this one. Some sentences were exactly the same if I remember correctly. Maybe someone did a copy/paste... but thanks, I’ll save this if I ever feel the need (or have the time) to fix it.


Now I have a problem with my board (supermicro x11ssm-f) not finding the boot ssd after installing freenas on it (even when it’s not connected to the hba)
But I’ve only tried for a short time yesterday and I’ve only used Asus gaming motherboards before so I’m not used to the interface. Hopefully easy to fix.
Something to look forward to a friday evening like this ;)

 

[2nd-in-charge]

The BIOS is handy for debugging, because it's better not to let your NAS boot with a broken pool.

I don't understand. We currently boot that server from mirrired SATA SSDs. What kind of debugging would a bios on the HBA allow?

Enthusiasts generally like their systems to "boot fast"

Our Dell T710 is a fine machine, but "booting fast" is not one of its virtues anyway :) The server is currently in operation, so I won't be touching it for now..

 

[Ericloewe]

The extension ROMs include an SAS tree viewer, which allows you to see which SAS devices the controllers actually see.

 

[jgreco]

I don't understand. We currently boot that server from mirrired SATA SSDs. What kind of debugging would a bios on the HBA allow?

You can do baseline debugging of SAS connectivity of all your drives, and, depending on the specific controller, certain types of maintenance too. Having to bring up FreeNAS with a broken pool is somewhat risky and it's nice to be able to validate that everything is attached properly and functioning without having to bring up FreeNAS.