LSI Logic SAS31601E and 8tb disks dont seem to play nice

[MR. T.]

Hi all,

i have an LSI Logic SAS31601E board i got off ebay. I chose that one after reading the forums and understanding that an HBA is beter than a RAID card.
I know some RAID cards can run as HBAs, but i didn't want to fiddle with things i'm not very comfortable with.

Rnd result is that i bought that card and was pretty happy with it as it would run all the disks i connected to it till now.
When i connect a 8tb disk it simply does not work. It's either recognised as the disk that was there before or is not recognised at all.

I google fu this half way to death and can't find anywhere saying that the HBA has some sort of limitation.

Usually the limitation is on 2tb disks but i have 3tb disks connected running happily.

does anybody know why this might happen, or a easy way to check any board limitations?
I would rather know now before attempting to find an firmware update and attempt to upgrade.

I have 16 ports on that board and 12 of the motherboard... and am running low.

The options i have are:
- a SAS expander (my favorite option) but the board needs to recognise whatever i throw at it
- a SATA port multiplier (my least favorite option).

Before i had the HBA i have bought a few of these:
https://www.aliexpress.com/item/Fre...Card-adapter-3Gbps-SATA-II-PM/2050941479.html

But most didn't even last a week so i ended using up the only pci express slot i have with the HBA. (i was going to use it to get an fiber NIC)

If i have to go down the route of the port multiplier ill get a few of these:
http://www.newegg.com/global/uk/Product/Product.aspx?Item=N82E16816124072

But i would rather get the right hardware for the right job.

Any thoughts? or alternatives i didn't thing about?

Thanks in advance

 

[Ericloewe]

i have 3tb disks connected running happily.

With all 3TB accessible? That sounds unlikely.

If i have to go down the route of the port multiplier ill get a few of these:
http://www.newegg.com/global/uk/Product/Product.aspx?Item=N82E16816124072

No, no, no, no, no, no! Don't even think of using SATA port multipliers! Those things are as unreliable and unsupported as it gets.

Before i had the HBA i have bought a few of these:
https://www.aliexpress.com/item/Fre...Card-adapter-3Gbps-SATA-II-PM/2050941479.html

Oh god, it somehow gets worse.

 

[depasseg]

Wasn't there a compatibility issue raised (and resolved) recently between SAS/SATA-1 ports and SAS/SATA-2 drives?

edit:
This is the thread I was thinking of-
https://forums.freenas.org/index.ph...lash-disk-detection-issues.47036/#post-322893

 

[MR. T.]

No, no, no, no, no, no! Don't even think of using SATA port multipliers! Those things are as unreliable and unsupported as it gets.

I actually didn't know that. It was the way i thought of getting more ports on the extremely hardware i have.
1 pci-e slot is quite limited. I have to do the most i can with what i have :)

Oh god, it somehow gets worse.

Yea... those are pretty bad... £100 into the bin.

With all 3TB accessible? That sounds unlikely.

I didn't think of it... the 3tb disks are on a pool with a couple 2tb disks so i never tried to tap into the 3rd tb.
If that is the case when i replace the 2tb disks there is a high chance that will corrupt the pool.
This is dangerous. Is there a way to check for this?

Wasn't there a compatibility issue raised (and resolved) recently between SAS/SATA-1 ports and SAS/SATA-2 drives?

edit:
This is the thread I was thinking of-
https://forums.freenas.org/index.ph...lash-disk-detection-issues.47036/#post-322893

Thanks!
I'll read through this and see what i can do.
If the issue is a bunch of slower disks getting in the way... i'm happy to boot them out or relegate them to a single port with a multiplier or something crappy like that.
I like the extra space but at this point i am about to grow my pool to several tens of tb and that for me should last for a while

 

[Ericloewe]

If that is the case when i replace the 2tb disks there is a high chance that will corrupt the pool.

No, ZFS will simply be confused and not use the additional capacity.

Realistically, you need to take the following steps:

1. Get an SAS2 or SAS3 expander
2. Replace the SAS1 HBA with an SAS2 HBA
3. When you're ready to expand, replace the 3TB drives one at a time - use a spare to bootstrap this process. Naturally, you can use the 3TB drives again after wiping them.

 

[MR. T.]

Replace the SAS1 HBA with an SAS2 HBA

Since i started posting in this forum i feel dumber and dumber...
People are pointing out things that are quite obvious but i had no idea.

I bought that SAS HBA because i wanted to connect the maximum number of disks possible (the 24 port one as too expensive), but i had no idea i just buy an HBAof 2 ports and a bunch of expanders...

I get what i need to do.. i just need to know what is the difference between SAS1, SAS2 and SAS3 and how to recognize between them.

i am assuming this hba i have is SAS1 and just not good?

 

[Ericloewe]

i am assuming this hba i have is SAS1 and just not good?

It's old, with all that implies.

i just need to know what is the difference between SAS1, SAS2 and SAS3 and how to recognize between them.

Check out the hardware recommendations guide. Link is in my sig.

 

[depasseg]

the 24 port one as too expensive

If you need that many disks, the general suggestion is to use a 2 port m1015 connected to a backplane.

 

[Ericloewe]

If you need that many disks, the general suggestion is to use a 2 port m1015 connected to a backplane.

Or a passive backplane plus an expander.

 

[MR. T.]

If you need that many disks, the general suggestion is to use a 2 port m1015 connected to a backplane

Or a passive backplane plus an expander.

If it's worth using... its worth (ab)using! :)

My motherboard only has a pci-e x8 slot.
Would a 2 port HBA capable of using that to the maximum?

I bought the HBA i have because it was PCI-e x8, so i assumed it would be close of using as much as i could from the limited resources i have.

I want to be able to use what i have to the maximum (budget allowing).


I am reading the hardware recommendations and a few other interesting threads and am thinking of getting a couple SAS expanders (hopefully will be compatible with SAS1,2 and 3) and use that when i run out of ports.
At a later time i buy a better HBA.
Since i have limited slots on the motherboard i need to make the maximum with it. If it can run 4 ports i'll get the 4 ports... if it only runs 3... i'll get that.


Do the SAS expanders work fine with SATA disks? or connecting a bunch of SAS expanders only works correctly with SAS disks?

 

[Arwen]

...
Do the SAS expanders work fine with SATA disks? or connecting a bunch of SAS expanders only works correctly with SAS disks?

SAS expanders work with SATA II, (3Gbps), disks, just fine. Try to get a SAS II, (or III),
expander. Otherwise you may end up with another 2TB limit in your path.

By the way, I looked up that SAS card you listed, it has 2 very early generation SAS I chips.
Thus, the 2TB limit.

 

[MR. T.]

By the way, I looked up that SAS card you listed, it has 2 very early generation SAS I chips.
Thus, the 2TB limit.

FreeNAS is now showing both 3tb disks as 2.2.
With the 2tb limit this makes sense.

I dont know why it was showing 3tb before but must be because i shuffled the disks around a lot while i was having problems.

If you need that many disks, the general suggestion is to use a 2 port m1015 connected to a backplane.

I have looked into this and read a lot about PCI-e speeds performances and the like andit seems pci-e 2.0 x8 can run a 4 port SAS adapter before hitting on bandwidth issues.
These boards are pretty expensive right now, and to be honest i don't think i need them, so i have looked at the M1015 as advised and looks pretty good:
http://www.ebay.co.uk/itm/NEW-IBM-L...372904?hash=item1eb1678ee8:g:BWwAAOSw0gdXTAkU

Cheap and cheerful. My problem with it is that it seems to me to be a raid controller and not an HBA.
Raid doesn't play nice with FreeNAS, and i never tried to flash a raid controller... and if i could i would rather avoid it if i could. Or would this be able to run out of the box?

I have also saw this one from LSI:
http://www.ebay.co.uk/itm/LSI-SAS-9...751462?hash=item21172d15e6:g:GHYAAOSw8w1X3~Pf

Seems equivalent... and also seems like a raid controller... so probably also suffers from the same problem of the IBM one.

I'm not sure what to do. Everything that i attempt to do that should be a couple of hours work ends up being months of pain and suffering... and at this moment with all the stuff i have to do i rather not tackle flashing a raid controller due to the risk of being another huge time drain.

 

[toadman]

Those HBAs can run with IR firmware (raid controller) or with IT firmware (non-raid). You would want to make sure you flash the card to IT mode. There are many posts on how to do it. Generally pretty easy if you have a machine available to run the flash software. (Though I remember it was a pain for me as I tried to do it on a new system that was UEFI. Took me a bit to get it working initially.)

 

[MR. T.]

Ok... i have bit the bullet and bought an IBM m1015.

I really hope this doesn't turn out to be a nightmare to update the firmware.

Any thoughts on this expander?
http://www.ebay.co.uk/itm/HP-24-Por...838322?hash=item465650ac72:g:BysAAOSwLF1X82fw

It seems to be able to run at 6gbps so probably won't get any disk size or speed issues and is fairly cheap.

If that is no good... any suggestions of what i should get?

Spinning up a bunch of disks in a go is quite harsh on the PSU, the motherboard allows to stagger the disks but i think it's only for the ones connected directly to the motherboard.
I googled how to do it for the rest of the disks but couldn't find a "right way" of doing it.
Is there a industry standard way of staggering the disk spin up? or i would have to hack something up?

 

[Ericloewe]

The proper way of handling that is through proper PSU sizing, an art described in documents available on the forum.

Staggered spinup is far from ideal. That said, LSI SAS controllers allow this to be setup from their BIOS/UEFI extension ROMs.

 

[MR. T.]

The proper way of handling that is through proper PSU sizing, an art described in documents available on the forum.

I know. My PSU is 850W and i'm ok for a while.
Staggering spinup has the advantage of if i mess up and connect too many disks (momentarily) wont kill the PSU or something else.

Staggered spinup is far from ideal. That said, LSI SAS controllers allow this to be setup from their BIOS/UEFI extension ROMs.

Would that also include the IBM m1015? (since it will run as the LSI counterpart)

 

[Ericloewe]

Would that also include the IBM m1015? (since it will run as the LSI counterpart)

Yes, all LSI SAS2 (and presumably SAS3) controllers running LSI firmware.