Failed HDDs - Need Hardware Advice

[MichaelGMorgan]

About 7-8 months ago I put together a FreeNAS box which is currently running 9.10 Stable.

It had 1 SSD for the OS and then 7x3TB Toshiba p300 HDDs. These were arranged in groups of two, 3 directly to the motherboard and 4 connected to a StarTech RAID card (https://www.amazon.co.uk/StarTech-Port-SATA-Controller-Adapter/dp/B0001Y7PU8)

I had it setup as two groups with the 3 motherboard connected disks together and then the other 4 RAID disks also together.

Everything was great for about 5 months and disaster struck. Almost out of nowhere 3 of the RAID card disks started failing. Long story short I lost all data. After investigating, all three failed drives showed high seek error counts. I've sent those disks off and should be getting replacements soon as I believe they were faulty.

I rebuild the system, now with 6 x 3TB Toshiba P300 disks in a ZFS-3 setup thinking that it would take 3 HDD failures to lose the data this time. It's using 3 original HDDs which were connected to motherboard, the non-failed drive from before and two new drives. I checked smartctl figures on all disks and all were looking fine. These are currently connected with 3 to the motherboard and the other three via my RAID card.

This was running for 2 days and I've hit problems again. The 3 drives attached to the motherboard are running fine, but two of the RAID card attached drives are now detached by FreeNAS and are inaccessible. I can't even run a smartctl on them. The other third drive which is also connected to the RAID card currently has a "Seek Error Rate" of 9 which is slowly increasing - in other words, that disk is also failing.

So in total, 5 failed HDDs, all of which were connected to the RAID card mentioned above.
This can't be coincidence - so I'm looking to remove that RAID card and replace it. I'll then replace the 3 HDDs one by one when my replacement drives arrive.

I have two initial questions...

- How the heck is that RAID card damaging disks? Is it actually damaging them or just incorrectly reporting SMART errors?
- I need a replacement card which supports at least 3 X 3TB HDDs, ideally 4. I've been looking at an "IBM LSI ServeRAID-M1015 6Gbps PCI-E controller" on Amazon for £50 GBP.

And a more general question - It's my understanding that a card like the IBM M1015 is SAS. Am I correct in saying that I'd simply then purchase a SAS to SATA splitter and connect up my drives?

Any help would be very much appreciated!

Thanks!

 

[Spearfoot]

About 7-8 months ago I put together a FreeNAS box which is currently running 9.10 Stable.

It had 1 SSD for the OS and then 7x3TB Toshiba p300 HDDs. These were arranged in groups of two, 3 directly to the motherboard and 4 connected to a StarTech RAID card (https://www.amazon.co.uk/StarTech-Port-SATA-Controller-Adapter/dp/B0001Y7PU8)

I had it setup as two groups with the 3 motherboard connected disks together and then the other 4 RAID disks also together.

Everything was great for about 5 months and disaster struck. Almost out of nowhere 3 of the RAID card disks started failing. Long story short I lost all data. After investigating, all three failed drives showed high seek error counts. I've sent those disks off and should be getting replacements soon as I believe they were faulty.

I rebuild the system, now with 6 x 3TB Toshiba P300 disks in a ZFS-3 setup thinking that it would take 3 HDD failures to lose the data this time. It's using 3 original HDDs which were connected to motherboard, the non-failed drive from before and two new drives. I checked smartctl figures on all disks and all were looking fine. These are currently connected with 3 to the motherboard and the other three via my RAID card.

This was running for 2 days and I've hit problems again. The 3 drives attached to the motherboard are running fine, but two of the RAID card attached drives are now detached by FreeNAS and are inaccessible. I can't even run a smartctl on them. The other third drive which is also connected to the RAID card currently has a "Seek Error Rate" of 9 which is slowly increasing - in other words, that disk is also failing.

So in total, 5 failed HDDs, all of which were connected to the RAID card mentioned above.
This can't be coincidence - so I'm looking to remove that RAID card and replace it. I'll then replace the 3 HDDs one by one when my replacement drives arrive.

I have two initial questions...

- How the heck is that RAID card damaging disks? Is it actually damaging them or just incorrectly reporting SMART errors?
- I need a replacement card which supports at least 3 X 3TB HDDs, ideally 4. I've been looking at an "IBM LSI ServeRAID-M1015 6Gbps PCI-E controller" on Amazon for £50 GBP.

And a more general question - It's my understanding that a card like the IBM M1015 is SAS. Am I correct in saying that I'd simply then purchase a SAS to SATA splitter and connect up my drives?

Any help would be very much appreciated!

Thanks!

Yes, the StarTech RAID card isn't a good choice for FreeNAS. For one thing, it only supports 1.5Gb/s transfer rates while typical modern NAS drives support 6Gb/s. Also, the Toshiba P300 drives are Desktop drives which aren't intended for use in a NAS. This doesn't necessarily mean they won't work... there are quite a few forum members who use WDC Green drives, which are also Desktop-class drives. But they aren't likely to have vibration-dampening, support TLER/ERC/SCT, or have other features desirable in NAS drives.

The IBM M1015 card you mention is a very good choice for FreeNAS. It's an OEM version of the LSI 9210/9211. Other OEM versions are the Dell H200/H310 and there are others. All of these cards can be used with SATA drives when connected with a 'forward breakout cable', which has an SFF-8087 connector that plugs into the HBA card and 4 SATA connectors that plug into the disks.

Did the RAID card 'break' the hard drives? I wouldn't think so, but I dunno... it's hard to say. The high seek error rates may simply be an artifact of the sub-optimal RAID card and the drives themselves may be fine. (See SMART Attribute: Seek Error Rate).

 

[danb35]

How many SATA ports do you have on your motherboard? Most motherboards that are suitable for FreeNAS in the first place have at least six, and if that's the case, you can move at least two more disks to your motherboard ports.

 

[MichaelGMorgan]

I don't have the exact model to hand but its a mATX board so it only has 4, one of which is for the OS SSD.
It's just bizarre how everything was fine and then suddenly everything is failing.

I don't have the budget to go out and replace all the drives with NAS ready drives, so I'll probably buy the HBA card and then replace the drives if/when they fail with better NAS-ready alternatives.

I've read a lot about flashing the IBM M1015 card - is this a requirement or is it just needed to get the very best performance? If I can avoid it I'd prefer to not have to flash it. Either way I'm going to order one now. It seems that they're all in China so it'll take around three weeks to reach me anyway.

 

[MichaelGMorgan]

Actually, before I buy - what about a PERC H200?
They're cheaper and I can have one delivered in a couple of days.

I've actually heard of these as I think we use them in our web servers in work.

 

[danb35]

I've read a lot about flashing the IBM M1015 card - is this a requirement

Yes. There are plenty of good guides, and it isn't hard, but there are a few steps.

 

[Spearfoot]

I don't have the exact model to hand but its a mATX board so it only has 4, one of which is for the OS SSD.
It's just bizarre how everything was fine and then suddenly everything is failing.

I don't have the budget to go out and replace all the drives with NAS ready drives, so I'll probably buy the HBA card and then replace the drives if/when they fail with better NAS-ready alternatives.

I've read a lot about flashing the IBM M1015 card - is this a requirement or is it just needed to get the very best performance? If I can avoid it I'd prefer to not have to flash it. Either way I'm going to order one now. It seems that they're all in China so it'll take around three weeks to reach me anyway.

Warning: quite a few counterfeit LSI boards come from China. You might consider buying from a reputable seller on eBay instead:

http://www.ebay.co.uk/sch/i.html?_f....A0.H0.Xibm+m101.TRS0&_nkw=ibm+m1015&_sacat=0

Flashing the HBA is pretty much a given, as the firmware needs to be in sync with the current version of FreeNAS. There are numerous instructions here on the forum for flashing these cards, including a new resource:

https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

Good luck!

 

[Spearfoot]

Yes, the Dell H200 cards work very well, too. I own 4 of them. Like the IBM M1015, they are essentially a re-branded LSI 9210.

 

[MichaelGMorgan]

Warning: quite a few counterfeit LSI boards come from China. You might consider buying from a reputable seller on eBay instead:

http://www.ebay.co.uk/sch/i.html?_f....A0.H0.Xibm+m101.TRS0&_nkw=ibm+m1015&_sacat=0

Flashing the HBA is pretty much a given, as the firmware needs to be in sync with the current version of FreeNAS. There are numerous instructions here on the forum for flashing these cards, including a new resource:

https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

Good luck!

I was planning on buying one of those ones from eBay anyway. If however you guys think a Dell H200 will do the job I'd prefer to get one of those.

 

[Stux]

M1115 works great too.

Btw, some drives report values for raw seek error rate even when there are no errros. For example seagte drives report raw positive values for most 'errors', but these are encodings, and don of indicate actual errors, rather they indicate how many seeks have been done!

 

[MichaelGMorgan]

The drives are definitely failing - the two in there right now are innaccessible. I can hear the clicking as the system boots up