Strange HDD behaviour

[sebyp]

Hello everyone,

I've just installed today my first FreeNAS storage. After a bit of playing around with the LSI 3108 controller (turns out the JBOD option was not so visible) I've finally managed to convince FreeNAS to see the existing HDD's.

Small issue: although all 6 HDD's in my system are 1TB (SAS @10k RPM) FreeNAS only reports 150GB.

Any ideas?

Thanks!

 

[sebyp]

LE: when trying to wipe one drive I get the following error:

Error: Failed to wipe mfisyspd0: dd: /dev/mfisyspd0: Operation not permitted

 

[Darren Myers]

Can we get some hardware specs? Motherboard, cpu, HBA(s),RAM, etc.

 

[sebyp]

Below all relevant (hopefully) specs

• Server: SuperServer 2028R-E1CR24N (Black)
• CPU: Intel E5-2609V3
• Motherboard: Super X10DRi-T4+
• RAM: 4 x 16GB
• RAID controller: LSI 3108 SAS3 AOC
• HDD: SEAGATE 2.5" 1200GB SAS 12Gb/s, 10K RPM, eMLC 32GB, 4KN (6 of them)

Hope this helps.

Thanks!

 

[Darren Myers]

for whatever reason the post didnt update, but i got an email...sooo here it is

Below all relevant (hopefully) specs

• Server: SuperServer 2028R-E1CR24N (Black)
• CPU: Intel E5-2609V3
• Motherboard: Super X10DRi-T4+
• RAM: 4 x 16GB
• RAID controller: LSI 3108 SAS3 AOC
• HDD: SEAGATE 2.5" 1200GB SAS 12Gb/s, 10K RPM, eMLC 32GB, 4KN (6 of them)

Hope this helps.

Far as i know the LSI 3xxx series drivers are VERY alpha, at least last i saw @cyberjock post about it....maybe he can weigh in on this

 

[BigDave]

From the FreeBSD supported hardware list Version 11.0
***this version (11.0) is NOT what FreeNAS 9.3 is based on BTW***

The following controllers are supported by the mpr(4) driver:

• LSI SAS 3004 (4 Port SAS)
  
  
• LSI SAS 3008 (8 Port SAS)
  
  
• LSI SAS 3108 (8 Port SAS) <<<<< Yours <<<<<
  

Now don't get me wrong here, it very well could be that this driver has now been included in FreeNAS 9.3
and I'm dead wrong. If the driver has not been included, you will see strange things going on like you have described.
IIRC @cyberjock has stated more than once that JBOD is more often than not, truely not JBOD for most RAID cards.

 

[BigDave]

I just thought of something here. If your card is NOT giving FreeNAS direct access to the drives,
you would be unable to run smart tests or view smart info from the shell, No?

 

[sebyp]

@BigDave, you are correct. Here's the output below

[root@freenas] ~# smartctl -a /dev/mfisyspd0
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p16 amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/mfisyspd0: Unable to detect device type
Please specify device type with the -d option.

Use smartctl -h to get a usage summary

@BigDave, agreed, I have seen that in FreeBSD10.0 LSI 3108 is supported. I tried upgrading to FreeNAS10 nightly, but there I had other issues (CLI was not responsive, lot's of errors, all in all quite a bad experience).

Maybe someone can still provide some insight...

 

[cyberjock]

Yeah.. you're looking at a situation where you either find hardware that is more friendly with FreeNAS 9.3 or you find another OS to use. Unfortunately that's the only two options. :/

 

[sebyp]

@cyberjock, thanks for the info. I'll discuss with my hardware vendor and see what he can provide (RAID or HBA, but I think I'll go with HBA).

 

[Robert Trevellyan]

RAID or HBA, but I think I'll go with HBA

It needs to be HBA if you're sticking with FreeNAS.

 

[sebyp]

Right, back with an updated.

For the moment I've installed an LSI 3008 IT/HBA mode controller (this was the controller that the vendor could send me today).

Funny thing: FreeNAS was already installed, booted up nicely and then I could see all the HDD's. When I saw this was working I wanted to proceed with a new system, but when trying to boot from the USB stick, right after memory count (which shows correctly 64GB of RAM), I get the message below:

Kernel trap 12 with interrupts disabled

I've tried changing the USB stick, no luck.

I'm taking apart the server now and re-seat every major component (CPU, RAM, HDD). In case this doesn't work, does anyone have a clue?

Thanks!

 

[sebyp]

Update: it seems that BIOS setting were wrong (too much experimenting :)) ), as resetting them to defaults seem to have done the trick.

Also, another issue encountered was that even though FreeNAS was able to see the HDD's, he couldn't create a zvol (something about GPT on the HDD).

After switching HDD mode from AHCI to IDE that seems to have solved the trick.

Hope this is stable...

 

[Robert Trevellyan]

After switching HDD mode from AHCI to IDE that seems to have solved the trick.

You'd probably be best off wiping the drives, then switching back to AHCI mode.

By the way, a zvol is something specific, not a synonym for ZFS volume or pool.

A ZVOL is a ZFS block device that resides in your storage pool.

 

[rogerh]

Just a passing comment based on what people say here, not my own experience. If you are using the LSI 3008 it may be important to flash it with the correct firmware version (I think phase 5?) for the FreeNAS driver.

 

[Ericloewe]

According to the man page, the 3108 should be usable in IT mode or something compatible, since it works with the mpr driver, besides the mfi driver.

 

[DaveFL]

Is the 3108 now stable?

 

[Ericloewe]

SAS3 stuff just got updated to P9 drivers and firmware. It's hoped that this is the first stable release, but there's not enough data yet.

The 3108 is also contingent on you figuring out how to use it as an HBA with no HW RAID features.

 

[DaveFL]

SAS3 stuff just got updated to P9 drivers and firmware. It's hoped that this is the first stable release, but there's not enough data yet.

The 3108 is also contingent on you figuring out how to use it as an HBA with no HW RAID features.

So in your opinion is it worth it to go with one of SM's boards that have a built in LSI 3xxx chip or easier to stick with an external card? I'm guessing the latter until we here more.

 

[Ericloewe]

So in your opinion is it worth it to go with one of SM's boards that have a built in LSI 3xxx chip or easier to stick with an external card? I'm guessing the latter until we here more.

SAS3 will eventually be as stable as SAS2 is now. If you can make do with just 10 ports for a while and are ok with the greater upfront investment, the integrated SAS 3008 controllers are interesting.