4TB disks attached to 3ware 9550SE

Status
Not open for further replies.
Joined
Oct 30, 2015
Messages
8
Hi there,
I am wondering, why my 4 TB disks on the controller shown as 3.6 TB appear splitted as 2.2 and 1.8 TB disks in the administrative gui. Running 9.3 stable.
Regards,
Hubertus
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Probably because the 3ware card is doing some funky RAID (i.e. non-HBA only) stuff. Or that the controller doesn't support drives over 2TB.
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Probably because the 3ware card is doing some funky RAID (i.e. non-HBA only) stuff. Or that the controller doesn't support drives over 2TB.

Both, actually. That's a very old controller and you should look at replacing it immediately, however this is likely going to be difficult if you have any existing data.

What you're seeing is the "auto-carving" feature to make the >2.2TB disks appear as multiple smaller virtual disks.

The 3Ware 9000 series also doesn't officially support JBOD passthrough, so it's very likely that your drives have been configured as a bunch of single-disk RAID0s, which means they have 3Ware RAID headers on them and won't be importable to a regular HBA.

1. Back up everything on that system. Now.

2. Replace that 3Ware with a supported LSI or equivalent HBA

3. Create your new zpool and restore data.
 

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
The cards are listed on the FreeBSD 10 compat list.


[i386,amd64] The twa(4) driver supports the following SATA RAID controllers:

  • AMCC's 3ware 9500S-4LP

  • AMCC's 3ware 9500S-8

  • AMCC's 3ware 9500S-8MI

  • AMCC's 3ware 9500S-12

  • AMCC's 3ware 9500S-12MI

  • AMCC's 3ware 9500SX-4LP

  • AMCC's 3ware 9500SX-8LP

  • AMCC's 3ware 9500SX-12

  • AMCC's 3ware 9500SX-12MI

  • AMCC's 3ware 9500SX-16ML

  • AMCC's 3ware 9550SX-4LP

  • AMCC's 3ware 9550SX-8LP

  • AMCC's 3ware 9550SX-12

  • AMCC's 3ware 9550SX-12MI

  • AMCC's 3ware 9550SX-16ML

  • AMCC's 3ware 9650SE-2LP
  • AMCC's 3ware 9650SE-4LPML
  • AMCC's 3ware 9650SE-8LPML
  • AMCC's 3ware 9650SE-12ML
  • AMCC's 3ware 9650SE-16ML
  • AMCC's 3ware 9650SE-24M8
Not sure if makes any diff or not. Just thought I'd throw that out there...
 

rs225

Guru
Joined
Jun 28, 2014
Messages
878
9650SE should be fine with 4TB drives. Just make sure you are on the 4.10.00.027 firmware.
And make sure the controller is configured in the controller BIOS to use JBOD mode. There is a 'secret' key you press in the controller BIOS config screen. I think it is 'J', could also be S, but I think that puts it in the mode you are probably using: single drive RAID mode.

To see all 4TB as one drive, you would turn off the Auto-Carving Policy.

You'll wreck the pool if you try to change these things after the pool is created.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
The 9650 should not be used, period. It has no true JBOD mode.

I had that card... bought it and sold it 2 months later when i realized it is totally not even close to what you really "need" from your HBA controller.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
The cards are listed on the FreeBSD 10 compat list.


[i386,amd64] The twa(4) driver supports the following SATA RAID controllers:
Not sure if makes any diff or not. Just thought I'd throw that out there...

It makes a diff. You don't see us suggesting Marvell controllers, 3Ware RAID cards, Realtek ethernets, or RAID cards in general: the fact that there's driver support for crap doesn't mean that it is a good idea to use crap.
 

emk2203

Guru
Joined
Nov 11, 2012
Messages
573
The 9650 should not be used, period. It has no true JBOD mode.

I had that card... bought it and sold it 2 months later when i realized it is totally not even close to what you really "need" from your HBA controller.
Any source for that? AFAIK, my 9650SE supports JBOD mode, in which it just acts as a SATA controller.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Pretty sure that what the 9650 *CALLS* JBOD mode is actually single disk mode, and not actually a SATA passthrough mode. Try removing one of those disks and attaching it to a conventional SATA port and see if it is still readable. Or see if your device correctly identifies itself in "camcontrol devlist."

I'm not really interested in setting up a test platform just to try it again. I did so back when I wrote the LSI sticky.
 

David Febo

Cadet
Joined
May 3, 2016
Messages
7
Just for information, I performed the test that jgreco suggested two different times, one in JBOD and the other in singledisk and after creating the volumes and putting some data on them I shutdown the test system, removed the 9650se card and connected the four HD’s directly to the motherboard booted up the system and everything was working just the same. I did not had to do anything. FreeNAS recognized all the drives and all the data and it was working like nothing happened. Both times. I’m not saying the RAID card is not going to give problems somewhere else down the line, or recommend using the card, I just confirmed that the disks where recognized and used by the board as normal. Just to share. FreeNAS-9.10-STABLE-201604261518 (881b70d)
 

David Febo

Cadet
Joined
May 3, 2016
Messages
7
Are you referring to disk name, serial, etc.? Because volume names and volume stats where all the same. I did not had to mount or reconfigure anything. I was expecting a total crash so I didn’t spend too much time looking around. I was just surprised that it worked at all. I think I left the system with the card in and drives in singledisk, maybe tomorrow I could look the information you want to verify and connect drives back to MB and report. Note that both test were done with wrcache, rdcache and autoverify off in the cards bios so FreeNAS would handle the drives.
 

rs225

Guru
Joined
Jun 28, 2014
Messages
878
The 9650SE is pretty good, but it is not ideal. camcontrol devlist does not show the drive names.
smartctl does.

There is no problem moving disks between JBOD mode and another controller.
 

David Febo

Cadet
Joined
May 3, 2016
Messages
7
I did the test with the disks in singledisk and this what I found in my case. If I move over 3 disks in raidz everything was ok. But if I did the same with 4 disks in raidz2 the volume will not mount. Here is the devlist… hopefully will show in readable format. If I have time will repeat in JBOD tomorrow.

devlist with 9650 card in singledisk

[root@freenas ~]# camcontrol devlist
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 0 lun 0 (da0,pass0)
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 1 lun 0 (da1,pass1)
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 2 lun 0 (da2,pass2)
<ST980811AS 3.CDE> at scbus3 target 0 lun 0 (ada0,pass3)
[root@freenas ~]#


devlist straight to board

[root@freenas ~]# camcontrol devlist
<ST980811AS 3.CDE> at scbus2 target 0 lun 0 (ada0,pass0)
<HDS725050KLA360 K2AOAB5A> at scbus3 target 0 lun 0 (ada1,pass1)
<Hitachi HUA721050KLA330 GK6OA74A> at scbus4 target 0 lun 0 (ada2,pass2)
<ST3500410AS CC34> at scbus5 target 0 lun 0 (ada3,pass3)
[root@freenas ~]#

Remember this is just a bunch of old hardware just for testing the 9650 card. This is not production or certified hardware.
 

David Febo

Cadet
Joined
May 3, 2016
Messages
7
I think the raidz2 could have been recovered but I don’t have time for that. I’m just testing for straight swap from one to the other. With the raidz2 back on the card everything was still there.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
[root@freenas ~]# camcontrol devlist
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 0 lun 0 (da0,pass0)
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 1 lun 0 (da1,pass1)
<AMCC 9650SE-12M DISK 4.10> at scbus0 target 2 lun 0 (da2,pass2)
<ST980811AS 3.CDE> at scbus3 target 0 lun 0 (ada0,pass3)
[root@freenas ~]#

Yeah, those are still being intercepted by the RAID firmware on the card, which confirms that this isn't a good card to use.
 
Status
Not open for further replies.
Top