Can't create Volume with disks attached to LSI 9300-8e

Matt Kent · Aug 23, 2016

depasseg said:
seeing /dev/mfi and /dev/pass devices.

Drives can be presented as individual "JBOD" via a HW RAID controller and some think that they aren't using HW RAID, when they actually are. In this case though, the LSI does look like it's in IT mode, but I'm not sure about the H730.

I think there is an issue with the driver and H730 controller combo. And either a physical issue between the LSI and the MD1400, or an incompatibility with the MD1400.

For testing I have now removed the H730 form the machine, and put 4 of the drives form the server in the MD1400 shelf. The remaining drive are disconnected from the server as well.

Are you saying that there may be a problem with the LSI talking to the MD1400 at this point? Would you suggest manual partition and volume creation on the drives to see if they are addressable by the base OS?

Matt

depasseg · Aug 23, 2016

It's possible that there is a compatibility issue with the LSI, MD1400 and FreeBSD. What's the output of camcontrol devlist -v?

Matt Kent · Aug 23, 2016

Here you go:

Code:

scbus0 on ahcich0 bus 0:
<>                                 at scbus0 target -1 lun ffffffff ()
scbus1 on ahcich1 bus 0:
<>                                 at scbus1 target -1 lun ffffffff ()
scbus2 on ahcich8 bus 0:
<>                                 at scbus2 target -1 lun ffffffff ()
scbus3 on ahcich9 bus 0:
<>                                 at scbus3 target -1 lun ffffffff ()
scbus4 on mpr0 bus 0:
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 8 lun 0 (pass0,da0)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 9 lun 0 (pass1,da1)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 10 lun 0 (pass2,da2)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 11 lun 0 (pass3,da3)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 12 lun 0 (pass4,da4)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 13 lun 0 (pass5,da5)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 14 lun 0 (pass6,da6)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 15 lun 0 (pass7,da7)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 16 lun 0 (pass8,da8)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 17 lun 0 (pass9,da9)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 18 lun 0 (pass10,da10)
<TOSHIBA MG04SCA40EN DS06>         at scbus4 target 19 lun 0 (pass11,da11)
<DELL MD1400 1.05>                 at scbus4 target 20 lun 0 (pass12,ses0)
<>                                 at scbus4 target -1 lun ffffffff ()
scbus5 on camsim0 bus 0:
<>                                 at scbus5 target -1 lun ffffffff ()
scbus6 on umass-sim0 bus 0:
<SanDisk Ultra 1.00>               at scbus6 target 0 lun 0 (pass13,da12)
scbus-1 on xpt0 bus 0:
<>                                 at scbus-1 target -1 lun ffffffff (xpt0)

Matt

depasseg · Aug 23, 2016

That looks good. Can you use the GUI to create a 12 disk vdev (or 2 6's) as a test?

Matt Kent · Aug 23, 2016

depasseg said:
That looks good. Can you use the GUI to create a 12 disk vdev (or 2 6's) as a test?

I'm assuming that you mean use the web GUI, click on "Volume Manager" and create a volume? If so, then that is the issue that brought me here in the first place.
This is in the messages log:
manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to GPT format the disk "da0"] Along with a whole host of ILLEGAL REQUEST messages from the driver. I can post them but they are essentially the same as the ones that I have posted before.

Matt

depasseg · Aug 23, 2016

Hmm. Can you trying wiping those disks? The simplest and fastest way I know is to use gpart destroy -F da0 (through da11). And then confirm the partitions are all clear with gpart show.
Let's try to figure out this MD1400 issue and then move to the H730 and IT mode issue.

Matt Kent · Aug 23, 2016

depasseg said:
Hmm. Can you trying wiping those disks? The simplest and fastest way I know is to use gpart destroy -F da0 (through da11). And then confirm the partitions are all clear with gpart show.
Let's try to figure out this MD1400 issue and then move to the H730 and IT mode issue.

Done:

Code:

[root@xxxxxxxx] ~# gpart destroy -F da0
da0 destroyed
[root@xxxxxxxx] ~# gpart destroy -F da1
gpart: arg0 'da1': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da2
gpart: arg0 'da2': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da3
gpart: arg0 'da3': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da4
gpart: arg0 'da4': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da5
gpart: arg0 'da5': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da6
gpart: arg0 'da6': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da7
gpart: arg0 'da7': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da9
gpart: arg0 'da9': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da8
gpart: arg0 'da8': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da10
gpart: arg0 'da10': Invalid argument
[root@xxxxxxxx] ~# gpart destroy -F da11
gpart: arg0 'da11': Invalid argument
[root@xxxxxxxx] ~# gpart show
=>      34  30463933  da12  GPT  (15G)
        34      1024     1  bios-boot  (512K)
      1058         6        - free -  (3.0K)
      1064  30462896     2  freebsd-zfs  (15G)
  30463960         7        - free -  (3.5K)

Next?

depasseg · Aug 23, 2016

My bad, try /dev/da0 I didn't feel like testing that command on my drives.

Matt Kent · Aug 23, 2016

depasseg said:
My bad, try /dev/da0 I didn't feel like testing that command on my drives. :)

I think the invalid argument is due to the drives not having any partitions set. The gpart show is at the bottom of my last message.

depasseg · Aug 23, 2016

That could be the case. Maybe try to create a partition manually to see if it's possible (as you guessed a couple hours ago).

Spearfoot · Aug 23, 2016

Matt Kent said:
I think the invalid argument is due to the drives not having any partitions set. The gpart show is at the bottom of my last message.

I may be wrong, but I'm pretty sure you need to run the command with the full device spec: /dev/da0 etc. We can't see the 'gpart show' results for any devices except da12.

EDIT: I'm wrong; you don't need to specify the /dev/ prefix with the gpart command. Carry on...

Matt Kent · Aug 23, 2016

And here is my results:

Code:

[root@xxxxxxxx] ~# gpart create -s GPT /dev/da3
gpart: Invalid argument
[root@xxxxxxxx] ~# tail -6 /var/log/messages
Aug 23 13:48:52 xxxxxxxx (da2:mpr0:0:10:0): Error 22, Unretryable error
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): WRITE(16). CDB: 8a 00 00 00 00 01 d1 c0 be 8f 00 00 00 20 00 00
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): CAM status: SCSI Status Error
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): SCSI status: Check Condition
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): Error 22, Unretryable error
[root@xxxxxxxx] ~# gpart show
=>      34  30463933  da12  GPT  (15G)
        34      1024     1  bios-boot  (512K)
      1058         6        - free -  (3.0K)
      1064  30462896     2  freebsd-zfs  (15G)
  30463960         7        - free -  (3.5K)

=>        34  7814037101  da0  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da1  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da2  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da3  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

I'm hoping that I have a simple error in my command syntax? Please? ^_^

Spearfoot · Aug 23, 2016

Matt Kent said:

And here is my results:

Code:

[root@xxxxxxxx] ~# gpart create -s GPT /dev/da3
gpart: Invalid argument
[root@xxxxxxxx] ~# tail -6 /var/log/messages
Aug 23 13:48:52 xxxxxxxx (da2:mpr0:0:10:0): Error 22, Unretryable error
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): WRITE(16). CDB: 8a 00 00 00 00 01 d1 c0 be 8f 00 00 00 20 00 00
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): CAM status: SCSI Status Error
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): SCSI status: Check Condition
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): Error 22, Unretryable error
[root@xxxxxxxx] ~# gpart show
=>      34  30463933  da12  GPT  (15G)
        34      1024     1  bios-boot  (512K)
      1058         6        - free -  (3.0K)
      1064  30462896     2  freebsd-zfs  (15G)
  30463960         7        - free -  (3.5K)

=>        34  7814037101  da0  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da1  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da2  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da3  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

I'm hoping that I have a simple error in my command syntax? Please? ^_^

I'm really starting to believe you simply have a bad card...

depasseg · Aug 23, 2016

try without the /dev/
https://www.freebsd.org/doc/handbook/disks-adding.html

Matt Kent · Aug 23, 2016

Same response.

Code:

[root@xxxxxxxx] ~# gpart create -s GPT da4
gpart: Invalid argument
[root@xxxxxxxx] ~# tail -6 /var/log/messages
Aug 23 13:49:10 xxxxxxxx (da3:mpr0:0:11:0): Error 22, Unretryable error
Aug 23 14:01:17 xxxxxxxx (da4:mpr0:0:12:0): WRITE(16). CDB: 8a 00 00 00 00 01 d1 c0 be 8f 00 00 00 20 00 00
Aug 23 14:01:17 xxxxxxxx (da4:mpr0:0:12:0): CAM status: SCSI Status Error
Aug 23 14:01:17 xxxxxxxx (da4:mpr0:0:12:0): SCSI status: Check Condition
Aug 23 14:01:17 xxxxxxxx (da4:mpr0:0:12:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
Aug 23 14:01:17 xxxxxxxx (da4:mpr0:0:12:0): Error 22, Unretryable error
[root@xxxxxxxx] ~# gpart show
=>      34  30463933  da12  GPT  (15G)
        34      1024     1  bios-boot  (512K)
      1058         6        - free -  (3.0K)
      1064  30462896     2  freebsd-zfs  (15G)
  30463960         7        - free -  (3.5K)

=>        34  7814037101  da0  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da1  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da2  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da3  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

=>        34  7814037101  da4  GPT  (3.6T)
          34  7814037101       - free -  (3.6T)

Manufacturer tech support there I come.

wblock · Aug 23, 2016

depasseg said:
try without the /dev/
https://www.freebsd.org/doc/handbook/disks-adding.html

Even if gpart did not accept both ways (it does), it should not be able to trigger low-level errors like that. That is the card or the drive.

bstaudt · Apr 11, 2017

I understand this is an old thread but I ran into a similar situation with similar hardware and was able to solve it by formatting my drives with the following command and rebooting FreeNAS. After that I was able to create volumes using webUI:

Code:

sg_format --format --fmtpinfo=0 /dev/daX

Other forum threads about this issue:
https://forums.freenas.org/index.ph...es-9240-9211-w-ibm-seagate-disks.43453/page-2
https://forums.freenas.org/index.php?threads/solved-freenas-9-10-1-u4-scsi-status-error.48177/

Important Announcement for the TrueNAS Community.

Can't create Volume with disks attached to LSI 9300-8e

Matt Kent

Dabbler

depasseg

FreeNAS Replicant

Matt Kent

Dabbler

depasseg

FreeNAS Replicant

Matt Kent

Dabbler

depasseg

FreeNAS Replicant

Matt Kent

Dabbler

depasseg

FreeNAS Replicant

Matt Kent

Dabbler

depasseg

FreeNAS Replicant

Spearfoot

He of the long foot

Matt Kent

Dabbler

Spearfoot

He of the long foot

depasseg

FreeNAS Replicant

Matt Kent

Dabbler

wblock

Documentation Engineer

bstaudt

Cadet

Similar threads