iSCSI service not start

[Maurizio]

Good afternoon,

I have a problem with the FreeNas iSCSI service. When i try to start it from GUI, this message appears:

"The service could not be started"

I have checked the messages log file and I found these errors:

Mar 10 13:10:57 freenas notifier: Starting istgt.
Mar 10 13:10:57 freenas notifier: istgt version 0.5 (20121028)
Mar 10 13:10:57 freenas istgt[7315]: istgt version 0.5 (20121028)
Mar 10 13:10:57 freenas istgt[7315]: normal mode
Mar 10 13:10:57 freenas notifier: normal mode
Mar 10 13:10:57 freenas istgt[7315]: using kqueue
Mar 10 13:10:57 freenas notifier: using kqueue
Mar 10 13:10:57 freenas istgt[7315]: using host atomic
Mar 10 13:10:57 freenas notifier: using host atomic
Mar 10 13:10:57 freenas istgt[7315]: istgt_lu_disk.c: 642:istgt_lu_disk_init: ***ERROR*** LU3: LUN0: open error(errno=17)
Mar 10 13:10:57 freenas notifier: istgt_lu_disk.c: 642:istgt_lu_disk_init: ***ERROR*** LU3: LUN0: open error(errno=17)
Mar 10 13:10:57 freenas istgt[7315]: istgt_lu.c:2091:istgt_lu_init_unit: ***ERROR*** LU3: lu_disk_init() failed
Mar 10 13:10:57 freenas notifier: istgt_lu.c:2091:istgt_lu_init_unit: ***ERROR*** LU3: lu_disk_init() failed
Mar 10 13:10:57 freenas istgt[7315]: istgt_lu.c:2166:istgt_lu_init: ***ERROR*** LU3: lu_init_unit() failed
Mar 10 13:10:57 freenas notifier: istgt_lu.c:2166:istgt_lu_init: ***ERROR*** LU3: lu_init_unit() failed
Mar 10 13:10:57 freenas istgt[7315]: istgt.c:2799:main: ***ERROR*** istgt_lu_init() failed
Mar 10 13:10:57 freenas notifier: istgt.c:2799:main: ***ERROR*** istgt_lu_init() failed
Mar 10 13:10:57 freenas notifier: LU2 HDD UNIT
Mar 10 13:10:57 freenas notifier: LU2: LUN0 file=/dev/zvol/MSA500G2/TestIdea/TestIdeaVolume, size=644245094400
Mar 10 13:10:57 freenas notifier: LU2: LUN0 1258291200 blocks, 512 bytes/block
Mar 10 13:10:57 freenas notifier: LU2: LUN0 600.0GB storage for iqn.2014-02.freenas.bim:testidea
Mar 10 13:10:57 freenas notifier: LU2: LUN0 serial 001018103d8e010
Mar 10 13:10:57 freenas notifier: LU2: LUN0 read cache enabled, write cache enabled
Mar 10 13:10:57 freenas notifier: LU2: LUN0 command queuing enabled, depth 32
Mar 10 13:10:57 freenas notifier: LU3 HDD UNIT
Mar 10 13:10:57 freenas notifier: LU3: LUN0 file=/dev/da12, size=146778685440
Mar 10 13:10:57 freenas notifier: LU3: LUN0 286677120 blocks, 512 bytes/block
Mar 10 13:10:58 freenas root: /usr/local/etc/rc.d/istgt: WARNING: failed to start istgt
Mar 10 13:10:58 freenas notifier: /usr/local/etc/rc.d/istgt: WARNING: failed to start istgt


I don't know what has happened... untill last Friday everything have worked fine, than suddently my server lose the iSCSI LUN.
Can someone help me?

Best regards

Maurizio

 

[HoneyBadger]

Code:

Mar 10 13:10:57 freenas istgt[7315]: istgt_lu_disk.c: 642:istgt_lu_disk_init: ***ERROR*** LU3: LUN0: open error(errno=17)
Mar 10 13:10:57 freenas notifier: istgt_lu_disk.c: 642:istgt_lu_disk_init: ***ERROR*** LU3: LUN0: open error(errno=17)
Mar 10 13:10:57 freenas istgt[7315]: istgt_lu.c:2091:istgt_lu_init_unit: ***ERROR*** LU3: lu_disk_init() failed
Mar 10 13:10:57 freenas notifier: istgt_lu.c:2091:istgt_lu_init_unit: ***ERROR*** LU3: lu_disk_init() failed
Mar 10 13:10:57 freenas istgt[7315]: istgt_lu.c:2166:istgt_lu_init: ***ERROR*** LU3: lu_init_unit() failed
Mar 10 13:10:57 freenas notifier: istgt_lu.c:2166:istgt_lu_init: ***ERROR*** LU3: lu_init_unit() failed
Mar 10 13:10:57 freenas istgt[7315]: istgt.c:2799:main: ***ERROR*** istgt_lu_init() failed
Mar 10 13:10:57 freenas notifier: istgt.c:2799:main: ***ERROR*** istgt_lu_init() failed
...
Mar 10 13:10:57 freenas notifier: LU3 HDD UNIT
Mar 10 13:10:57 freenas notifier: LU3: LUN0 file=/dev/da12, size=146778685440
Mar 10 13:10:57 freenas notifier: LU3: LUN0 286677120 blocks, 512 bytes/block

There's a fault with LU3:LUN0 (/dev/da12)

What are your hardware specs and what version of FreeNAS are you running? You're definitely not on 9.3 since you're still using istgt instead of ctl.

 

[Maurizio]

Hi HoneyBadger, thank for replay


FreeNas are running on HP proliant ML350 G5

Platform Intel(R) Xeon(R) CPU E5320 @ 1.86GHz
Memory 7145MB
8 SAS disks 2.5"" 146 Gbyte (InternalBay)

Storage array HP MSA500 with 8 146 gbyte Ultra SCSI 320 and 3 300Gbyte Ultra SCSI 320

FreeNAS-9.2.0-RELEASE-x64 (ab098f4)

 

[HoneyBadger]

Oh dear. Unless you've added in a SAS controller, which is unlikely given the presence of an MSA500, there's definitely hardware RAID at play here.

The ML350 G5 has an integrated E200i SAS RAID card, which I believe is also driving the MSA500. Did you configure these drives in a single large RAID array, or as individual RAID0 volumes? Either way it is bad news for FreeNAS as ZFS does not like hardware RAID cards in the slightest. It needs direct disk access.

I'd look at your E200i controller logs to see if the virtual RAID disk mapped to /dev/da12 is broken or failing, but in all likelihood it's toast.

 

[cyberjock]

I'm betting your pool is like 98% full.

Can you post a debug file from System -> Advanced -> Save Debug?

 

[Maurizio]

Good morning HoneyBadger, Good morning Cyberjick,
the E200i SAS controller driver the 8 SAS disks 2.5"" 146 Gbyte (InternalBay) and the MSA500 is drived by a proprietary HP controller,and each drive is configured as a singole RAID0 volume.
Where can I find the E200i controller logs?

Actually I have only 8 Gbyte free in the MSA500 storage and 3.6 Gbyte free in the InternalBay storage. As you requested I uploded the debug file.

Thanks

Maurizio

 

[cyberjock]

Well, if you are doing single disk RAID-0s, then I'm done with this conversation. I really can't spend time on threads that are using inappropriate hardware to start off with anymore. There's too many of them that turn into 'it's your inappropriate hardware" and I really am tired of troubleshooting a dozen or more every week.

As I've written in several places, RAID0 of single disks is not appropriate for ZFS and creates problems. RAID0 of single disks is NOT JBOD.

 

[HoneyBadger]

Where can I find the E200i controller logs?

You'll have to reboot the system most likely and hit a hotkey combination (F8? Ctrl-S?) on boot when the E200i initializes itself. I can't recall offhand.

As @cyberjock bluntly put it, ZFS on hardware RAID is not supported whatsoever and is repeatedly discouraged for this reason. Your data on that device is very likely gone.

 

[Maurizio]

Good morning HoneyBadge, good morning Cyberjock

ok, I'm going to read the guide and rebuild the Freenas...

Thanks

Maurizio