aborted command with Fibre Channel Targets

Status
Not open for further replies.
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Hello everyone,

I've setup FreeNAS 9.3 on my HP SE326M1.

Hardware:
HP P800 SmartArray Raid Controller
2x Intel Xeon L5220 CPU
32GB DDR3-ECC RAM
1x QLogic 2462 with 2 FC-Ports (4GBit/s)
1x QLogic 2462 with 4 FC-Ports (4GBit/s)

Logical Drives (created through the P800):
RAID 1: 2 x 146GB SAS 2,5 10K (FreeNAS is installed on this logical drive)
RAID 5: 6 x 300GB SAS 2,5, 10K (ZVOL on top as LUN0)
RAID 5: 5 x 300GB SAS 2,5, 10K (ZVOL on top as LUN1)
RAID 1: 2 x 300GB SAS 2,5, 10K (ZVOL on top as LUN2)

The FreeNAS is used as a SAN device to serve my ESXi-Hosts with LUNs over 4GBit Fibre Channel (MPIO). With the help of a very good tutorial on how to set up fibre channel in FreeNAS I got everything to work, so that the ESXi-Hosts can now see the LUNs.

Speed is excellent: IOmeter results in a Win2012R2 virtual machine got 417MB/s after 30 minutes tests with 256KB 100% sequential read.

Now there is only one problem: the performance is continuously breaking down and the following messages appear on the FreeNAS console:

Bildschirmfoto 2015-12-14 um 12.51.10.png


Can anybody help?
 
M

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
Are those messages all that you see? Are there any other messages before that on screen or in logs?

FreeNAS nightly branch recently got large set of FC updates, so you may try that.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Hi,

here are the first lines which are outputted on the console when the error occurs. The list goes continuously on:

Code:
Dec 14 02:54:15 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x11d100
Dec 14 02:54:15 DE001-STR-44-2 (ctl12:isp4:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x11d100
Dec 14 02:54:15 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x11d158
Dec 14 02:54:15 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x11d208
Dec 14 02:54:15 DE001-STR-44-2 (ctl12:ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x114334
Dec 14 02:54:15 DE001-STR-44-2 isp4:0:ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126580
Dec 14 02:54:15 DE001-STR-44-2 (ctl21:256:0): isp3:0:ctlfe_done: returning task I/O tag 0xffffffff seq 0x11d158
Dec 14 02:54:15 DE001-STR-44-2 256:(ctl13:2): isp4:0:ctlfe_done: returning task I/O tag 0xffffffff seq 0x114334
Dec 14 02:54:15 DE001-STR-44-2 256:1): ctlfe_done: returning task I/O tag 0xffffffff seq 0x11d208
Dec 14 02:54:15 DE001-STR-44-2 (ctl6:isp3: isp_handle_platform_ctio: CTIO7[114334] seq 0 nc 1 sts 0x2 flg 0x1 sns 0 resid 0 MID
Dec 14 02:54:15 DE001-STR-44-2 isp2:0:isp4: isp_handle_platform_ctio: CTIO7[11d208] seq 2 nc 1 sts 0x2 flg 0xa060 sns 0 resid 0 FIN
Dec 14 02:54:15 DE001-STR-44-2 (ctl21:isp3:0:256:256:0): 2): ctlfe_done: returning task I/O tag 0xffffffff seq 0x126580
Dec 14 02:54:15 DE001-STR-44-2 ctlfestart: aborted command 0x114334 discarded
Dec 14 02:54:15 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x11d184
Dec 14 02:54:15 DE001-STR-44-2 (ctl12:isp4:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x11d184
Dec 14 02:54:15 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126764
Dec 14 02:54:15 DE001-STR-44-2 (ctl6:isp2:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x126764
Dec 14 02:54:16 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x11d1dc
Dec 14 02:54:16 DE001-STR-44-2 (ctl12:isp4:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x11d1dc
Dec 14 02:54:20 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126bdc
Dec 14 02:54:20 DE001-STR-44-2 (ctl3:isp1:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x126bdc
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126b58
Dec 14 02:54:21 DE001-STR-44-2 (ctl3:isp1:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x126b58
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126d68
Dec 14 02:54:21 DE001-STR-44-2 (ctl3:isp1:0:256:0): ctlfe_done: returning task I/O tag 0xffffffff seq 0x126d68
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126bb0
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126ce4
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126d10
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126d3c
Dec 14 02:54:21 DE001-STR-44-2 ctlfedone: got XPT_IMMEDIATE_NOTIFY status 0x36 tag 0xffffffff seq 0x126b84
...


BTW: The ESXi hosts as well as the SAN Appliance use the same Fibre Channel HBAs (QLogic QLE2462) with the same firmware.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Hi,

I recently updated to the latest Nightlies Build but the same errors appear :-(

EDIT: Here is the output which ctladm dumpooa gives:

Code:
[root@DE001-STR-44-2] ~# ctladm dumpooa
Dumping OOA queues
LUN 0 tag 0x124854 DMA RTR: WRITE(10). CDB: 2a 00 67 bf 41 80 00 00 80 00  (0 ms)
LUN 1 tag 0x129688 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10047 ms)
LUN 1 tag 0x1296b4 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10047 ms)
LUN 1 tag 0x1296e0 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10047 ms)
LUN 1 tag 0x12970c RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10047 ms)
LUN 1 tag 0x129738 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10047 ms)
LUN 1 tag 0x129764 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x129790 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x1297bc RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x1297e8 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x129814 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x129840 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x12986c RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 1 tag 0x129898 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (10046 ms)
LUN 2 tag 0x0aad RTR: READ(10). CDB: 28 00 37 45 b0 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0aae BLOCKED: READ(10). CDB: 28 00 37 45 b8 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0aaf BLOCKED: READ(10). CDB: 28 00 37 45 c0 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab0 BLOCKED: READ(10). CDB: 28 00 37 45 c8 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab1 BLOCKED: READ(10). CDB: 28 00 37 45 d0 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab2 BLOCKED: READ(10). CDB: 28 00 37 45 d8 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab3 BLOCKED: READ(10). CDB: 28 00 37 45 e0 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab4 BLOCKED: READ(10). CDB: 28 00 37 45 e8 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab5 BLOCKED: READ(10). CDB: 28 00 37 45 f0 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab6 BLOCKED: READ(10). CDB: 28 00 37 45 f8 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab7 BLOCKED: READ(10). CDB: 28 00 37 46 00 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab8 BLOCKED: READ(10). CDB: 28 00 37 46 08 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0ab9 BLOCKED: READ(10). CDB: 28 00 37 46 10 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0aba BLOCKED: READ(10). CDB: 28 00 37 46 18 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0abb BLOCKED: READ(10). CDB: 28 00 37 46 20 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0abc BLOCKED: READ(10). CDB: 28 00 37 46 28 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0abd BLOCKED: READ(10). CDB: 28 00 37 46 30 00 00 08 00 00  (10046 ms)
LUN 2 tag 0x0abe BLOCKED: READ(10). CDB: 28 00 37 46 38 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0abf BLOCKED: READ(10). CDB: 28 00 37 46 40 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac0 BLOCKED: READ(10). CDB: 28 00 37 46 48 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac1 BLOCKED: READ(10). CDB: 28 00 37 46 50 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac2 BLOCKED: READ(10). CDB: 28 00 37 46 58 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac3 BLOCKED: READ(10). CDB: 28 00 37 46 60 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac4 BLOCKED: READ(10). CDB: 28 00 37 46 68 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac5 BLOCKED: READ(10). CDB: 28 00 37 46 70 00 00 08 00 00  (10044 ms)
LUN 2 tag 0x0ac6 BLOCKED: READ(10). CDB: 28 00 37 46 78 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0ac7 BLOCKED: READ(10). CDB: 28 00 37 46 80 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0ac8 BLOCKED: READ(10). CDB: 28 00 37 46 88 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0ac9 BLOCKED: READ(10). CDB: 28 00 37 46 90 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0aca BLOCKED: READ(10). CDB: 28 00 37 46 98 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0acb BLOCKED: READ(10). CDB: 28 00 37 46 a0 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0acc BLOCKED: READ(10). CDB: 28 00 37 46 a8 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0acd BLOCKED: READ(10). CDB: 28 00 37 46 b0 00 00 08 00 00  (10043 ms)
LUN 2 tag 0x0ace BLOCKED: READ(10). CDB: 28 00 37 46 b8 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0acf BLOCKED: READ(10). CDB: 28 00 37 46 c0 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad0 BLOCKED: READ(10). CDB: 28 00 37 46 c8 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad1 BLOCKED: READ(10). CDB: 28 00 37 46 d0 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad2 BLOCKED: READ(10). CDB: 28 00 37 46 d8 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad3 BLOCKED: READ(10). CDB: 28 00 37 46 e0 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad4 BLOCKED: READ(10). CDB: 28 00 37 46 e8 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad5 BLOCKED: READ(10). CDB: 28 00 37 46 f0 00 00 08 00 00  (10042 ms)
LUN 2 tag 0x0ad6 BLOCKED: READ(10). CDB: 28 00 37 46 f8 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0ad7 BLOCKED: READ(10). CDB: 28 00 37 47 00 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0ad8 BLOCKED: READ(10). CDB: 28 00 37 47 08 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0ad9 BLOCKED: READ(10). CDB: 28 00 37 47 10 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0ada BLOCKED: READ(10). CDB: 28 00 37 47 18 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0adb BLOCKED: READ(10). CDB: 28 00 37 47 20 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0adc BLOCKED: READ(10). CDB: 28 00 37 47 28 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x0add BLOCKED: READ(10). CDB: 28 00 37 47 30 00 00 08 00 00  (10041 ms)
LUN 2 tag 0x117ac4 RTR: READ(10). CDB: 28 00 18 8a 4d 68 00 00 88 00  (4069 ms)
LUN 2 tag 0x117af0 RTR: READ(10). CDB: 28 00 18 89 f0 f0 00 00 90 00  (4063 ms)
LUN 2 tag 0x1182d8 RTR: READ(10). CDB: 28 00 18 a4 06 28 00 00 90 00  (1327 ms)
LUN 2 tag 0x11a068 RTR: READ(10). CDB: 28 00 36 06 17 b8 00 00 98 00  (419 ms)
LUN 2 tag 0x11a0c0 RTR: READ(10). CDB: 28 00 36 05 ea e0 00 00 20 00  (150 ms)
LUN 2 tag 0x1186f8 RTR: READ(10). CDB: 28 00 18 90 87 c0 00 00 08 00  (137 ms)
OOA queues dump done
[root@DE001-STR-44-2] ~# ctladm dumpooa
Dumping OOA queues
LUN 0 tag 0x11f054 DMA RTR: WRITE(10). CDB: 2a 00 68 01 73 80 00 00 80 00  (0 ms)
LUN 1 tag 0x129790 ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x1297bc ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x1297e8 ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x129814 ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x129840 ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x12986c ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x129898 ABORT RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (30186 ms)
LUN 1 tag 0x11b6c0 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b6ec RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b718 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b744 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b770 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b79c RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 1 tag 0x11b7c8 RTR: EXTENDED COPY. CDB: 83 00 00 00 00 00 00 00 00 00 00 00 00 6c 00 00  (9936 ms)
LUN 2 tag 0x0ac5 RTR: READ(10). CDB: 28 00 37 46 70 00 00 08 00 00  (30184 ms)
LUN 2 tag 0x0ac6 BLOCKED: READ(10). CDB: 28 00 37 46 78 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ac7 BLOCKED: READ(10). CDB: 28 00 37 46 80 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ac8 BLOCKED: READ(10). CDB: 28 00 37 46 88 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ac9 BLOCKED: READ(10). CDB: 28 00 37 46 90 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0aca BLOCKED: READ(10). CDB: 28 00 37 46 98 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0acb BLOCKED: READ(10). CDB: 28 00 37 46 a0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0acc BLOCKED: READ(10). CDB: 28 00 37 46 a8 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0acd BLOCKED: READ(10). CDB: 28 00 37 46 b0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ace BLOCKED: READ(10). CDB: 28 00 37 46 b8 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0acf BLOCKED: READ(10). CDB: 28 00 37 46 c0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad0 BLOCKED: READ(10). CDB: 28 00 37 46 c8 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad1 BLOCKED: READ(10). CDB: 28 00 37 46 d0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad2 BLOCKED: READ(10). CDB: 28 00 37 46 d8 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad3 BLOCKED: READ(10). CDB: 28 00 37 46 e0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad4 BLOCKED: READ(10). CDB: 28 00 37 46 e8 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad5 BLOCKED: READ(10). CDB: 28 00 37 46 f0 00 00 08 00 00  (30182 ms)
LUN 2 tag 0x0ad6 BLOCKED: READ(10). CDB: 28 00 37 46 f8 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0ad7 BLOCKED: READ(10). CDB: 28 00 37 47 00 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0ad8 BLOCKED: READ(10). CDB: 28 00 37 47 08 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0ad9 BLOCKED: READ(10). CDB: 28 00 37 47 10 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0ada BLOCKED: READ(10). CDB: 28 00 37 47 18 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0adb BLOCKED: READ(10). CDB: 28 00 37 47 20 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0adc BLOCKED: READ(10). CDB: 28 00 37 47 28 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0add BLOCKED: READ(10). CDB: 28 00 37 47 30 00 00 08 00 00  (30181 ms)
LUN 2 tag 0x0b05 RTR: READ(10). CDB: 28 00 37 46 70 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b06 BLOCKED: READ(10). CDB: 28 00 37 46 78 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b07 BLOCKED: READ(10). CDB: 28 00 37 46 80 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b08 BLOCKED: READ(10). CDB: 28 00 37 46 88 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b09 BLOCKED: READ(10). CDB: 28 00 37 46 90 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0a BLOCKED: READ(10). CDB: 28 00 37 46 98 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0b BLOCKED: READ(10). CDB: 28 00 37 46 a0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0c BLOCKED: READ(10). CDB: 28 00 37 46 a8 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0d BLOCKED: READ(10). CDB: 28 00 37 46 b0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0e BLOCKED: READ(10). CDB: 28 00 37 46 b8 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b0f BLOCKED: READ(10). CDB: 28 00 37 46 c0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b10 BLOCKED: READ(10). CDB: 28 00 37 46 c8 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b11 BLOCKED: READ(10). CDB: 28 00 37 46 d0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b12 BLOCKED: READ(10). CDB: 28 00 37 46 d8 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b13 BLOCKED: READ(10). CDB: 28 00 37 46 e0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b14 BLOCKED: READ(10). CDB: 28 00 37 46 e8 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b15 BLOCKED: READ(10). CDB: 28 00 37 46 f0 00 00 08 00 00  (9902 ms)
LUN 2 tag 0x0b16 BLOCKED: READ(10). CDB: 28 00 37 46 f8 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b17 BLOCKED: READ(10). CDB: 28 00 37 47 00 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b18 BLOCKED: READ(10). CDB: 28 00 37 47 08 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b19 BLOCKED: READ(10). CDB: 28 00 37 47 10 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b1a BLOCKED: READ(10). CDB: 28 00 37 47 18 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b1b BLOCKED: READ(10). CDB: 28 00 37 47 20 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b1c BLOCKED: READ(10). CDB: 28 00 37 47 28 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x0b1d BLOCKED: READ(10). CDB: 28 00 37 47 30 00 00 08 00 00  (9900 ms)
LUN 2 tag 0x11d89c RTR: READ(10). CDB: 28 00 18 e9 65 68 00 00 20 00  (1094 ms)
LUN 2 tag 0x11dc38 RTR: READ(10). CDB: 28 00 18 9d fc b8 00 00 30 00  (584 ms)
LUN 2 tag 0x11d52c RTR: READ(10). CDB: 28 00 35 fb 2d 98 00 00 10 00  (392 ms)
LUN 2 tag 0x11dc90 RTR: READ(10). CDB: 28 00 18 a5 24 78 00 00 20 00  (218 ms)
LUN 2 tag 0x11dcbc RTR: READ(10). CDB: 28 00 18 9a eb 60 00 00 08 00  (202 ms)
LUN 2 tag 0x11d608 RTR: READ(10). CDB: 28 00 36 09 ea 50 00 00 08 00  (197 ms)


Operation was moving a virtual machine from LUN1 to LUN2 in the vSphere Client.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Ok, just to be sure that it is not a host specific problem I have migrated some virtual machines to another ESXi host. Same Errors.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Since we need the FreeNAS appliance as one of our production SAN storages and the problem even is not solved with Emulex Fibre Channel HBAs in the ESXi hosts I will replace the complete gear with 10GbE HBAs (Intel 10GbE XF SR2 Dual-Port) from Intel.

Hope FreeNAS will perform well over 10GbE iSCSI.
 
M

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
I would say that this may be not a problem of Fibre Channel. All I see there is commands aborted by initiator, probably due to exceeding some timeout. I see there some reads that took more then 30 seconds, that can be caused by some issues or bottlenecks on disk or ZFS layer.

Looking on configuration of your setup I can only repeat that RAID5 is usually not recommended for VM storage. Using ZFS on top of RAID array is not recommended either.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Hi,

thanks for your answer. Can you tell me which configuration of RAID is recommended for VM storage?
 
M

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
fc7t said:
Can you tell me which configuration of RAID is recommended for VM storage?
Click to expand...
It is already described many times on this forum. Instead of hardware RAID it is recommended to use plain HBA, passing through all raw disks. On top of that single ZFS pool with as many spindles as possible in many mirrored vdevs to get maximal IOPS rates. If read loads prevail and data are critical, some even using triple mirrors. Again, if data are critical, fast dedicated flash- or RAM-based log device can be recommended to make `sync=always` option viable.
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Thanks for your answer. In terms of ZFS as a reliable filesystem with access to the raw disks FreeNAS will not be an option for us anymore because the HP SmartArray P800 RAID Controller does not support the passhrough of raw disks to the OS (FreeNAS). The only option is to create single RAID 0 logical volumes but this configuration does not really serve the purpose as JBOD volumes.

Sorry to say that but we will use Win2012R2 server with Starwind Virtual SAN on it to present our LUNs over iSCSI to our ESXi hosts. Nonetheless I can say FreeNAS is a great open source NAS appliance from the experiences I have made with it.
 
T

Tony Wilkins

Cadet
Joined
Dec 18, 2015
Messages
1
fc7t said:
Thanks for your answer. In terms of ZFS as a reliable filesystem with access to the raw disks FreeNAS will not be an option for us anymore because the HP SmartArray P800 RAID Controller does not support the passhrough of raw disks to the OS (FreeNAS). The only option is to create single RAID 0 logical volumes but this configuration does not really serve the purpose as JBOD volumes.

Sorry to say that but we will use Win2012R2 server with Starwind Virtual SAN on it to present our LUNs over iSCSI to our ESXi hosts. Nonetheless I can say FreeNAS is a great open source NAS appliance from the experiences I have made with it.
Click to expand...
We've went ahead with that options as well. Have you tried using the HA functionality, because we have been using the storage as standalone and were wondering if we're missing anything?
 
F

fc7t

Dabbler
Joined
Dec 14, 2015
Messages
26
Hi,

we were struggling with the performance under Windows Server 2008 R2 and StarWind Virtual SAN Free. 4K random reads/writes were really bad despite the HP P410i Controller and a RAID 10 Array with 14 300GB 10K hard drives.

At the moment we have reinstalled FreeNAS again with the 10 GbE Adapters installed.

Because we want maximum performance and compatibility I have ordered a LSI 9211-8i Controller which will hopefully be in the mail on monday. In combination with the 10 GbE adapters, 64 GB of RAM and ZFS filesystem with HBA mode of the controller (with IT firmware flashed on the LSI controller) the performance should be acceptable.

Next month I will also order some of Samsung EVO 850 500 GB SSDs so performance, especially random reads/writes should be very good.
 
Status
Not open for further replies.

Similar threads

M
  • Locked
FreeNAS 11 Fibre Channel Target Mode LUN Port Mapping
Replies
1
Views
5K
mrkl
M
F
  • Locked
FC: write performance is fine, read stuck at 200MB/s @ 4GB FC
Replies
6
Views
3K
iqarmy
I
asrrin29
  • Locked
Can't get 2nd port of QLogic FC card to connect.
Replies
16
Views
5K
stillnick
stillnick
T
GUI access lost and rando errors happenning.
Replies
10
Views
958
TheUsD
T
S
  • Locked
Notes: using Freenas as a SAN with Fibre Channel
Replies
2
Views
2K
slushieken
S
Top