Hi, we've been running freenas for some time now and on the whole it works well. However we are continually having problems with our VMware host which is using storage from freenas via iSCSI. The symptoms are always the same, guest vm's stop working, datastore offline (ESX5.5), freenas generally unresponsive. The freenas host machine specs are below
Freenas version: 9.2.1.5
Supermicro M/B
Opteron 6272
64GB ECC Ram
IBM1015 HBA crossflashed to LSI
6 x 3TB Seagate Constellation CS SAS
1 x 128gb Samsung 840 Pro L2 arc
8 x 1GB Intel NICs on various cards
ESXi and Freenas are connected using MPIO over 3 different paths all on seperate IP ranges and physical NICs
Here is the extract from /var/log/messages
Jul 23 01:33:49 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:49 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:50 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:53 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:56 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:59 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:34:02 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Any ideas?
Freenas version: 9.2.1.5
Supermicro M/B
Opteron 6272
64GB ECC Ram
IBM1015 HBA crossflashed to LSI
6 x 3TB Seagate Constellation CS SAS
1 x 128gb Samsung 840 Pro L2 arc
8 x 1GB Intel NICs on various cards
ESXi and Freenas are connected using MPIO over 3 different paths all on seperate IP ranges and physical NICs
Here is the extract from /var/log/messages
Jul 23 01:33:49 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:49 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:50 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:52 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:53 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:55 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:56 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:33:58 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:33:59 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt_iscsi.c:6256:istgt_create_conn: ***ERROR*** pthread_create() failed
Jul 23 01:34:01 shfstorage01 istgt[26343]: istgt.c:2480:istgt_acceptor: ***ERROR*** istgt_create_conn() failed
Jul 23 01:34:02 shfstorage01 istgt[26343]: istgt_iscsi.c:5551:worker: ***ERROR*** pthread_create() failed
Any ideas?