Pulling hair out with building FreeNAS server

[John Weissberg]

We have tried to build a FreeNAS server using a re-purposed Supermicro chassis. Although building the system was simple enough, we cannot make it operate smoothly.

Here are the problems we have been experiencing:

1. After the machine is booted for a few hours, we lose contact with the machine and see this type of message on screen concerning disks connected to the HBA:

(probe4:mps0:0:4:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 500 completed timedout cm 0xffffff800100c0a0 ccb 0xfffffe0010f03000 during recovery ioc 8048 scsi 0 state c xfer 0


2. Whenever we try a large file transfer into the disks connected to the HBA, we begin to see error messages like this:
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 ba f8 00 00 80 00 length 65536 SMID 77 terminated ioc 804b scsi 0 state c xfer 0
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 ba 78 00 00 80 00 length 65536 SMID 334 terminated ioc 804b scsi 0 state c xfer 0
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 b8 e8 00 00 80 00 length 65536 SMID 720 terminated ioc 804b scsi 0 state c xfer 0
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 b8 68 00 00 80 00 length 65536 SMID 1003 terminated ioc 804b scsi 0 state c xfer 0
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 b9 f0 00 00 80 00 length 65536 SMID 552 terminated ioc 804b scsi 0 state c xfer 0
(da0:mps0:0:0:0): WRITE(10). CDB: 2a 00 14 a4 b9 70 00 00 80 00 length 65536 SMID 547 terminated ioc 804b scsi 0 state c xfer 0

The probe and write problems above are seen on multiple different disks connected to the HBA. These different disks are connected via different 8087 breakout cables.


--- Server information --

Server:
2U Supermicro chassis
Motherboard: X8DTN+
CPU: 2 x Xeon 5520
RAM: 24Gb

HDD:
8 x Hitachi Ultrastor 2Tb Sata III
4 x WD 1TB Sata II

The 8 x 2Gb are connected to the HBA via 2 x 8087 cables
The 4 x 1Tb are connected to the motherboard via SATA cables

HBA:
IBM M1015 cross-flashed to LSI 9211 IT-mode
Firmware version: 16

FreeNAS:
Version: 9.3

ZFS pools:
he25mainzfs with 8 x 2Tb HDD
he25db with 4 x 1Tb HDD

 

[Ericloewe]

Possibly a bad HBA? Everything looks right, hardware-wise. Any chance you can try moving the 4x 1TB pool to the HBA and try writing to it? It's possible that it may be some very unusual interaction with the disks.

 

[jgreco]

Missing is any mention of the backplane in use. If you're trying to pump SATA 6Gbps through an old SATA 1 backplane ... pain.

There's a good thread in the hardware forum on burning in your system, and also links to testing your disks. All of this should be followed before you try to start doing anything with your NAS. The time to shake out bugs is during the build process, before it is put into production.

 

[sfcredfox]

Did you resolve the problems you mentioned in this post?

I am looking to purchase a very similar product if not the same one.

Was it a backplane issue? Or something wrong with the HBA?