BabyDatahoarder
Cadet
- Joined
- Apr 8, 2017
- Messages
- 2
Hello all.
I recently moved from using a normal internal sas card to using a sas expander and external chassis.
Here is the weird thing; If i turn on the external system (including the expander) before starting freenas then it will see
at most one or two drives.
To get it working I have to turn of the power to the storage system and turn it back on again.
Then it takes a while and nothing happens,
I now have to open a ssh-connection and issue sas2ircu 0 display which usually times out at the first try.
Doing it again and everything comes to life, all the disks are listed and after a few seconds they also show up in freenas gui.
Using Freenas 9.10.2-U2.
SAS2008 fw 20.00.04.00
HP 36 port SAS expander fw 2.08 (sierra)
Card and expander is running at 6gb, the fastest hgst disk are also running 6gb, the rest are either running at 3gb or 1,5 gb (I have no idea there either, assuming disk manufacturer sets the bus speed depending on how fast the data can get from the drive).
I have absolutely no idea except possibly that the nas is running on a hyperconverged main server and the sas-card is passed through. Never had a single problem with this config when the disks where attached directly to the sas card.
Attaching the security log as it shows exactly what happens.
First the startup at the top, then the detach after i turn off the power and finally when all the disks are recognized.
Not a single hickup (I/O-error or such) after the disks are detected even during hard work.
Thanks.
I recently moved from using a normal internal sas card to using a sas expander and external chassis.
Here is the weird thing; If i turn on the external system (including the expander) before starting freenas then it will see
at most one or two drives.
To get it working I have to turn of the power to the storage system and turn it back on again.
Then it takes a while and nothing happens,
I now have to open a ssh-connection and issue sas2ircu 0 display which usually times out at the first try.
Doing it again and everything comes to life, all the disks are listed and after a few seconds they also show up in freenas gui.
Using Freenas 9.10.2-U2.
SAS2008 fw 20.00.04.00
HP 36 port SAS expander fw 2.08 (sierra)
Card and expander is running at 6gb, the fastest hgst disk are also running 6gb, the rest are either running at 3gb or 1,5 gb (I have no idea there either, assuming disk manufacturer sets the bus speed depending on how fast the data can get from the drive).
I have absolutely no idea except possibly that the nas is running on a hyperconverged main server and the sas-card is passed through. Never had a single problem with this config when the disks where attached directly to the sas card.
Attaching the security log as it shows exactly what happens.
First the startup at the top, then the detach after i turn off the power and finally when all the disks are recognized.
Not a single hickup (I/O-error or such) after the disks are detected even during hard work.
Thanks.
Code:
freenas.local kernel log messages: > CPU: AMD Opteron 240 (Gen 1 Class Opteron) (3500.46-MHz K8-class CPU) > random: unblocking device. > mps0: SAS Address from SATA device = 17443d6b92718268 > mps0: SAS Address from SATA device = 17533b6692718267 > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x000b > mpssas_fw_work: failed to add device with handle 0xb > mps0: SAS Address from SATA device = 49806163dab4ba74 > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x000c > mpssas_fw_work: failed to add device with handle 0xc > mps0: SAS Address from SATA device = 4965694bf9a3b997 > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x000d > mpssas_fw_work: failed to add device with handle 0xd > mps0: SAS Address from SATA device = 4e43455fc98e9c6d > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x000e > mpssas_fw_work: failed to add device with handle 0xe > mps0: SAS Address from SATA device = 462d475fc98e9c6d > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x000f > mpssas_fw_work: failed to add device with handle 0xf > mps0: SAS Address from SATA device = 4875495ac9cfd085 > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x0010 > mpssas_fw_work: failed to add device with handle 0x10 > mps0: SAS Address from SATA device = 4875495ac9b7ad94 > failure at /freenas-9.10-releng/_BE/os/sys/dev/mps/mps_sas_lsi.c:675/mpssas_add_device()! Could not get ID for device with handle 0x0011 > mpssas_fw_work: failed to add device with handle 0x11 > da0 at mps0 bus 0 scbus2 target 14 lun 0 > da0: <ATA HGST HDN726050AL T517> Fixed Direct Access SPC-4 SCSI device > da0: Serial Number NAG3BK7P > da0: 4769307MB (9767541168 512 byte sectors) > ses0: da0,pass2: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: phy 0: parent 5001438022126226 addr 5001438022126200 > SMP: AP CPU #2 Launched! > vboxdrv: fAsync=0 offMin=0xa5a offMax=0x50d2d > GEOM_ELI: Device gptid/9453606d-be60-11e6-baf1-000c29af1675.eli created. > mps0: mpssas_prepare_remove: Sending reset for target ID 14 > da0 at mps0 bus 0 scbus2 target 14 lun 0 > mps0: da0: Unfreezing devq for target ID 14 > <ATA HGST HDN726050AL T517>mps0: s/n NAG3BK7P Unfreezing devq for target ID 18 > GEOM_ELI: Device gptid/9453606d-be60-11e6-baf1-000c29af1675.eli destroyed. > (da0:mps0:0:14:0): Periph destroyed > mps0: SAS Address for SATA device = 4875495ac9cfd085 > mps0: SAS Address for SATA device = 4875495ac9b7ad94 > mps0: SAS Address from SATA device = 4875495ac9cfd085 > mps0: SAS Address from SATA device = 4875495ac9b7ad94 > da1 at mps0 bus 0 scbus2 target 14 lun 0 > da1: <ATA HGST HDN726050AL T517> Fixed Direct Access SPC-4 SCSI device > da1: Serial Number NAG3BK7P > da1: 4769307MB (9767541168 512 byte sectors) > da2 at mps0 bus 0 scbus2 target 12 lun 0 > da2: <ATA WDC WD80EFZX-68U 0A83> Fixed Direct Access SPC-4 SCSI device > da2: Serial Number VK0DB2VY > da2: 7630885MB (15628053168 512 byte sectors) > da3 at mps0 bus 0 scbus2 target 15 lun 0 > da3: <ATA HGST HDN726050AL T517> Fixed Direct Access SPC-4 SCSI device > da3: Serial Number NAG2BZ5K > da3: 4769307MB (9767541168 512 byte sectors) > da4 at mps0 bus 0 scbus2 target 9 lun 0 > da4: Serial Number WD-WCC4ENPN3V37 > da5 at mps0 bus 0 scbus2 target 10 lun 0 > da5: <ATA WDC WD40EFRX-68W 0A80> Fixed Direct Access SPC-4 SCSI device > da5: Serial Number WD-WCC4E3X6RE2Z > da5: 300.000MB/s transfers > da5: Command Queueing enabled > da5: 3815447MB (7814037168 512 byte sectors) > da5: quirks=0x8<4K> > da6 at mps0 bus 0 scbus2 target 24 lun 0 > da6: <ATA WDC WD60EFRX-68L 0A82> Fixed Direct Access SPC-4 SCSI device > da6: Serial Number WD-WX41DC6PF25Y > da6: 150.000MB/s transfers > da6: Command Queueing enabled > da6: 5723166MB (11721045168 512 byte sectors) > da6: quirks=0x8<4K> > da7 at mps0 bus 0 scbus2 target 23 lun 0 > da7: <ATA WDC WD60EFRX-68L 0A82> Fixed Direct Access SPC-4 SCSI device > da7: Serial Number WD-WX41DC6PFJXJ > da7: 150.000MB/s transfers > da7: Command Queueing enabled > da7: 5723166MB (11721045168 512 byte sectors) > da7: quirks=0x8<4K> > ses0: da1,pass4: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: da3,pass6: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: da4,pass7: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: da5,pass8: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: da2,pass5: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: da7,pass10: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: phy 0: SATA device > ses0: phy 0: parent 5001438022126226 addr 5001438022126206 > ses0: da6,pass9: SAS Device Slot Element: 1 Phys at Slot 0 > ses0: phy 0: SATA device > ses0: phy 0: parent 5001438022126226 addr 5001438022126207 > GEOM_ELI: Device gptid/f874bba6-1bb6-11e7-80f8-3763eadf4da7.eli created. > GEOM_ELI: Encryption: AES-XTS 128 > GEOM_ELI: Crypto: hardware > GEOM_ELI: Device gptid/f94fcb68-1bb6-11e7-80f8-3763eadf4da7.eli created. > nfsd: can't register svc name > GEOM_ELI: Device da5p1.eli created. > GEOM_ELI: Encryption: AES-XTS 128 > GEOM_ELI: Crypto: hardware > GEOM_ELI: Device da6p1.eli created. > GEOM_ELI: Encryption: AES-XTS 128 > GEOM_ELI: Crypto: hardware > GEOM_ELI: Device da7p1.eli created. > GEOM_ELI: Encryption: AES-XTS 128 > GEOM_ELI: Crypto: hardware