Hi All,
I've been running FreeNAS/TrueNAS for years. So far, I could always find help in other threads. This time it seems to be too specific, I have little experience with HBA and SAS Expander. I hope the title is not misleading, but it is my latest revelation.
First off, I am setting up a new System for testing purposes and I am Stuck.
The hardware and setup seems to work fine so far, I know I am using an 16i for external connection but with a little adapter card its working fine. When connection just one of the DS4246 to the HBA everything seems fine ass 24 drives are recognized, hooking up the second one is when the trouble starts. All the drives are seen by the HBA in BIOS as well as the IOM6 they are hooked up to.
So, I did a fresh install of TrueNAS CORE 12.0-U6.1 and to my surprise there are a lot of drives missing. Most of the time it just recognizes one complete shelve, but when I try to mix drives or when I do not populate all the bays the behavior starts to get random. The best I could achieve is 24 out of 48 disks, most of the time TrueNAS only sees 21 drives.
At a first thought I tried different cabling, chaining the shelves vs direct hook up and so on. Here is what I tried so far
So I figured it must be a driver Issue or something BSD related. I tried some tunables for mpr spinup time and what else I could find but nothing.
Since there is no newer driver I could find, and the newest I could find (v20) is much older than the used (v23) I am not sure how to proceed.
I really hope that someone can point me in the right direction. Since it is my first post please forgive me if I missed to attach some necessary Information.
I've been running FreeNAS/TrueNAS for years. So far, I could always find help in other threads. This time it seems to be too specific, I have little experience with HBA and SAS Expander. I hope the title is not misleading, but it is my latest revelation.
First off, I am setting up a new System for testing purposes and I am Stuck.
ASRock Rack ROMED6U-2L2T
AMD EPYC 7232P 8-Core Processor
2x Micron 64 GB DDR4-3200 reg ECC
SAS9305-16i 9305-16i (sas3224 - 16.00.12.00 - IT-Mode)
2x NetApp DS4246 (2xIOM6) on QSFP(SFF-8436) to SFF-8644
various SATA drives, some on Interposer (only got 24 Interposer so far)
The hardware and setup seems to work fine so far, I know I am using an 16i for external connection but with a little adapter card its working fine. When connection just one of the DS4246 to the HBA everything seems fine ass 24 drives are recognized, hooking up the second one is when the trouble starts. All the drives are seen by the HBA in BIOS as well as the IOM6 they are hooked up to.
So, I did a fresh install of TrueNAS CORE 12.0-U6.1 and to my surprise there are a lot of drives missing. Most of the time it just recognizes one complete shelve, but when I try to mix drives or when I do not populate all the bays the behavior starts to get random. The best I could achieve is 24 out of 48 disks, most of the time TrueNAS only sees 21 drives.
At a first thought I tried different cabling, chaining the shelves vs direct hook up and so on. Here is what I tried so far
- Hook up the shelves chained and both IOMs of the first
- Hook up the shelves directly by using only one IOM each
- Interposers on the drives for SAS connection
- No interposers at all (only first IOM recognizes the SATA drives but that's normal behavior as far as other forum threads told me)
- Tried different ports on the controller
- Tried different cables ect.
So I figured it must be a driver Issue or something BSD related. I tried some tunables for mpr spinup time and what else I could find but nothing.
Since there is no newer driver I could find, and the newest I could find (v20) is much older than the used (v23) I am not sure how to proceed.
I really hope that someone can point me in the right direction. Since it is my first post please forgive me if I missed to attach some necessary Information.