Mellanox Connect-X 3 mcx353a-fcbt not showing up in network tab

MetaCerberus

Cadet
Joined
Nov 13, 2023
Messages
2
First of all an introduction. Hi, I'm pretty much brand new to TrueNAS and the TrueNAS forums. My hardware configuration should be in my signature but in case its not I will reply to this post with the details and will be happy to provide any logs or additional information requested to help with my request. I have also searched the forums but seeing as Scale is still relatively new, I have yet to find anything that describes or fixes my issue. Now onto the issue at hand:

I recently just built a new server to serve as a backup for my current UnRaid server. While everything works fine, I decided I wanted a faster link between my main UnRaid server and my newly built TrueNAS Scale backup server. Luckily a friend of mine had 3 Mellanox ConnectX 3 mcx353a 40GbE network cards which he offered to me for free. After doing some research I came to the conclusion that these cards are kind of hit or miss in the community. Seeing as they are free however, I figured I would try my hand with them. After manually switching the port configuration from InfiniBand to Ethernet using the MFT in Windows, I installed them into the relevant servers. While UnRaid is able to detect and use the cards just fine, my TrueNAS system detects the cards but I cant see them under the network tab or by using
Code:
ifconfig -a
. I am absolutely sure they are configured in ethernet mode as when popping the cards into my UnRaid server or my Windows PC, they are all instantly recognized and properly configured. I've provided screenshots below of them showing up when using
Code:
lspci | grep Mellanox
as well as a screenshot of my network tab in the TrueNAS GUI. The listed NICs are all onboard 1GbE NICs. I would appreciate any help on the matter and as I stated before, I am willing to provide any additional requested information. Thank you.

lspci command and output:


Code:
root@truenas[~]# sudo lspci | grep Mellanox
05:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
81:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
 

Attachments

  • lspci.PNG
    lspci.PNG
    4.5 KB · Views: 56
  • network.PNG
    network.PNG
    475.5 KB · Views: 62

MetaCerberus

Cadet
Joined
Nov 13, 2023
Messages
2
issue seems to have resolved itself.
just in case anyone has a similar issue, I will retrace my steps even though they almost certainly didnt fix it and certainly dont recommend anyone else try it unless they know what they are doing since I looked through normally inaccessible directories.

01) check network tab and refresh to check for miss devices. they arent there.
02) set up a debian VM and notice the NICs are listed as available to be assigned. dont assign them.
03) finish setting up vm and check network tab. They still werent there.
04) modinfo mlx4_en
05) hwinfo | grep mellanox
06) cd into driver directory
07) nano driver
08) exit without saving
09) close ssh
10) reopen ssh
11) sudo dmesg | grep mlx
12) refresh network tab
 
Top