Truenas Scale installation support Mellanox ConnectX-4 Lx chip by default?

sinbeta

Cadet

Apr 23, 2023

Hi folks,
I'm a fresh man for Truenas scale. Could you pleae help me check if TrueNas scale installation can support Mellanox ConnectX-4 Lx chip by default or not?

artlessknave

Wizard

Apr 24, 2023

99.99999% should. x2's work just fine in core and they are ancient at this point.
I dont have one but I would be very surprised if it didn't.

(unrepentantly stolen from the Grinch)

Sorry to hear you're having trouble. Please take a few moments to review the Forum Rules, conveniently linked at the top of every page in red, and pay particular attention to the section on how to formulate a useful problem report, especially including a detailed description of your hardware.

Without this, You've basically given no one anything to work with, so the responses will tend to be random guesses rather than anything useful.
Forum Rules: https://www.ixsystems.com/community/threads/forum-rules.45124/
Resources: https://www.truenas.com/community/resources/

avoid raidz1(and RAID5); realtek is usually eWASTE; fake LSI/Intel cards suck; HBA not RAID; just say no to dedup; USB + ZFS = bad; cheap SATA = eWASTE; Recommended Hardware; Power supply FTW; SAS primer; SMR bad; WD RED BAD; FIND SMR GOOD; Flash that firmware, LSI; Jargon; virtualize right; SLOG is not a cache; give SLOG and L2ARC a pass; [d]RAID(z) is not a backup; gaming =/= server

This list is WWAAYY too long now. I might have a problem. it's not even half complete anymore. halp!

supermicro CSE836 3U 16x 3.5" (no model number on it, was a nescout with an x8dth)
Redundant 920W
SUPERMICRO MBD-X11SSM-F-O
E3-1230 v5 SkyLake
64GB (2x Crucial 32GB Kit [2 x 16GB] DDR4-2400 ECC)
chelsio T520-CR
TrueNAS-13-U36.1

sudo mprutil show adapters; sudo cat /var/log/dmesg* | grep Firmware | uniq
Device Name Chip Name Board Name Firmware
/dev/mpr0 LSISAS3224 SAS9305-24i 09006400
mpr0: Firmware: 09.00.100.00, Driver: 18.03.00.00-fbsd

sudo mprutil show adapter

mpr0 Adapter:
Board Name: SAS9305-24i
Board Assembly: 03-25699-02004
Chip Name: LSISAS3224
Chip Revision: A1
BIOS Revision: 8.27.00.00
Firmware Revision: 9.00.100.00
Integrated RAID: no
SATA NCQ: ENABLED
PCIe Width/Speed: x0 (8.0 GB/sec)
IOC Speed: Full
Temperature: 54 C

12x WD+seagate NAS/ENT drives (raidz3+spare) [varying sizes over 4TB]
4x 2.5" SSD misc
4x 2.5" HDD jails

2x 120GB Kingston SSD

isilon nl108/IQ108NL (modified, barely not standard UIO EATX size. buggers)
36 2.5" bays but not all in use

Redundant 1000W
SUPERMICRO X9DBU-F
2x E5-2430L v2
64 GB samsung RAM
chelsio t320
TrueNAS-13-U6.1

sudo mpsutil show adapters; sudo dmesg | grep Firm| uniq; sudo dmesg | grep ses0 | grep Fix | uniq
Device Name Chip Name Board Name Firmware
/dev/mps0 LSISAS2008 SMC2008-IT 14000700
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
cxgbc0: Firmware Version 7.12.0
ses0: <Isilon SAS_EXP 0 7156C 1024> Fixed Enclosure Services SCSI-3 device

sudo mpsutil show adapter

mps0 Adapter:
Board Name: SMC2008-IT
Board Assembly: 050-0034-02
Chip Name: LSISAS2008
Chip Revision: ALL:
BIOS Revision: 7.39.02.00
Firmware Revision: 20.00.07.00
Integrated RAID: no
SATA NCQ: DISABLED
PCIe Width/Speed: x8 (5.0 GB/sec)
IOC Speed: Full
Temperature: Unknown/Unsupported

24x 2TB 2xraiz2 mixed old drives (slowly being replaced)

boot: 2x 120GB KINGSTON SA400S3

Quanta LB6M
Quanta LB4M

dont remember, nearly identical to ids2 MITX chassis.
~450W SFX
A1SRi-2758F
32GB 4x Kingston 8GB 204-Pin DDR3 SO-DIMM ECC Unbuffered DDR3 1600 (PC3 12800) Server Memory Model KVR16LSE11/8
pfsense 2.5.2-RELEASE
Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller

Rosewill RS-MI-01 BK - Mini ITX Tower
~450W SFX
A1SRi-2758F
32GB 4x Samsung 8GB 204-Pin DDR3 SO-DIMM ECC Unbuffered DDR3 1600 (PC3 12800)
pfsense 2.5.2-RELEASE
Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller

U-NAS NSC-810A
FLEX/1U 500W? not sure, no access.
SUPERMICRO MBD-X11SSM-F-O
Pentium g4620
32GB (2x16GB) DDR4 ECC)
TrueNAS-13-U3.1

SATA controller: Intel Corporation Sunrise Point-H SATA controller [AHCI mode]

6x Ironwolf + 2x Barracuda green 2TB (mirror)

3x 32GB SanDisk Cruzer Fit

SUPERMICRO CSE-721TQ-250B
FLEX/1U probably ~250W?
TrueNAS-13-U3.1

Forum Rules
https://www.ixsystems.com/community/threads/forum-rules.45124/
Chris Moore's Super Duper Useful Links (™):
https://forums.freenas.org/index.php?resources/links-to-useful-threads.108
an into to ZFS
https://www.ixsystems.com/community/resources/introduction-to-zfs.111/
ZFS Storage Pool Layout
https://www.truenas.com/community/resources/zfs-storage-pool-layout.201/
Power.
https://www.truenas.com/community/threads/proper-power-supply-sizing-guidance.38811/
migrate to new pool
https://www.ixsystems.com/community/threads/howto-migrate-data-from-one-pool-to-a-bigger-pool.40519/
forum resources
https://www.truenas.com/community/resources/categories/hardware.1/

disklist.pl
https://github.com/nephri/FreeNas-DiskList
I highly recommend, and use, disklist.pl. it shows all the disks in one nice table (its basically 8 commands in one)
(this line requires root)

wget https://raw.githubusercontent.com/nephri/FreeNas-DiskList/master/disklist.pl -O /usr/bin/disklist 2> /dev/null 1> /dev/null;chmod 755 /usr/bin/disklist

then you can:sudo disklist
or for everything:sudo disklist -smartctl -all
or CSV: sudo disklist -o:csv -smartctl -all | awk -F\; '{$1=$1} 1' OFS=, (the author thinks a CSV is semi-colon separated for some reason *shrugs*)
the webUI has improved but it still comes nowhere close the same, concise, layout of info that disklist.pl does.
I find it makes it so much easier to see which gptid is which /dev/ is which serial, and smart info, poweronhours, temp all in once is so convenient.

zfsprune
https://github.com/bahamas10/zfs-prune-snapshots

currently no content

[_SPOILER="put a title here and remove the 2 underscores"]
some text goes here, like, describing how you can nest spoilers to make a convoluted and confusing signature like mine that you have to do a university course in recursive tags every time you want to update the damn thing
also, in the ellipses (3 dots) format section there is a "show" option which will build the spoiler for you based on the text you highlighted.
[_/SPOILER]

LarsR

Guru

Apr 25, 2023

I can only speak for x3's but mine worked out of the box.

Important Announcement for the TrueNAS Community.

Truenas Scale installation support Mellanox ConnectX-4 Lx chip by default?

sinbeta

Cadet

artlessknave

Wizard

LarsR

Guru

Similar threads

Important Announcement for the TrueNAS Community.

Truenas Scale installation support Mellanox ConnectX-4 Lx chip by default?

sinbeta

Cadet

artlessknave

Wizard

LarsR

Guru

Important Announcement for the TrueNAS Community.

Related topics on forums.truenas.com for thread: "Truenas Scale installation support Mellanox ConnectX-4 Lx chip by default?"

Similar threads