How to get a 10Gb SFP+ network interface with a PCI Express 3.0 x4 slot

[LaGrandePieuvre]

Hello,

I'm planning to build a networked storage system with TrueNas. I'm having a problem finding a SFP+ 10Gb network card compatible and reliable with TrueNas on a single PCI Express 3.0 X4 slot.

My configuration:
Based on a DELL OptiPlex 7060 SmallFormFactor ( Manufacturer's data sheet )

Processor: Intel i7 8700
Ram: 8 Go DDR4 2666MHz
Cache disk: M.2 NVME PCI 3.0 x4 P3 Crucial 1TO
System disk: Crucial BX500 480 GB 3D NAND
Storage disk: In process of purchase
Disk controller: LSI 9207-8i PCIe gen3 x8
Power supply: 200W

My question is: do you know of a network card with at least 1 SFP+ 10Gb port that fits into a PCI Express 3.0 x4 slot?


For my part, I have a back-up if I can't find a suitable card, but I need your expertise, folks. I was thinking of getting an Intel X520-DA1 board, as the model looks robust and is easily supported by TrueNas. This network card has a PCI Express 2.0 x8 connector. However, I only have one PCI Express 3.0 x4 slot. If I connect the card with a PCI Express 3.0 x4 to x8 adapter, I'll have a PCI Express 2.0 x4 connection at 2GB/s (i.e. 16Gb/s) which (i think ?), in terms of throughput, is more than enough to support a 10Gb SFP+ port. My processor can handle 16 PCI Express lanes, 4 of which are taken by the M.2 NVME, 8 by the disk controller, leaving me with 4 free lanes for the network interface.

Do you think this will work?
Will there be any loss of performance on the rest of my PCI Express hardware if I connect my network interface this way?
Do you have any advice on the PCI adapters mentioned above?


Here's my concern, I hope I've been clear enough, I'm new to this, please help me with kindness,
Have a nice day and thank you for your help.

 

[emerkle826]

I don't have much TrueNAS/FreeNAS experience yet, but I have an OptiPlex 7060 that I use with pfSense (FreeBSD based) and I put a Chelsio T520-CR in it and it works well. It is a dual SFP+ port card that fits into the PCIe x4 slot. I got the card off eBay (prices range from $50-80 USD).

One note, with the latest version of pfSense (2.7) I had to set a kernel Loader tunable to get it to work at full speeds. The info is here:

Hardware — Hardware Tuning and Troubleshooting | pfSense Documentation

docs.netgate.com

I image that being FreeBSD based, TrueNAS might need the same setting.

 

[LaGrandePieuvre]

Thanks for your reply, obviously the Chelsio T520-CR are not perfectly supported compared to Intel network cards, I think I rushed with this OptiPlex 7060, I'm going to sell it and opt for a machine supporting DDR4 ECC and with more PCI Express slots. However, if you have any experience of using a Chelsio T520-CR with TrueNas, I'd love to hear from you. Have a nice day!

 

[MrGuvernment]

Thanks for your reply, obviously the Chelsio T520-CR are not perfectly supported compared to Intel network cards, I think I rushed with this OptiPlex 7060, I'm going to sell it and opt for a machine supporting DDR4 ECC and with more PCI Express slots. However, if you have any experience of using a Chelsio T520-CR with TrueNas, I'd love to hear from you. Have a nice day!

Look into something like a HP Z4 workstation with a Xeon Silver, if you only need 4 SATA drives. Otherwise def look at xeon level workstation systems if you do not wish to go up to tower or rack servers yet.

 

[Etorix]

obviously the Chelsio T520-CR are not perfectly supported compared to Intel network cards,

Obviously, you have not fully done your research on recommended hardware.

10 Gig Networking Primer

This is a discussion of high speed networking for newcomers, with specific emphasis on practical small- to medium-scale deployments for home labs or small office users. It originated with a forum thread located here that has many pages of...

www.truenas.com

 

[artlessknave]

obviously the Chelsio T520-CR are not perfectly supported

these are absolutely fully supported. why do you think they are not?
chelsio, mellanox and intel should all work immediately and work well, though connectx2's (pcie2.0) are a bit old, and chelsio t3XX (pcie1.0) would be extra old. they will work run just fine though. connectx2 2 port can be often found for pretty cheap. I got a bunch, they let me do all kinds of server shenanigans.

additionally, you only need pcie3.0 x1 for 10gbe. pcie3.0 x4 will do 4x 10gbe. pcie2.0 x4 will get you 2x 10GBE just fine.

 

[LaGrandePieuvre]

Obviously, you have not fully done your research on recommended hardware.

10 Gig Networking Primer

This is a discussion of high speed networking for newcomers, with specific emphasis on practical small- to medium-scale deployments for home labs or small office users. It originated with a forum thread located here that has many pages of...

www.truenas.com

I'd been over this document before, obviously not enough times ahah! I got tangled up :)

 

[LaGrandePieuvre]

Look into something like a HP Z4 workstation with a Xeon Silver, if you only need 4 SATA drives. Otherwise def look at xeon level workstation systems if you do not wish to go up to tower or rack servers yet.

I was thinking of getting a DELL POWEREDGE T330, I'll have a look at HP, thanks for the advice.

 

[LaGrandePieuvre]

ceux-ci sont absolument entièrement pris en charge. pourquoi pensez-vous qu'ils ne le sont pas?
chelsio, mellanox et intel devraient tous fonctionner immédiatement et bien, bien que les connectx2 (pcie2.0) soient un peu vieux, et chelsio t3XX (pcie1.0) serait très vieux. ils fonctionneront très bien cependant. Le port connectx2 2 peut souvent être trouvé pour pas cher. J'en ai un tas, ils m'ont laissé faire toutes sortes de manigances de serveur.

de plus, vous n'avez besoin que de pcie3.0 x1 pour 10gbe. pcie3.0 x4 fera 4x 10gbe. pcie2.0 x4 vous donnera 2x 10GBE très bien.

Thanks for the advice, I'll go for the cheapest one that meets my criteria between these 3 brands then! Quick question, are there any SFP+ modules recommended for Chelsio network interfaces? Can I use any of them? Is there a limitation like on intel network cards that would force me to take modules specially designed for Chelsio? (I'm not an expert, so I may be talking nonsense, sorry!)

 

[artlessknave]

modules specially designed for Chelsio

nope. don't just get the cheapest offbrand sfps though.
I dont have an intel but I dont remember hearing about them having a restriction. many of those restrictions are like Ciscos: they will not troubleshoot unsupported hardware (huh, sounds like these forums!)

 

[MrGuvernment]

I have bought from fs.com mine sfp+ modules for my Brocade switch, going to intel x520's as well as an older HPE card and they have all worked great so far.