Best nic for FreeNAS that supports 2.5Gbps

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
I am buying a 2.5Gbps switch as I need more speed between my NAS and clients. I wanted to ask what people recommend for a 2.5Gbps NIC for FreeNAS? I have no issue buying a 10Gbps Nic if it supports 2.5Gbps.. just need something that works well and preferably out of the box. I saw this thread https://www.truenas.com/community/threads/realtek-rtl8125-2-5gbps-ethernet.70485/ about running a realtek nic but realtek nics generally aren't that good and it was a bit of a hack to get it to work... Curious what others are doing?
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193

jenksdrummer

Patron
Joined
Jun 7, 2011
Messages
250
I've used the 10G-Tek version of the x540-T2, works well and I've not had problems; though I'm on a 10G network; not sure about 2.5G negotiation.

 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
I've used the 10G-Tek version of the x540-T2, works well and I've not had problems; though I'm on a 10G network; not sure about 2.5G negotiation.


According to Intel the 540 chip only does 100/1000/10000 I believe...
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Well I just installed the nic, the switch is detecting it and auto negotiated 1Gbps (which it should, don't have the 2.5Gbps switch yet). But FreeNAS doesn't seem to be identifying it. Its a Intel x550 chip, which I thought was supported by freenas?? Any thoughts on what might be going on? Its not even showing up in a pciconf -lv.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
False Alarm :) Turns out I put the card in a PCI 2.0 vs PCI 3.0 slot... I thought this Mobo had both PCI 3.0 but upon a second look at the specs, one was PCI 2.0... FreeNAS now detects the card.
 

no_connection

Patron
Joined
Dec 15, 2013
Messages
480
It should not matter what gen PCIe slot it is, it's all backwards compatible.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Your right, it technically shouldn’t matter.. not quite sure why it didn’t work though.. not a big deal as I don’t have anything else to put in the gen 3 slot so I can just leave it there.. I leaned something new in the process though, never knew how forward and backward compatible all those slots were!

my new switch should arrive Monday then I can really see if the mic supports 2.5Gbps, technically it should according to Intel. Fingers crossed and can’t wait for 1.5X increase in performance! Plus they will be future proof for 10Gbps in the future, maybe in the next 10 years lol, network hardware prices are the slowest to drop out of any component for a PC it seems.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Well the results are in, I got the Unifi 6 24 Switch which includes 12 2.5Gbps ports. I have everything hooked up now, the x550 Intel startech.com card in my Freenas (Little) backup server, I wanted to test it out there first, have a second Intel nic coming for my main Freenas server tomorrow. I have my PC using a USB-C Realtek 2.5Gbps ethernet adapter in Windows. Everything is working great, the switch recognizes both adapters as 2.5Gbps. Everything was left as Auto-negotiation as well. I think the Freenas box needed a reboot after plugging into the 2.5Gbps switch port as I couldn't manually get it to update its speed, that was really the only hiccup in the process. The picture speaks for itself, 282MB/s! Anyone looking for a solid 2.5Gbps setup, this worked great and out of the box, no special drivers needed!
 

Attachments

  • 2.5gbps.jpg
    2.5gbps.jpg
    54.3 KB · Views: 1,004

jayecin

Explorer
Joined
Oct 12, 2020
Messages
79
Really tempting me to pull the trigger on my 10gbps interface build...
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Well I got the other Intel Nic installed and working great as a single nic.. but when I tried to do link aggregation I could not get the nic to talk to the switch, I tried static IP's and dynamic. I know the switch can do link aggregation as it was doing it before with the other 1Gbps nic's I had so not sure why the intel nic wasn't working... any thoughts?
 

jayecin

Explorer
Joined
Oct 12, 2020
Messages
79
Link Aggregation isnt really a well put together solution on FreeNas. I attempted to do it with my 4 port Intel NIC to my enterprise Juniper switch and it was causing some mac flapping issues. For the record I am 12+ year network engineer who does this for a living. Basically dont bother with it.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
That’s interesting, I have had no issues in the past, but they were all built in nics on mobo’s so not sure if that really matters... I mean 2.5Gbps is nice now, but always nice to have access to more if multiple people are accessing.. I’ll probably play around a little more tomorrow with it but I feel like your probably right in this case.
 

jayecin

Explorer
Joined
Oct 12, 2020
Messages
79
It could be the hardware, ive definitely seen a lot of situations where LAG has vendor specific issues. It just started to feel like more work than its worth type situation, where I would spend more time troubleshooting and trying to get something to work and dealing with little quirks. Id rather just get a 10gbps NIC and call it a day.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Might have to get a 10GBPS switch to augment the 2.5 one ;). I have all UniFi gear and have been hoping they would be coming out with some newer 10G gear as the stuff they have now is a bit older. But yeah I won’t buy another nic if it truly is that.. although I have two servers each with one new x550 10G nic so could test separate nics and see if that works then I’d know for sure these nics with two ports really can’t be used for aggregation.
 

no_connection

Patron
Joined
Dec 15, 2013
Messages
480
Now I like Mikrotik more than UniFi for switches for more than one reason. 10sec upgrade and reboot for upgrade each switch is pretty neat. Dual PSU (PoE and adapter). Web UI is a bit "odd" on some parts tho but not too bad and it's not that often you need to set it up anyway. I think the RJ45 plugs have 2.5G even if it's not printed on them but I would ask them first. Make sure you get newer generations of SFP to RJ45 with the lower power draw else you could melt something. Especially with fanless designs.
CRS309-1G-8S+IN or 4 port model (would stick a fan onto that one) would be pretty cheap option right?

If you need more speed I would look at multipath rather than aggregation.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Well I have given up getting LACP working on a Single NIC with Two Ethernet Ports like the X550 nic. That said, I decided to just use the loadbalance aggregation option in Freenas as it really accomplishes the same thing for me. I tested it on my backup server first and was able to saturate both ports when copying files to two separate PC's on the network which is exactly what I wanted... With both me and my wife using the NAS as our primary storage for photo/video editing both of us having access to 2.5Gbps is critical, so very happy with the upgrade and the fact I can still use the loadbalance link aggregation option in FreeNAS to keep my speeds maxed out!
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Well looks like I spoke a little too soon perhaps on the 2.5Gbps front.. I don't think this is related to the nic but ill explain what is happening...

On my freenas backup server I can easily hit 280MB/s reading and writing files all day.
If I do a zpool iostat -v while copying I can see all disks are reading at about 40MB/s (7 disks in this one, raidz1)

On my freenas main server I struggle to get over 1Gbps speeds some files seems to do better then others which is the weird part.
If I do a zpool iostat -v while copying I see a lot of fluctuation seems completely random disks going from 8MB/s to 30MB/s typically two disks will remain a little more consistent and greater then 20MB/s, (6 disks in this one, raidz2)

Not sure what is happening exactly, the freenas main server is a full blow server with 2 Intel CPUs 96GB ram, etc... it should be more the capable of handling this load, the disk drivers are also brand new 7200rpm disk 14tb each.. Not sure how my older backup server is able to be rock solid at 2.5Gbps but the main server is struggling now..
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Also just did a test on the main nas to test disk speeds,

root@freenas:~ # dd if=/dev/zero of=/mnt/v01/Data/Temp/testfile bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 16.315533 secs (2570742821 bytes/sec)

Or 2500MB/s... So the disks can more then handle the speed I believe...


Also when looking at the zpool iostat of each server I see the following

Backup Server:
pool alloc free read write read write
---------------------------------------------- ----- ----- ----- ----- ----- -----
freenas-boot 21.4G 8.05G 0 0 0 0
da7p2 21.4G 8.05G 0 0 0 0
---------------------------------------------- ----- ----- ----- ----- ----- -----
v01 22.6T 28.2T 8.53K 0 303M 0
raidz1 22.6T 28.2T 8.53K 0 303M 0

Main Server:
pool alloc free read write read write
---------------------------------------------- ----- ----- ----- ----- ----- -----
freenas-boot 20.0G 9.27G 0 0 0 0
mirror 20.0G 9.27G 0 0 0 0
ada0p2 - - 0 0 0 0
da6p2 - - 0 0 0 0
---------------------------------------------- ----- ----- ----- ----- ----- -----
v01 37.3T 38.9T 1.92K 325 137M 3.08M
raidz2 37.3T 38.9T 1.92K 325 137M 3.08M


It looks like the read operations on the backup server are around 8000, while the main server only gets about 2000... Not sure why its so much less on the main server??
 
Last edited:

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Also, write speeds are pegged at 283 MB/s... So write performance is right on the mark and pretty consistent (sometimes it does drop to just above 200MB/s)... But the reads are fairly consistently slow :(
 
Last edited:
Top