NIC options for 2.5GbE?

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
6Ghz suffers from many of the same issues as 5, Ie less range compared to the more congested 2.4 but theoretically higher throughput, IF you have a good line of sight from the AP to the antenna. Too many of these APs get tested as if the users computer will always sit next to the AP, with a clear LOS.

We can theorize about the speeds that these APs can get but ultimately many of them only function that well if you’re for all intents and purposes enjoying a wired connection. Works well enough in some environments (open plan office with the APs mounted to the ceiling) but not as great in most residences.

the big benefit of these newer systems is the beam forming, multi-antenna MIMO, and better resistance against interference from fellow APs. Not like the “good ole days” when we effectively had three useable wireless channels. I’m not upgrading my airport extremes anytime soon, the old ones work well enough and if I need speed I just plug in my fiber-thunderbolt adapter.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
One reason for these, were newer WiFi standards that have a theoretical throughput of greater than 1Gbps, PER band.

"Theoretical" being the operative word. And it's mostly bullchips anyways. 802.11ac wave 2 already had a theoretical throughput of something like 2.6Gbps.

And we now have 6Ghz band too. So, if all are bursting at full speed to / from wired devices, they are limited to the wired connection that the WiFi access point has configured. In the past, that was 1Gbps Ethernet.

Newer WiFi access points are starting to include 2.5Gbps Ethernet ports because of those wireless speed improvements.

That's not really what's going on here.

Someone had an "aha" moment and realized that the theoretical max speed of 6 and 6E exceeded the 1Gbps speed of GbE. That's an obvious fact, but it was also an obvious fact for 802.11ac, yet there you see 1G uplinks, because you generally don't get theoretical speeds out of real world deployments.

The difference here is that the 1Gbps switch market has been a race to the bottom. I've still got early 1G switchgear around that cost five figures, yet you can buy an $80 switch these days that does more and better.

Vendors are finding that speeds above 10G are not compelling to the end user market, and even 10G copper has been a rough sell, given that reasonable products were available nearly a decade ago, but there just wasn't a lot of uptake. Certainly some of that had to do with Cat6A/Cat7, but the other part is that gigabit hit the point of sufficiency for most general networking needs.

That last statement ALSO applies to wifi; there just isn't a lot of need for in-excess-of 1G speeds. All of you geeks who are taking exception to that are not the average use case.

But it did provide an opportunity to present a new argument: "let's upgrade the existing switches out there." This results in new sales that would not otherwise have appeared. It is largely a con. There will indeed be some sites that benefit from 2.5G or 5G ethernet for their wifi, but for the most part, these ports will not routinely break 1G speeds. The vast majority of this revolves around convincing people to buy new gear that they generally don't really need. This is great for vendors but not so great for the world's electronics recycling.

And @Arwen's post puts the errant logic on display excellently; "replace the switch first, and then replace the WiFi access points" is exactly backwards. What you really want to do is to replace the access points, and then, if you find you are slagging out an uplink, you can then use that to justify replacing the switch.

What's actually going to happen is that there's going to be a lot of short lifecycle switchgear as the "new" stuff rapidly becomes obsolete, as the next wifi (802.11be) standard pushes out to even faster speeds, possibly as fast as 40Gbps.

So what I'm saying is that the switchgear that the shysters sell you for "WiFi 6" isn't going to survive WiFi 7. Most people will be better off just chucking the WiFi 6 stuff onto their existing gigabit networks, unless you're a site with a mad amount of high bandwidth wifi clients.
 

Arwen

MVP
Joined
May 17, 2014
Messages
3,600
...

And @Arwen's post puts the errant logic on display excellently; "replace the switch first, and then replace the WiFi access points" is exactly backwards. What you really want to do is to replace the access points, and then, if you find you are slagging out an uplink, you can then use that to justify replacing the switch.

...
Most of the rest of your reply is good.

However, even if the far end can't handle >1Gbps, (thus, the WiFi access point does not need more bandwidth), latency and other factors apply.

Back in 2016 the company I worked for, was migrating contractors to an out source company, from out source company own laptops, (with dedicated wired 1Gbps Ethernet ports), to contract company owned laptops, with WiFi. Somewhat bizarre. This meant perhaps 100 people on my building's floor would move from wired to WiFi.

Around the same time frame the building's WiFi access points in the ceiling were changed out. That took weeks, because it was probably >30 minutes per access point. There were probably 20 access points on my floor, and another 20 on the floor below. So, that is where I got the idea that the switch should be changed out first, (if the new access points supported >1Gbps Ethernet). Trivial to change out the switch, compared to the time it took to change out all the WiFi access points.

And to be clear, some of us NEEDED the bandwidth. One time I needed to download a specific OS ISO image from the vendor to my laptop, (the servers I supported did not have direct Internet access). Then copy it to the production network. It was literally taking hours. Even though it was still using the wired connection. So I gave up and figured I'd finish it over night from home. Whence home, my Google Fiber 1Gbps wired connection finished the task in minutes.

Doing that over WiFi, (though a VPN, even at the office), sounds like it would be quite frustrating.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
Wireless can be a huge cost saver in a large open plan office. Far fewer cables have to be pulled, cubes and desks can be shrunk, more people can be squeezed into the same space. The CFO is happy.

but it also reminds me of a former employer who had a grand total of three T1 lines to serve the world HQ with over 1000 local employees. There are costs to this behavior that don’t show up in spreadsheets. Productivity will suffer.

The BYOD trend is another wrinkle. Everyone thinks nothing of streaming content etc while using the company WiFi and then complain that the network is slow. I had that issue at the preschool where I used to work and it really came to a head when teachers started abusing the network during nap time.

I finally resorted to QOS though that only works to a limited extent unless you go through the trouble of individually registering MAC addresses also. That’s because Apple devices have this really annoying habit of sharing network login credentials with each other. So the private, official-use-only network would then also experience a large number of BYODs. Argh.
 

QonoS

Explorer
Joined
Apr 1, 2021
Messages
87
Follow up, as of 12u4, TrueNAS can support the RTL8125 chipset as long as you set two rc.conf tuneables. Not sure why ixSystems announced RTL8125 support yet still requires the two tuneables to be set but such is life. See here.
Thanks for the update. I tried this method and it does not do well with a "Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 04)".
It runs fine with Win10, Ubuntu and drivers from realtek.com but performance is bad when using TrueNAS 12.0-U4 and the described method.

Win 10 21H1 <> USB-C <> RTL8156B <> RJ45 <<<..cable..>>> RJ45 <> RTL8125B <> PCIe 2.0 x1 <> Ubuntu 20.04 HWE 5.8
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec sender
[ 4] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec receiver

Win 10 21H1 <> USB-C <> RTL8156B <> RJ45 <<<..cable..>>> RJ45 <> RTL8125B <> PCIe 2.0 x1 <> TrueNAS 12.0-U4
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.01 sec 333 MBytes 279 Mbits/sec sender
[ 4] 0.00-10.01 sec 333 MBytes 279 Mbits/sec receiver
 

rvassar

Guru
Joined
May 2, 2018
Messages
971
Apple devices have this really annoying habit of sharing network login credentials with each other.

With each other? Mine shared my WiFi password with my pickup truck! :rolleyes:
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
The real meaning of promiscuous mode…

… as for the bad RTL8125 driver performance just on TrueNAS, I’m mystified. That’s not even 1Gb/s performance. Is Ubuntu and TrueNAS running on the same hardware or are we comparing different motherboards too?

if it’s the latter, you may want to investigate whether the BIOS has a sleep mode enabled. This has also bit others who had crummy SLOG performance.
 
Last edited:

QonoS

Explorer
Joined
Apr 1, 2021
Messages
87
… as for the bad RTL8125 driver performance just on TrueNAS, I’m mystified. That’s not even 1Gb/s performance. Is Ubuntu and TrueNAS running on the same hardware or are we comparing different motherboards too?
Hardware, BIOS/UEFI settings, etc. everything is identical. I've just replaced Ubuntu with TrueNAS for the test.
Link speed properly negotiated to 2500Base-T.
Maybe I find time for more thorough testing later...
 

idgar

Dabbler
Joined
Feb 4, 2022
Messages
18
Bumping this thread to get more related data for 2023.
I want to get faster connectivity to the storage, especially for one client, so for now I plan to directly connect the 2 computers without switch.
My motherboard has only PCIe 2.0 x1 frer slots (a slot of x16 is being occupied by SAS card).
With the free slots throughput limitation and lanes (0.5GB/s) , the only relevant NIC is 2.5Gbit (also true for USB3), although 5Gbit can also be highly utilized but will not be able to reach 100%, and nevertheless I believe it will be hard to find 5Gbit with PCIe x1.

Are there a supported / recommended 2.5Gbit NICs these days for TrueNAS core 12/13?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
No, and do not expect one anytime soon. Realtek is as awful as always and Intel is struggling to get theirs to work properly.
If you want/need something faster than gigabit, the only sane option is 10GbE.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
If you want/need something faster than gigabit, the only sane option is 10GbE.
Which isn't an option for an x1 slot (PCIe v2 or v3)... usually x8
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Which isn't an option for an x1 slot (PCIe v2 or v3)... usually x8
Well, that's where the world of crazy adapters shines. Assuming the available slot is Full-Profile, I'm sure someone makes a Lalf-Profile riser that converts the slot into an open-ended one. Or provides a full x16 with extra power input.
Alternatively, things get even crazier more fun if you throw in ribbon cable adapters. Or adapters that use USB 3.0 cables. If there's a silver lining to all the cryptocurrency nonsense, it's that the market for weird PCIe adapters grew a lot and there's surely something out there that will satisfy OP's needs.
Hell, if the board supports bifurcating the x16, it could be split into two LP x8 slots.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
(Sidenote: There's a story of the team at Microsoft responsible for plug-in-play playing around with the longest chain of PCI adapters they could throw at their test machine and still have it boot. PCI, PCI-X and PCIe have always allowed for crazy contraptions.)
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
if the board supports bifurcating the x16, it could be split into two LP x8 slots.
I guess this would be the most practically useful advice to investigate... except that I think bifurcation only really arrived late in the PCIe v3 chipset controllers, so probably won't help.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
It's always been a thing, it's just been typically controlled fully automatically by the system firmware based on detecting installed cards. The specifics will vary with the system...
 

idgar

Dabbler
Joined
Feb 4, 2022
Messages
18
From all I read here it sounds like I should consider aggregating 1Gbit ports if I want a simple (somehow) solution. I'm just not sure if TrueNAS supports a link aggregation that can utilize all links for one session (like splitting the packets by some ID that is not MAC/IP).

But just to make sure, how complicated is the PCIe slot splitter?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
it sounds like I should consider aggregating 1Gbit ports if I want a simple (somehow) solution.
Depends what your expectation is... LAGG of 4 1Gbps NICs doesn't mean one client can get 4Gbps (actually at most 1Gbps for any one client).

I'm just not sure if TrueNAS supports a link aggregation that can utilize all links for one session (like splitting the packets by some ID that is not MAC/IP)
It can do LACP, but as mentioned, that doesn't help to create a fake 4Gbits interface for a single client to use.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
From all I read here it sounds like I should consider aggregating 1Gbit ports
Not unless you're a masochist and actively looking for pain with no gain associated.
I'm just not sure if TrueNAS supports a link aggregation that can utilize all links for one session (like splitting the packets by some ID that is not MAC/IP).
There's no such magical beast. SMB sort of supports something like it, but it's finicky to setup, linked to some data corruption and generally of limited usefulness.
But just to make sure, how complicated is the PCIe slot splitter?
In what way? It's dead simple, it either works or it doesn't. You can get extra-fancy and buy one that includes a PCIe switch and pretty much guaranteed to work, but at that point it'd be cheaper to replace your whole machine.

As for the adapters to make use of the x1 slot, there's tons of them out there, pick whatever form factor works for you and give it a try.
 
Top