Rosewill RC-20001 2.5 GB NIC use RTL8125 driver on TrueNAS 12.0-U1

[davidh3f]

I got a pair of Rosewill RC-20001 2.5 GB NIC. I got it work fair well in Ubuntu 12.10 desktop after updated the kernel to version 5.9. However, I still cannot get it fully work in TrueNAS 12.0-U1. Once I followed the steps described here: https://forums.serverbuilds.net/t/g...2-4-5-2-5-0-opnsense-use-2-5gb-realtek/3555/8, TrueNAS can recognize the card and and somewhat functional. However, if I transfer large files, the transfer will freeze. I don't know if anyone has worked out the RTL8125 driver issue.

I've searched this forum before posting this note. Apparently, I couldn't find a real answer so far from what's posted in this forum.

Thanks all for your help.

 

[Kris Moore]

I don't believe that card will be very well supported on TrueNAS CORE due to the FreeBSD drivers being different from what Linux Kernel 5.9 ships. However TrueNAS SCALE does include Linux Kernel 5.9 and if it supports your use-case it may be the way to go on this particular hardware. It is on its 2nd Alpha release, but a lot of the core file sharing functionality is very stable already, if you're open to being an early adopter :)

Download TrueNAS SCALE - Data Storage Software

Download TrueNAS SCALE - the data storage software that helps you store and access your data with ease. Add free apps to expand NAS capabilities.

www.truenas.com

 

[davidh3f]

Thanks Kris for your comment.

I gave it a try by following a few online posts and now are able to get this card to work (somewhat, will explain below).

My TrueNAS is up-to-date, version 12.0-U1.1. FreeBSD version is 12.2-RELEASE-p2.

I installed FreeBSD-12.2-RELEASE-amd64-dvd1.iso (here https://download.freebsd.org/ftp/re...AGES/12.2/FreeBSD-12.2-RELEASE-amd64-dvd1.iso) on a Virtualbox in a Ubuntu 12.10 Desktop by following this: https://linuxhint.com/install_freebsd_virtualbox/. I then compiled the RTL8125 driver, version 1.96.04, by following this: https://www.embedded-ideas.de/2020/08/08/freenas-on-odroid-h2-rtl8125b-driver-installation/.

After getting the newly compiled driver loaded by adding the line if_re_load="YES" in /boot/loader.conf, The TrueNAS box detected the NIC flawlessly (I am using the QNAP 2.5GB switch), and showed a correct interface at 2500Base-T.

However, for some reason, iperf3 test only gave me about 1.45 Gbits/sec. I successfully transferred a 20GB file between my Ubuntu desktop (installed with the same Rosewill 2.5 GB NIC and got it work in Kernel 5.9 as mentioned above) and the TrueNAS box via NFS share and the transferring speed was indeed about 1.45 Gbits/sec only, but at least it was relatively stable (without huge fluctuation or even freezing that I experienced in the past using a driver on the internet that someone else posted).

I don't know why the speed is still so low; but I will keep the settings this way for a few days and see if it is stable enough.

I will report back after a few days, but at the meantime, it would be appreciated if anyone can guide me to a direction to improve the transferring speed.

Thanks.

 

[davidh3f]

As it turns out, the bottleneck of the network speed is resulted from the CPU. I have dual Xeon E5440 CPUs. Once the file transferring speed reached to 1.60 Gbit/sec, a single thread reaches to 95% usage and the rest of the threads (7 of them) stayed at 20-30%. Temperature throttle probably set in at this point and the usage of this particular thread dropped to 20-30%, and the file transferring speed dropped propotionaly. Then the single thread usage went back up, the speed picked up again. During a 20GB file transfer, this pattern repeats, which caused speed fluctuation.

My question is, how come other threads of the CPU were not used in file transfer via NFS?

Thanks anyone can offer some ideas!

 

[davidh3f]

After almost a month of trying all different kind of things, here I want to update everyone my experience (not totally working yet, mind you):
1. I suspected that my old Xeon CPU has a bottleneck (also with only PCIe gen 1, call this as Server A), so I installed TrueNAS on a new server (call this as Server B) with much newer hardware (Xeon E3-1225 v3 and faster PCIe bus, etc.). Now, iperf speed reaches 2.35 Gbit/sec! However, if I transfer a real large 20 GB file, the speed could not sustain and was jumpy all over the places and sometimes crashes TrueNAS. I suspected that there was a driver issue.
2. I know the RTL8125 driver works perfectly fine in my Ubuntu 20.10 desktop with a kernel version 5.9.0. So I installed Ubuntu server 20.10 on Server B and then updated the kernel from the default 5.8 version to 5.9 version. To my dismay, somehow the NIC was not detected at all! It seems to me that Ubuntu server version, even after upgraded to a newer kernel, the RTL8125 driver was not included. I just can't figure out what else the reason could be.
3. I then installed Arch linux on Server B, which comes with a new kernel version at 5.10. The NIC was detected without any problem and iperf speed reached to 2.35 Gbit/sec. Again, I transferred a 20 GB file via NFS between my Ubuntu 20.10 desktop and Arch linux box. The transfer crashed at a very high transferring speed, may be 250 MB/sec. Well, another dead end.
4. I then installed unraid 6.9 RC-1 on Server B, since I know unraid 6.9 version uses kernel 5.10 which includes the RTL8125 driver. This has been the most successful experience. With this unraid box, I had a 128 GB SSD cache drive. During the first two transfers of this 20 GB file, the transfer speed sustained at 250 MB/sec! I mean the network interface monitoring tool showed a strict straight line at 250 MB/sec during the entire transfer. This has never happened in all my earlier experience. So, now I know that 1) my Server B hardware can support this speed and sustain it; 2) the RTL8125 driver included in unraid 6.9 RC-1 worked perfectly. Unfortunately, future file transfers could not repeat the same success. What happened was the transfer always started at high speed at 250 MB/sec, then dropped to miserable speed at 80-90 MB/sec. I suspected that it was because my cache drive was filled up and the spinning drive speed was too slow even using a two parity drives configuration. Somehow, unraid disk management using raid technology is just so inferior to TrueNAS' ZFS file system. This is what I concluded; but unraid's linux kernel support is vastly superior, per my own priorities, of course.
5. Yesterday, I found a new youtube video explained the driver compiling process for RTL8125 in BSD, which is a bit different than the method I used as described above in the first post. I then recompiled the driver and put it in Server A again. iperf speed remains at 1.45 Gbit/sec. File transfer speed remains a bit jumpy, ranged between 80-160 MB/sec. However, it feels better than the first version I compiled and it has not crashed my TrueNAS box yet. Overall, it provided 20% faster speed than my 1 Gbit NIC card. So, I am staying with this configuration for now.

At the meantime, I am putting together a new Server with newer hardware, and will try the above again. I will report back again, probably in another month.

 

[sretalla]

At the meantime, I am putting together a new Server with newer hardware, and will try the above again. I will report back again, probably in another month.

It seems to me that you completely ignored the feedback from Kris Moore and don't intend to stop going down this dead-end on FreeBSD-based TrueNAS... what was wrong with using TrueNAS SCALE as suggested? If you're playing with untested hardware and compiling your own drivers, why not be equally open to new software?

 

[davidh3f]

Thanks scretalla for the feedback. I always have an impression (maybe mistakenly?) that TrueNAS SCALE is for enterprise and will cost quite a bit. What I described here is just a personal hobby and I have no ambition to "achieve anything". This is why I did not pursue Kris' suggestion.

At the end of the day, if I can't get the 2.5 Gb NIC work on TrueNAS core, so be it. I will live with the 1 Gb card, which is not the end of the day for me. At the meantime, if I learned anything through this process, I can share my learning here in case others can benefit from my experience.

That's all. Hope I adequately explained my "motivation". :)

 

[sretalla]

that TrueNAS SCALE is for enterprise and will cost quite a bit.

Not so. It's just as free as TrueNAS CORE. There may be additional options that cost somewhere down the track, but the basics (probably all you need) will remain free.

An Enterprise version (with enterprise support) will perhaps come with some new Hardware from iX, just like for TrueNAS Enterprise now.

 

[davidh3f]

I didn't know that! sretalla, much appreciated for offering me this knowledge. I will definitely look into the "SCALE" version now. I am in your debt. :)

 

[davidh3f]

Want to have another update for everyone. I installed TrueNAS Scale on Server B. The RLT8125 2.5 Gb NIC card was detected right away. However, file transfer speed test showed it momentarily reached to 270 MB/sec and then dropped to zero, and then back up to high speed. In a way, the file transfer speed cannot sustain, like many other times that I tried as described in my earlier posts. The onboard 1 Gb NIC was totally fine and sustained the transfer speed at 117 MB/sec for the entire 20 GB file copy. So I now believe it is either the 2.5 Gb NIC hardware, or its linux driver, or the combination of both are just not tuned mature enough for this kind of utility. I think I've exhausted all my attempts and will stay with the mature 1 Gb NIC for the near future. Thanks all the the help.

 

[Lars of NM]

After almost a month of trying all different kind of things, here I want to update everyone my experience (not totally working yet, mind you):
1. I suspected that my old Xeon CPU has a bottleneck (also with only PCIe gen 1, call this as Server A),

The Rosewill RC-20001 2.2Gbe (Realtek RTL1825) NIC requires a PCIe 2.1 slot. I think your problem is that your "Server A" has a PCIe gen 1 slot which is hampering its performance.
I am using a pair of these cards with the Qnap 2.5Gbe switch (~$110) between my computer and my WHS 2011 box and it works great. I get full speed disk transfers of 210MB/s. I don't know what the true top speed is yet but it is higher than my hard drives, it is consistent with no drop outs. You can't beat them for $21/delivered. I have been looking for NAS software to replace my old WHS 2011 box but haven't found one that supports the RealTek RTL1525 chipset yet. On my windows machines I just plugged them in and they worked.

Question: Does TrueNAS Scale support the RTL1825 NIC or did it require compiling and loading a driver?