TrueNAS SCALE.. good iperf but <2MBytes/s transfers on gigabit network

[nooobz]

Hey so i have a problem Ive been trying to tackle for a few months now i like to read all i can before posting which i never do and im sure its pretty obvious what im doing wrong.
I am a complete novice when it comes to networking things so sorry in advanced.
So basically i have small home network i play with as a hobby my desktop is a elitedesk 705 transferring to a hp 6200 pro as my truenas scale server, going through a cisco sg300-28pp
Old hardware i know but for the life of me i cant get more than 2Mb/s transfers ether way
For one transfer i got >30Mb/s which i understand is fine for a megabit connection but haven't been able to replicate it since
Iperf is normal but samba transfers dont go above 2Mb/s ill attach what i can think of let me know if there's anything else.
also the same result from a vm to freenas using a separate nic

 

[jgreco]

2Mb/s transfers ether way

Your fourth screen shot shows transfer speeds of 1.19MBytes/sec, which is about 10 megabits per second. Making up random abbreviations for your units is confusing to the discussion; please see

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

I suspect the HP gear has decent ethernet chipsets so the usual inquiry about Realtek ethernet can be skipped, I think.

 

[nooobz]

Your fourth screen shot shows transfer speeds of 1.19MBytes/sec, which is about 10 megabits per second. Making up random abbreviations for your units is confusing to the discussion; please see

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

I suspect the HP gear has decent ethernet chipsets so the usual inquiry about Realtek ethernet can be skipped, I think.

yup sorry im just a hobbyist..regardless the transfers are rather slow?
is there any other info i can provide ?
the realtek is a pci-e card i used to test on the same machine using different nics

 

[nooobz]

Your fourth screen shot shows transfer speeds of 1.19MBytes/sec, which is about 10 megabits per second. Making up random abbreviations for your units is confusing to the discussion; please see

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

I suspect the HP gear has decent ethernet chipsets so the usual inquiry about Realtek ethernet can be skipped, I think.

and sorry i just realized i meant MB/s not Mb/s

 

[nooobz]

Im sorry for messing up on the terminology but cant anyone help me?
its basically useless takes an hour to transfer 2 GB
both ends say connected at 1000/1000

 

[jgreco]

Im sorry for messing up on the terminology but cant anyone help me?
its basically useless takes an hour to transfer 2 GB
both ends say connected at 1000/1000

I don't think it's the terminology that's blocking responses at this point. It may just be that no one knows. Unless you reveal something that's obviously wrong, such as a Realtek ethernet, having too little memory, enabling dedup, aggressive compression, random Linux thing, etc., there just aren't a ton of people using SCALE on random desktops. The details provided don't really provide much in the way of clues to my eyes.

 

[nooobz]

all good... just did a fresh install see if that helps ill also look into the things you mentioned to
thanks though

 

[nooobz]

somehow its my switch decided to run straight to my isp router now its running at 40MB/s which is way better.
its a sg300-28pp running in level 3 mode and reports to be connected at 1000 full duplex on all connected ports.
might not be the best place to ask but dose any one have a clue why that would that would be?

 

[Davvo]

The elitedesk has a realtek NIC. That should be the cause of first issue. At least that what I can assume given the scarcity of infos.

 

[nooobz]

The elitedesk has a realtek NIC. That should be the cause of first issue. At least that what I can assume given the scarcity of infos

 

[nooobz]

The realtek is pci-e and just for testing before i knew they caused problems .
When i deleted the switch i got normal results
Its definitely something im doing but is weird for enterprise level gear with the only thing changed being running in l3 instead of l2

 

[jgreco]

While HP isn't known to use Realteks, can we please identify what sort of ethernet adapters are in play on each end here? In particular, HP is known for using Broadcom at times, which can be nearly as evil as Realtek.

 

[nooobz]

While HP isn't known to use Realteks, can we please identify what sort of ethernet adapters are in play on each end here? In particular, HP is known for using Broadcom at times, which can be nearly as evil as Realtek.

 

[nooobz]

Not at home but looks like the elitedesk has a broadcom nic.
You can delete this thread if you want nothing of value in it ill figure it out or go back to proxmox or unraid

 

[jgreco]

You can delete this thread if you want nothing of value in it ill figure it out or go back to proxmox or unraid

If you're going to have a bad attitude, then, by all means.

On the other hand, if you want help from the community: we are not able to see your system, we cannot try experiments on your system, and we have only what you tell us to go on. It is necessary for us to talk back and forth. One of the reasons that the Forum Rules, conveniently linked at the top of every page in red, says

It's important to remember that hardware information is extremely important when diagnosing problems so please make sure to include this information if you need help troubleshooting a problem.

is because this is very true. There's a lot of garbage hardware out there and it isn't expected to work well. If you are running Broadcom ethernet on the TrueNAS side, this could very well be a problem. As I previously mentioned, they're nearly as problematic as the Realteks.

 

[nooobz]

Nah sorry didn't mean for it to come across like that, only said that because i read truenas is geared towards enterprise deployments (i guess most are but anyways) and i had more luck with those just didn't want to virtualize a nas or pay for a key

And also now that i removed the sg300 im seeing normal results so the problem must be with that.

The broadcom is on the client side the server is a Intel Corporation 82579LM

Im now getting probably 60ish MB/s on average plugged strait into the isp router but yeah i included what i included because i wasnt to sure what else to include if you have any suggestions on what else i should have included i would love to know

 

[nooobz]

confirmed client is Broadcom BCM5762 NetXtreme PCI-E Gigabit Ethernet Controller

is it an issue on the client side or just the server?

 

[jgreco]

confirmed client is Broadcom BCM5762 NetXtreme PCI-E Gigabit Ethernet Controller

is it an issue on the client side or just the server?

FreeBSD in particular has problems with at least SOME Broadcom chipsets, and I am aware of issues with Linux and Broadcom as well (maybe not the same chipsets, I do not have particulars handy).

From an ethernet perspective, the roles of "client" and "server" may not be meaningful. A "client" role requires a chipset capable of the low bar of handling a handful, maybe a dozen, active TCP streams at once. We just discussed this recently in a Realtek thread that ended up as a resource. A "server" role is best handled by a chipset capable of the much higher bar of handling thousands of active TCP streams efficiently, with TCP offload, interrupt coalescing, virtual function support, multiple queues, etc. But a small "server" may be no more stressy than a "client".

We expect that an Intel-to-Intel connection from FreeBSD to FreeBSD will see a well performing flat 940Mbps or thereabouts, and that should be the same for Linux too. The interesting thing is that Intel has put in a hell of a lot of work on their driver, and their "Intel Desktop CT" card, a "low bar"/"client" card, is absolutely capable of the 940Mbps. It is possible it would start to suffer under a massive "server" workload, but most people are able to use it in a "NAS" server role very successfully.

What's less certain is the Broadcom stuff. The BCM5762 is a "Tigon III" chipset, a chipset based around two MIPS R4000 CPU cores, building on the Alteon Tigon I and II design. By the time Broadcom acquired this, the driver and firmware was a pile of workarounds for known bugs, conditionals for various oddities, and everything else you'd expect in a driver that started life as a PCI (non-e) driver in the late 1990's. Part of the original design model was to have optional licensed features that could be activated via firmware, but this just seems to have been more trouble than it is worth. My own estimation is that the Tigon III may be usable in some cases, but more often than not is a pile of garbage. This is not foreign stuff to me, I still have some of the earlier Tigon cards in inventory here from all those years ago.

I get suspicious of these complex cards with MIPS CPU cores on them every time someone shows up with one having weird performance problems. Some of the Broadcom chipsets are known to have problems on FreeBSD or Linux, and others are known to perform poorly on FreeBSD or Linux. I'm not certain how much effort Broadcom puts into the drivers for these chipsets on the Windows platform, either. So basically I just don't have a lot of faith in them.

It may sound crazy but I find one of the best things for network debugging is just to start swapping stuff out to see if you can isolate a problem. Ethernet cards, switches, routers, cables, SFP+ modules, etc. It is sometimes necessary to test stuff against a known working endpoint to help isolate problems. It can be a bunch of messing around.