Please Help Determining Bottleneck (Speed Issues)

[Callipygous]

Hello. I'm having some throughput issues and am looking for advice on where to start checking for issues. I spent a bit of time in the IRC channel and was advised to move on to better hardware. While I would love to do that, I have much more time than money. I'm aware this is not the ideal hardware. I'm most concerned with finding the most cost effective way to increase speeds. If it is determined that upgrading the hardware is the only way to do that, then so be it; I would like to exhaust all other options first though.

Running FreeNAS 9.2.1.3 x64

Hardware:
Motherboard/CPU combo here
8GB of G.Skill RAM here
2 Hitachi Deskstars here
2 Seagate Barracudas here
Intel Gigabit NIC here

The NAS is a few feet from my RT-AC66U connected using a couple feet of CAT5E. My desktop is connected using power line converters (CAT5E both ends) and my HTPC is connected using a 50 foot CAT5E cable. The powerline converters don't appear to be the bottleneck to me as I get consistent results between my HTPC and the desktop. It's easy to eliminate though if anyone thinks contrary.

DMESG output here

ifconfig output here

pciconf -lv here

iperf results:

Code:

[root@freenas] ~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 192.168.1.156 port 5001 connected with 192.168.1.145 port 51743
[ ID] Interval      Transfer    Bandwidth
[  4]  0.0-10.0 sec  21.5 MBytes  18.0 Mbits/sec

Actually speeds are around 4MBps read/write.

While testing the server by transferring large files using cifs shares I have never seen the CPU nor RAM rise above 50% usage per the monitoring graph in the http GUI.

 

[praecorloth]

If you're happy with the powerline adapter, then I'd say start checking cables. Even if they're store bought cables, they can go bad if they've had a rough life. I have a length of store bought CAT6 that used to be awesome, but now only registers as high as 600Mb/s in iperf and won't transfer actual data faster than about 15MB/s.

Also, you're a pretty lean on CPU horsepower. Are you using any special features of ZFS or anything?

 

[cyberjock]

uhh.. what?! I'm really missing how you think the powerline adapter *isn't* to blame.

Your iperf results show 18Mbit/sec.. that comes out to about 4MB/sec. That's exactly what your actual speeds are according to your post. If you search the forums for "powerline" you'll see that they have ALWAYS been to blame for low speed. Powerline speeds vary widely, and if you want more than about 20MB/sec powerline and wifi are out of the question unless you get very very lucky. And nobody here will help anyone troubleshoot a "slow server" complain that involves the words "wifi" or "powerline" because we know better. ;)

Also, that E-350 is NOT a powerhouse at all. It was horribly ill-suited for a FreeNAS box 2 years ago when the E350 was more popular, and due to the OS having more overhead than 2 years ago it's an even worse choice. Not to mention the horrible Realtek NIC and the lack of ECC RAM support and you've got a sever that you should replace if your data is important. That E350 CPU came out in 2011, and it was a step backwards in 2011. It doesn't do any better in 2014. ;)

 

[Callipygous]

I eliminated the powerline adapter and iperf speeds jumped to 340/280MBps. Actual speeds are around 55MBps. I still get 4MBps actual on the HTPC which has always been hooked up directly with CAT5E , so that's interesting. Realtek NIC on the HTPC a possible issue?

I'm very happy about getting 55MBps though, a 14x improvement in speed. Thank you both for your replies in helping me improve my setup!

Is that roughly the maximum throughput possible with my current hardware?

Cyberjock, I hear you loud and clear. As soon as I have my other financial priorities taken care of I will most assuredly upgrade my server hardware. Until then, I will have to make do.

 

[cyberjock]

it's possible. Youv'e kind of ruled out the FreeNAS box if you are getting 55MB/sec on your desktop.

Yeah.. typically 50-60MB/sec is the most anyone has been able to get out of an E350 without doing things that are dangerous to your data.

 

[Callipygous]

Came across an Intel Pro 1000 MT Server NIC today. It's a 64 bit card and I only have 32 bit pci slots, so performance would be downgraded relative to 64 bit performance. I'm curious if it would be an upgrade over my current NIC? Thanks.

 

[joeschmuck]

I doubt you would see any benefits of moving from one Intel NIC to another. your limitation is likely the CPU. If you want better performance, try to run an older FreeNAS version, 8.3.x or even better, 8.0.4. It all depends on what you use your FreeNAS system for.

 

[Nick2253]

Don't forget about the limitation of the physical drive in your HTPC. Any weak link in the chain, can give you slow speeds, and you need to check each link:

FreeNAS HDD -> cable -> controller -> mobo/cpu -> network card -> network cable -> switch -> network cable -> HTPC nic ->mobo/cpu -> controller -> cable -> HTPC HDD

It's really easy to have a bad cable screw you up. If you've tested it successfully to a different computer (using the switch), you know that everything left of the switch (in my ordering here) works correctly. But that says nothing about everything on the right. I actually had this bite me last night, because I had a faulty SATA cable that was capping my transfer speeds at about 60 MB/s. Replace the cable and, boom, 115 MB/s transfer.

To test the other stuff, try doing a transfer between the HTPC and your desktop. To check the controller, you can try different cables, or you can test direct HDD to HDD transfer inside the HTPC. Just make sure you're thinking about the data transfer chain.

 

[cyberjock]

If you are using PCI, you are in for a world of hurt. PCI is parallel bus architecture. It is also limited to 133MB/sec.. for all of your PCI devices.... combined. So you can expect that *any* other PCI device using *any* bandwidth on the bus will take away from your network speed. That's why nobody uses PCI anymore.