SOLVED The network transfer speed difference

[diatezzz]

Hi all,
I've been using FreeNas for few month and suddenly noticed the problem: The transfer speed from NAS much better than transfer speed to NAS.
I've checked and replaced the network cables and router but no effect.
Current configuration:
Router: ASUS RT-AC66U
MB: ASUS E45M1-i deluxe with AMD e-450 processor and Realtek 8111E network adapter.
RAM: 8 Gb
HDD: 4xWD Green 1Tb (RAIDZ1, no encryption, no dedup) + Seagate 2Tb

What am I doing wrong?

For example, the output of iperf:
iperf -c 192.168.1.138 -t 300 -i 60 -d
-----------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.1.138, TCP port 5001
TCP window size: 161 KByte (default)
------------------------------------------------------------
[ 5] local 192.168.1.131 port 58560 connected with 192.168.1.138 port 5001
[ 6] local 192.168.1.131 port 5001 connected with 192.168.1.138 port 34584
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-60.0 sec 816 MBytes 114 Mbits/sec
[ 6] 0.0-60.0 sec 4.12 GBytes 590 Mbits/sec
[ 6] 60.0-120.0 sec 4.18 GBytes 598 Mbits/sec
[ 5] 60.0-120.0 sec 848 MBytes 119 Mbits/sec
[ 5] 120.0-180.0 sec 794 MBytes 111 Mbits/sec
[ 6] 120.0-180.0 sec 4.24 GBytes 607 Mbits/sec
[ 6] 180.0-240.0 sec 4.19 GBytes 600 Mbits/sec
[ 5] 180.0-240.0 sec 838 MBytes 117 Mbits/sec
[ 5] 240.0-300.0 sec 834 MBytes 117 Mbits/sec
[ 5] 0.0-300.0 sec 4.03 GBytes 115 Mbits/sec
[ 6] 240.0-300.0 sec 4.20 GBytes 601 Mbits/sec
[ 6] 0.0-300.0 sec 20.9 GBytes 599 Mbits/sec

and systat:
systat -ifstat
/0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10

Load Average ||||
Interface Traffic Peak Total
epair0a in 0.000 KB/s 0.000 KB/s 188.739 MB
out 0.000 KB/s 0.000 KB/s 7.370 GB

bridge0 in 0.000 KB/s 0.000 KB/s 7.599 GB
out 391.425 KB/s 84.833 MB/s 216.513 GB

lo0 in 1.325 KB/s 17.426 KB/s 189.941 GB
out 1.325 KB/s 17.426 KB/s 189.944 GB

re0 in 7.156 MB/s 27.024 MB/s 203.395 GB
out 391.424 KB/s 84.833 MB/s 243.839 GB

 

[hugovsky]

Probably it's a problem with your realtek card.

HDD: 4xWD Green 1Tb (RAIDZ1, no encryption, no dedup) + Seagate 2Tb

What does this mean? 1 pool with 4x1TB green + 2TB disk?

 

[diatezzz]

Probably it's a problem with your realtek card.

It's sad because there is no place in the case to put another =(

What does this mean? 1 pool with 4x1TB green + 2TB disk?

No :) 1 pool with 4x1TB and 2Tb separate one

 

[hugovsky]

And how is pool 4x1TB configured?

EDIT: Sorry, RAIDZ1. I've seen it.

 

[hugovsky]

Well, your erratic perfomance could be the sum of a number of factors. Realtek nic, only minimum memory... Can't really tell. But if you can't at least install an intel nic card, I don't think there's much you can do. Try to upgrade memory to 16GB

 

[diatezzz]

Try to upgrade memory to 16GB

Thank you for advise.
I think, if it is not a software issue, than it's time to buy new motherboard because 8Gb is maximum amount for this MB.

 

[hugovsky]

Do take the time to take a look at this thread. It might be useful.

 

[depasseg]

I can't tell if your iperf testing is from or to your freenas, but my suggestion would be to test both directions.

Then I would test the storage separately using dd or copying large files to see if it is storage related.

Then combine the two.

And can you clarify what your zpool status says (please paste it in Code tags so the formatting is preserved).

8 Gb of RAM should be ok, but did you apply the wdidle fix to the WD Green drives?

 

[diatezzz]

I can't tell if your iperf testing is from or to your freenas, but my suggestion would be to test both directions.

I did iperf both directions, with -d key on 3 different computers and they resulted the same.

And can you clarify what your zpool status says (please paste it in Code tags so the formatting is preserved).

Code:

zpool status
  pool: freenas-boot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(7) for details.
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    freenas-boot  ONLINE       0     0     0
      da0p2     ONLINE       0     0     0

errors: No known data errors

  pool: vol1
state: ONLINE
  scan: none requested
config:

    NAME                                          STATE     READ WRITE CKSUM
    vol1                                          ONLINE       0     0     0
      gptid/947eacb2-f901-11e5-bae1-c8600085ffc3  ONLINE       0     0     0

errors: No known data errors

  pool: vol2
state: ONLINE
  scan: scrub repaired 0 in 1h42m with 0 errors on Sun Apr  3 11:42:29 2016
config:

    NAME                                            STATE     READ WRITE CKSUM
    vol2                                            ONLINE       0     0     0
      raidz1-0                                      ONLINE       0     0     0
        gptid/01b4dfc9-8d66-11e1-b291-c8600085ffc3  ONLINE       0     0     0
        gptid/02472e2c-8d66-11e1-b291-c8600085ffc3  ONLINE       0     0     0
        gptid/02e20e06-8d66-11e1-b291-c8600085ffc3  ONLINE       0     0     0
        gptid/036fb88e-8d66-11e1-b291-c8600085ffc3  ONLINE       0     0     0

errors: No known data errors

but did you apply the wdidle fix to the WD Green drives?

No, I didn't. Unfortunately, it's the first time I hearing about it.

But now, my opinion, that drives have nothing to do with the issue because transferring speed files proves the ipefr results. But I'm just new user of FreeNAS. I have bought Intel NIC yet and plan to install it tomorrow when I will get the riser card. So, will see. =)

 

[diatezzz]

BTW,
I've just checked HDD's, I was wrong, not WD green. I have 4x1Tb Seagate barracuda green... But it doesn't matter =)

 

[hugovsky]

If you get the same results, try to connect freenas to pc directly and test with iperf.

 

[diatezzz]

Hello again,
So. I put the new NIC (Intel EXPI9301CT) inside my tiny freenas case and got good results!

iperf is doing much better:

Code:

iperf -c 192.168.1.131 -t 300 -i 60 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.1.131, TCP port 5001
TCP window size:  321 KByte (default)
------------------------------------------------------------
[  5] local 192.168.1.138 port 63852 connected with 192.168.1.131 port 5001
[  4] local 192.168.1.138 port 5001 connected with 192.168.1.131 port 50611
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec  3.51 GBytes   502 Mbits/sec
[  4]  0.0-60.0 sec  3.78 GBytes   541 Mbits/sec
[  4] 60.0-120.0 sec  4.05 GBytes   579 Mbits/sec
[  5] 60.0-120.0 sec  3.09 GBytes   442 Mbits/sec
[  5] 120.0-180.0 sec  2.76 GBytes   396 Mbits/sec
[  4] 120.0-180.0 sec  4.32 GBytes   618 Mbits/sec
[  5] 180.0-240.0 sec  2.90 GBytes   414 Mbits/sec
[  4] 180.0-240.0 sec  4.18 GBytes   599 Mbits/sec
[  5] 240.0-300.0 sec  2.87 GBytes   411 Mbits/sec
[  4] 240.0-300.0 sec  4.18 GBytes   599 Mbits/sec
[  5]  0.0-300.0 sec  15.1 GBytes   433 Mbits/sec
[  4]  0.0-300.0 sec  20.5 GBytes   587 Mbits/sec

Downloading and uploading of 16Gb .mkv file gives the good statistics too:

Code:

systat -ifstat

                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
     Load Average   |||||||||      

      Interface           Traffic               Peak                Total
        epair1a  in      0.522 KB/s          0.879 KB/s          565.804 KB
                 out     0.241 KB/s          2.477 KB/s          726.727 KB

        epair0a  in      0.013 KB/s          0.535 KB/s          109.668 KB
                 out     0.030 KB/s          3.848 KB/s         1000.767 KB

        bridge0  in      0.808 KB/s          7.067 KB/s            2.288 MB
                 out     4.803 KB/s        109.938 MB/s           41.648 GB

            lo0  in      0.000 KB/s         22.848 KB/s            2.199 MB
                 out     0.000 KB/s         22.848 KB/s            2.199 MB

            em0  in      2.948 KB/s         79.024 MB/s           47.716 GB
                 out     4.530 KB/s        109.942 MB/s           41.646 GB

But it seems that it is not the best of my network and it seems that my processor hit the ceiling because it had been loaded for 100% during the transfer so its time to change the MB.