Slow transfer speeds, 30-35MB/s

[hayer]

Okey,

I have tried a couple of things, like enabling jumbo frames, and other fancy network stuff. Got a 1GBit/s network going.

My hardware is:
i3-3225 CPU @ 3.30GHz
2x8GB DDR3 1600MHz ECC/Unbuffered
X9SCL-F

For disks I have 6 WD Red 1TB.

The problem is I am getting 30-35MB/s. This seems kinda slow to me? Or should I not except anything more?

In the near future, 1-2 months, I'm getting 16GB more RAM and 6 more disks, WD Red 3TB. Any obvious problems with that other than the 32GB RAM with 6x1TB Disks + 6x3TB Disks?

 

[jgreco]

You should probably be doing substantially better than that, though you left out some specifics like what sort of pool you have (RAIDZ2? etc)

What sort of speeds do you get out of iperf to your client?

What sort of speeds do you get locally for reads and writes?

For comparison:

E3-1230 (3.2GHz), two cores, 8GB RAM, virtualized under ESXi, running on some Supermicro X9S class system, so somewhat similar to your system. Local read/write speeds: Pool is 4 x ST3000DM001 in RAIDZ2, 4K, 80% capacity. Sees ~80MB/sec write speeds with "dd if=/dev/zero of=file bs=1048576", and about 220MB/sec read speeds.

I would guess that your problem is likely to be that your network is not performing up to par, but check the pool performance first.

 

[hayer]

Using RAIDZ1 with offsite backup atm. Going to set the 6x1TB disks in RAIDZ2 when I get the 6x3TB disks. The 6x3TB disks are for media so I'm going to run them in RAIDZ1.

Currently running "dd if/dev/zero of=file bs=1048576".. Used about 1 minute already now..

Okey, I CTRL+C'd that one.. Here is the results:

[hayer@stronghold01] /mnt/storage01> dd if=/dev/zero of=file bs=104857
287026+0 records in
287025+0 records out
30096580425 bytes transferred in 320.606043 secs (93874027 bytes/sec)

 

[jgreco]

So your pool is able to write at 94MB/sec. Check network performance next.

 

[hayer]

Well,.. how do I do that? Whats the best approach?

 

[jgreco]

See msg #2: "What sort of speeds do you get out of iperf to your client?"

 

[hayer]

bin/iperf.exe -c 10.13.37.209 -P 1 -i 1 -p 5001 -f M -n 1073741824
------------------------------------------------------------
Client connecting to 10.13.37.209, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[152] local 10.13.37.224 port 34877 connected with 10.13.37.209 port 5001
[ ID] Interval Transfer Bandwidth
[152] 0.0- 1.0 sec 37.2 MBytes 37.2 MBytes/sec
[152] 1.0- 2.0 sec 37.2 MBytes 37.2 MBytes/sec
[152] 2.0- 3.0 sec 37.1 MBytes 37.1 MBytes/sec
[152] 3.0- 4.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 4.0- 5.0 sec 36.7 MBytes 36.7 MBytes/sec
[152] 5.0- 6.0 sec 36.8 MBytes 36.8 MBytes/sec
[152] 6.0- 7.0 sec 36.5 MBytes 36.5 MBytes/sec
[152] 7.0- 8.0 sec 35.9 MBytes 35.9 MBytes/sec
[152] 8.0- 9.0 sec 35.8 MBytes 35.8 MBytes/sec
[152] 9.0-10.0 sec 35.9 MBytes 35.9 MBytes/sec
[152] 10.0-11.0 sec 36.4 MBytes 36.4 MBytes/sec
[152] 11.0-12.0 sec 37.1 MBytes 37.1 MBytes/sec
[152] 12.0-13.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 13.0-14.0 sec 37.1 MBytes 37.1 MBytes/sec
[152] 14.0-15.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 15.0-16.0 sec 36.4 MBytes 36.4 MBytes/sec
[152] 16.0-17.0 sec 36.6 MBytes 36.6 MBytes/sec
[152] 17.0-18.0 sec 36.7 MBytes 36.7 MBytes/sec
[152] 18.0-19.0 sec 36.7 MBytes 36.7 MBytes/sec
[152] 19.0-20.0 sec 36.8 MBytes 36.8 MBytes/sec
[ ID] Interval Transfer Bandwidth
[152] 20.0-21.0 sec 36.7 MBytes 36.7 MBytes/sec
[152] 21.0-22.0 sec 36.9 MBytes 36.9 MBytes/sec
[152] 22.0-23.0 sec 36.8 MBytes 36.8 MBytes/sec
[152] 23.0-24.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 24.0-25.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 25.0-26.0 sec 37.0 MBytes 37.0 MBytes/sec
[152] 26.0-27.0 sec 36.8 MBytes 36.8 MBytes/sec

Tried running iperf, dunno about my parameters.

 

[cyberjock]

And that tells me you have a networking hardware issue somewhere. You should be getting more than double that at the minimum but probably over 100MB/sec.

You have an issue with your network cards(on either or both ends), the cabling in between the 2 machines, or the switches/routers in between.

 

[hayer]

Okey, checked the cables. The network goes

pfSense -> Linksys SLM2008 -> Linksys SLM2008 -> My Computer

|
FreeNAS​

Any settings that I should check? Or just "brute-test" everything?

 

[cyberjock]

Brute test everything to find the cause... That's the best way to find it.

 

[gpsguy]

So, it looks like you have 2 - Linksys SLM2008's linked together. Can you plug your "my computer" into the same Linksys that the FreeNAS server is connected to? That will simplify the testing.