New to FreeNAS - 8x3tb in RAIDZ2 slow

Status
Not open for further replies.
A

andoy31

Explorer
Joined
Apr 29, 2012
Messages
65
Btw, how's your iperf test? Might be worth to check that as well.
 
L

Lastb0isct

Explorer
Joined
Jun 15, 2011
Messages
96
Hi all, so I was able to do a couple tests last night with iPerf...here are the results:

This was from my workstation to the NAS with a buffer of 12MB:
Code:
bin/iperf.exe -c 192.168.1.38 -P 1 -i 1 -p 5001 -N -l 12.0M -f M -t 20
------------------------------------------------------------
Client connecting to 192.168.1.38, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[224] local 192.168.1.127 port 59452 connected with 192.168.1.38 port 5001
[ ID] Interval       Transfer     Bandwidth
[224]  0.0- 1.0 sec   108 MBytes   108 MBytes/sec
[224]  1.0- 2.0 sec   120 MBytes   120 MBytes/sec
[224]  2.0- 3.0 sec  96.0 MBytes  96.0 MBytes/sec
[224]  3.0- 4.0 sec   120 MBytes   120 MBytes/sec
[224]  4.0- 5.0 sec  96.0 MBytes  96.0 MBytes/sec
[224]  5.0- 6.0 sec   120 MBytes   120 MBytes/sec
[224]  6.0- 7.0 sec   120 MBytes   120 MBytes/sec
[224]  7.0- 8.0 sec  96.0 MBytes  96.0 MBytes/sec
[224]  8.0- 9.0 sec   120 MBytes   120 MBytes/sec
[224]  9.0-10.0 sec  96.0 MBytes  96.0 MBytes/sec
[224] 10.0-11.0 sec   120 MBytes   120 MBytes/sec
[224] 11.0-12.0 sec   120 MBytes   120 MBytes/sec
[224] 12.0-13.0 sec  96.0 MBytes  96.0 MBytes/sec
[224] 13.0-14.0 sec   120 MBytes   120 MBytes/sec
[224] 14.0-15.0 sec  96.0 MBytes  96.0 MBytes/sec
[224] 15.0-16.0 sec   120 MBytes   120 MBytes/sec
[224] 16.0-17.0 sec   120 MBytes   120 MBytes/sec
[224] 17.0-18.0 sec  96.0 MBytes  96.0 MBytes/sec
[224] 18.0-19.0 sec   120 MBytes   120 MBytes/sec
[224] 19.0-20.0 sec  96.0 MBytes  96.0 MBytes/sec


This was from my workstation to my NAS with default buffer sizes:
Code:
bin/iperf.exe -c 192.168.1.38 -P 1 -i 1 -p 5001 -N -f M -t 20
------------------------------------------------------------
Client connecting to 192.168.1.38, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[224] local 192.168.1.127 port 59537 connected with 192.168.1.38 port 5001
[ ID] Interval       Transfer     Bandwidth
[224]  0.0- 1.0 sec  41.0 MBytes  41.0 MBytes/sec
[224]  1.0- 2.0 sec  41.0 MBytes  41.0 MBytes/sec
[224]  2.0- 3.0 sec  40.9 MBytes  40.9 MBytes/sec
[224]  3.0- 4.0 sec  40.7 MBytes  40.7 MBytes/sec
[224]  4.0- 5.0 sec  40.9 MBytes  40.9 MBytes/sec
[224]  5.0- 6.0 sec  40.6 MBytes  40.6 MBytes/sec
[224]  6.0- 7.0 sec  40.9 MBytes  40.9 MBytes/sec
[224]  7.0- 8.0 sec  38.8 MBytes  38.8 MBytes/sec
[224]  8.0- 9.0 sec  40.8 MBytes  40.8 MBytes/sec
Done.


This is from the NAS to my workstation with both default and 12MB behaving roughly the same:

m2cHJ1k.png

This is after I disabled the Sophos I had installed on my workstation. I think that maybe the NAS to Workstation benchmarks aren't that great because I'm running the NAS off of a usb stick. I'm not sure if i mentioned that earlier or not, but could that be causing these issues? Other than that I have no idea what could be causing such low benchmarks.

When I do an actual transfer I'm getting ~100MB/s to the NAS and still ~50MB/s from the NAS. So the iPerf doesn't seem completely accurate.
 
L

Lastb0isct

Explorer
Joined
Jun 15, 2011
Messages
96
Here is the iozone output:

Code:
Excel chart generation enabled
	Record Size 4 KB
	File size set to 1024000 KB
	Command line used: iozone -R -l 2 -u 2 -r 4k -s 1000m -F /mnt/freenas/f1 /mnt/freenas/f2 /mnt/freenas/f3
	Output is in Kbytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 Kbytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
	Min process = 2 
	Max process = 2 
	Throughput test with 2 processes
	Each process writes a 1024000 Kbyte file in 4 Kbyte records

	Children see throughput for  2 initial writers 	=  104753.08 KB/sec
	Parent sees throughput for  2 initial writers 	=   92322.01 KB/sec
	Min throughput per process 			=   49314.46 KB/sec 
	Max throughput per process 			=   55438.62 KB/sec
	Avg throughput per process 			=   52376.54 KB/sec
	Min xfer 					=  918096.00 KB

	Children see throughput for  2 rewriters 	=  123172.79 KB/sec
	Parent sees throughput for  2 rewriters 	=  108864.92 KB/sec
	Min throughput per process 			=   55455.61 KB/sec 
	Max throughput per process 			=   67717.18 KB/sec
	Avg throughput per process 			=   61586.39 KB/sec
	Min xfer 					=  856808.00 KB

	Children see throughput for  2 readers 		=  430075.89 KB/sec
	Parent sees throughput for  2 readers 		=  421365.09 KB/sec
	Min throughput per process 			=  206548.34 KB/sec 
	Max throughput per process 			=  223527.55 KB/sec
	Avg throughput per process 			=  215037.95 KB/sec
	Min xfer 					=  932536.00 KB

	Children see throughput for 2 re-readers 	=  378131.20 KB/sec
	Parent sees throughput for 2 re-readers 	=  377983.91 KB/sec
	Min throughput per process 			=  175150.92 KB/sec 
	Max throughput per process 			=  202980.28 KB/sec
	Avg throughput per process 			=  189065.60 KB/sec
	Min xfer 					=  883760.00 KB

	Children see throughput for 2 reverse readers 	=  395411.47 KB/sec
	Parent sees throughput for 2 reverse readers 	=  394231.97 KB/sec
	Min throughput per process 			=  189583.59 KB/sec 
	Max throughput per process 			=  205827.88 KB/sec
	Avg throughput per process 			=  197705.73 KB/sec
	Min xfer 					=  948412.00 KB

	Children see throughput for 2 stride readers 	=  333470.31 KB/sec
	Parent sees throughput for 2 stride readers 	=  332478.42 KB/sec
	Min throughput per process 			=  158016.30 KB/sec 
	Max throughput per process 			=  175454.02 KB/sec
	Avg throughput per process 			=  166735.16 KB/sec
	Min xfer 					=  916792.00 KB

	Children see throughput for 2 random readers 	=  284735.88 KB/sec
	Parent sees throughput for 2 random readers 	=  284661.26 KB/sec
	Min throughput per process 			=  136853.88 KB/sec 
	Max throughput per process 			=  147882.00 KB/sec
	Avg throughput per process 			=  142367.94 KB/sec
	Min xfer 					=  947836.00 KB

	Children see throughput for 2 mixed workload 	=   66035.75 KB/sec
	Parent sees throughput for 2 mixed workload 	=   20539.55 KB/sec
	Min throughput per process 			=   16196.28 KB/sec 
	Max throughput per process 			=   49839.47 KB/sec
	Avg throughput per process 			=   33017.87 KB/sec
	Min xfer 					=  334944.00 KB

	Children see throughput for 2 random writers 	=   17296.68 KB/sec
	Parent sees throughput for 2 random writers 	=    9152.54 KB/sec
	Min throughput per process 			=    6199.22 KB/sec 
	Max throughput per process 			=   11097.47 KB/sec
	Avg throughput per process 			=    8648.34 KB/sec
	Min xfer 					=  572044.00 KB

	Children see throughput for 2 pwrite writers 	=  102785.50 KB/sec
	Parent sees throughput for 2 pwrite writers 	=   93024.41 KB/sec
	Min throughput per process 			=   50553.65 KB/sec 
	Max throughput per process 			=   52231.85 KB/sec
	Avg throughput per process 			=   51392.75 KB/sec
	Min xfer 					= 1003512.00 KB

	Children see throughput for 2 pread readers 	=  359840.88 KB/sec
	Parent sees throughput for 2 pread readers 	=  359763.59 KB/sec
	Min throughput per process 			=  168849.77 KB/sec 
	Max throughput per process 			=  190991.11 KB/sec
	Avg throughput per process 			=  179920.44 KB/sec
	Min xfer 					=  905496.00 KB



"Throughput report Y-axis is type of test X-axis is number of processes"
"Record size = 4 Kbytes "
"Output is in Kbytes/sec"

"  Initial write "  104753.08 

"        Rewrite "  123172.79 

"           Read "  430075.89 

"        Re-read "  378131.20 

"   Reverse Read "  395411.47 

"    Stride read "  333470.31 

"    Random read "  284735.88 

" Mixed workload "   66035.75 

"   Random write "   17296.68 

"         Pwrite "  102785.50 

"          Pread "  359840.88 


iozone test complete.
 
C

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Holy smokes! Did you look at that screenshot!? 17592186044416MB/sec! That's about 17.6 exabytes a second! Just for comparison the entire month of Dec 2012 saw all internet traffic total approximately 885PB, or about 20x less than your server in 1 second! Can I buy your server from you? Just for a few weeks. :D

I'm sending you a PM in a few mins...
 
L

Lastb0isct

Explorer
Joined
Jun 15, 2011
Messages
96
cyberjock said:
Holy smokes! Did you look at that screenshot!? 17592186044416MB/sec! That's about 17.6 exabytes a second! Just for comparison the entire month of Dec 2012 saw all internet traffic total approximately 885PB, or about 20x less than your server in 1 second! Can I buy your server from you? Just for a few weeks. :D

I'm sending you a PM in a few mins...
Click to expand...

I went into the future and brought back some quantum powered networking equipment...way ahead of you all. :)
Responding to your PM now...
 
L

Lastb0isct

Explorer
Joined
Jun 15, 2011
Messages
96
Thanks to cyberjock the problem has been solved!!

The issue wasn't hardware/network/freenas. Rather, it was teracopy, a program I installed to handle transfers a little more seamlessly. It was giving large hits on read/writes to/from the workstation i was doing my testing from. After disabling it as my main file transfer software we were seeing +100MB/s Read & Writes!! Let this be a lesson to all...teracopy sorta, kinda sucks.

Thanks cyberjock!!
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "New to FreeNAS - 8x3tb in RAIDZ2 slow"

Similar threads

T
  • Locked
Slow transfer speeds
Replies
14
Views
3K
eddie200112
eddie200112
C
  • Locked
Slow write speeds on a ZFS setup and a gigabit network
Replies
17
Views
16K
paleoN
paleoN
A
How to make SMB shares on FreeNAS with Mac clients smoking fast! OS X solved
Replies
13
Views
13K
seanm
S
B
  • Locked
5MB/s Transfer Speed on New FreeNAS Box
Replies
4
Views
3K
mikensan
M
J
  • Locked
Slow speeds transferring large files from personal computer to new FreeNas build
2
Replies
22
Views
4K
pschatz100
P
Top