Be careful with benchmarking your zpools...

[Rainwulf]

Just been testing out the 8x8tb drives i have with Cyberjock today, we test wrote a 1tb block (average 328meg a second).

Reading back from it showed 13-20 meg a second?
Waited minutes just to see if it was the 8tb "caching". No change.
Waited hours just in case it was some weird drive issue. No change.
Rebooted just in case. No change.
Powered down completely, no change.

So whatever was the issue.. stuck around.. but......

Thought i would try copying directly to dev/null....

[root@Riva] ~# dd if=/mnt/Riva/Data/testfile bs=1M
^C75+0 records in
71+6 records out
77594624 bytes transferred in 6.220215 secs (12474589 bytes/sec)

[root@Riva] ~# dd if=/mnt/Riva/Data/testfile of=/dev/null bs=1M
^C13951+0 records in
13951+0 records out
14628683776 bytes transferred in 12.342114 secs (1185265653 bytes/sec)
[root@Riva] ~#

When not specifying an output device, which i assumed defaulted to dev/null, speed was 12mb a second.
When specifying dev/null, speed was 1.1Gb a second.

So have i either found a bug in the implementation of dd?
Or have i just found out that not specifying an output device defaults to something else?
Some more research points to the fact that without an output device, it defaults to stdout. It looks like stdout is very slow!

 

[cyberjock]

I just tried this on my machine. I got higher numbers (40MB/sec) when not listing the output device to direct to. I did add the of=/dev/null and got almost 80MB/sec. That's slow for my server, but it is streaming a movie right now.

It does appear that providing the of=/dev/null is required. /shrug

You cancelled the test at 13951 blocks. What speed do you get if you let it run for the full 1TB testfile?

 

[Rainwulf]

Will do that right now.

 

[Rainwulf]

1000000+0 records in
1000000+0 records out
1048576000000 bytes transferred in 1362.062916 secs (769844027 bytes/sec)

 

[titan_rw]

Default for dd is to use stdin / stdout unless specified.

So if you specify "dd if=foobar", the the output should go to stdout, which I would have thought would go to the screen because you aren't sending it elsewhere. Maybe this data is going to the screen "aka terminal, aka putty, etc", and this is then more a test of ssh throughput or something.

You could do "dd if=foobar > /dev/null" if you wanted to utilize stdout. Or even "dd < foobar > /dev/null". Adding the BS if that's what you want of course.

I would very much hope that dd's default destination would NOT be /dev/null. If it was, that would sound like a bug, or at least bad design.

 

[titan_rw]

Yea, this is an ssh test if you have dd output to stdout. Try doing it with a file that's not full of zero's. You quickly get garbage all over your putty window.

I guess ssh / putty handles this differently when you're outputting nulls. (nulls are not displayed, or ignored?)

Running "dd if=foobar bs=1m" while foobar contains nulls shows this in top:

Code:

64397 root  99  0 76060K  4604K CPU9  9  0:46  84.96% sshd
93102 root  52  0  9912K  2628K CPU10  10  0:37  75.78% dd

So basically a single threaded test on sshd. Network utilization was showing 2% of 10gig, so about 200 mbit.

 

[Rainwulf]

Well there we go. learning!!! Learnt so much about bsd in the last week. Nothing like a crash course in "fix your own shit"