Performance issues

[Ulrich Jorgensen]

Hi everyone,

I have built a storage server, that I would like to run FreeNAS on.

The specs are:
Mainboard: SuperMicro X10SRL-F
CPU: Hexacore Xeon CPU
RAM: 128GB DDR4 ECC RAM
Controller: LSI SAS 9300-8i Host Bus Adapter
Disks: 1x Samsung 128GB SSD for FreeNAS installation
4x Samsung SM863 960GB SSD

I have done a clean installation, made a Striped Mirror of the 4 SSD's.

The problem is that the volume is only performing about 150 MB/s in write speed and about half that in read speed.

I used the following commands to do the testing:
# dd if=/dev/zero of=testfile bs=1024 count=50000
# dd if=testfile of=/dev/zero bs=1024 count=50000

Can anyone in here point me in a direction to get it running faster?
I mean the disk themselves should perform around 500 MB/s, and I presume RAID 10 should perform on par or even faster, right?

I would appreciate any input, that can help resolve this issue.

Thank you.

Best regards,
Ulrich

 

[MatthewSteinhoff]

# dd if=/dev/zero of=testfile bs=1024 count=50000

You may want to consider how block size affects your write speed. Here is my four-drive SSD striped mirror...

Code:

$ dd if=/dev/zero of=testfile bs=1024 count=100000
102400000 bytes transferred in 0.776452 secs (131,881,934 bytes/sec)

$ dd if=/dev/zero of=testfile bs=2048 count=100000
204800000 bytes transferred in 0.834657 secs (245,370,267 bytes/sec)

$ dd if=/dev/zero of=testfile bs=4096 count=100000
409600000 bytes transferred in 0.914390 secs (447,948,863 bytes/sec)

$ dd if=/dev/zero of=testfile bs=8192 count=100000
819200000 bytes transferred in 1.115028 secs (734,690,133 bytes/sec)

$ dd if=/dev/zero of=testfile bs=16384 count=100000
1638400000 bytes transferred in 1.512976 secs (1,082,898,748 bytes/sec)

$ dd if=/dev/zero of=testfile bs=32768 count=100000
3276800000 bytes transferred in 2.233185 secs (1,467,321,302 bytes/sec)

Also to consider... your stream of zeros is highly-compressible. Do you have compression turned on for that pool?

Cheers,
Matt

 

[Bidule0hm]

You may want to consider how block size affects your write speed. Here is my four-drive SSD striped mirror...

Code:

$ dd if=/dev/zero of=testfile bs=1024 count=100000
102400000 bytes transferred in 0.776452 secs (131,881,934 bytes/sec)

$ dd if=/dev/zero of=testfile bs=2048 count=100000
204800000 bytes transferred in 0.834657 secs (245,370,267 bytes/sec)

$ dd if=/dev/zero of=testfile bs=4096 count=100000
409600000 bytes transferred in 0.914390 secs (447,948,863 bytes/sec)

$ dd if=/dev/zero of=testfile bs=8192 count=100000
819200000 bytes transferred in 1.115028 secs (734,690,133 bytes/sec)

$ dd if=/dev/zero of=testfile bs=16384 count=100000
1638400000 bytes transferred in 1.512976 secs (1,082,898,748 bytes/sec)

$ dd if=/dev/zero of=testfile bs=32768 count=100000
3276800000 bytes transferred in 2.233185 secs (1,467,321,302 bytes/sec)

Also to consider... your stream of zeros is highly-compressible. Do you have compression turned on for that pool?

Cheers,
Matt

Be careful, this isn't a valid test result as you don't copy the same data quantity each time...

 

[MatthewSteinhoff]

Be careful, this isn't a valid test result as you don't copy the same data quantity each time...

For the purposes of spit-balling an example, it was close enough. But, of course, you are right. Below is an example with the block count adjusted downward to offset the increased block size.

Code:

$ dd if=/dev/zero of=testfile bs=1024 count=100000
102400000 bytes transferred in 0.800337 secs (127,946,129 bytes/sec)

$ dd if=/dev/zero of=testfile bs=2048 count=50000
102400000 bytes transferred in 0.411407 secs (248,901,943 bytes/sec)

$ dd if=/dev/zero of=testfile bs=4096 count=25000
102400000 bytes transferred in 0.231780 secs (441,798,612 bytes/sec)

$ dd if=/dev/zero of=testfile bs=8192 count=12500
102400000 bytes transferred in 0.137086 secs (746,976,816 bytes/sec)

$ dd if=/dev/zero of=testfile bs=16384 count=6250
102400000 bytes transferred in 0.093142 secs (1,099,396,235 bytes/sec)

$ dd if=/dev/zero of=testfile bs=32768 count=3125
102400000 bytes transferred in 0.070131 secs (1,460,128,267 bytes/sec)

Not much change.

Cheers,
Matt

 

[Bidule0hm]

Yep, just a question of lisibility, now we can clearly see there's a more than a 10 ratio between 1k and 32k ;)