Help identifying bottleneck

[Eniqmatic]

Hi all,

Was looking for a bit of help identifying what could be a potential bottleneck or perhaps my mis-understanding, both entirely possible :D

First off the specs:
Dell R510
2 x E5645 @2.4GHz
16 GB DDR3
Intel X520-DA2 SFP+ NIC
Dell Perc H310 cross flashed to IT mode
8 x 2TB WD Red disks in RAIDz2.
Freenas 11.1-U4

Bit of back story, I originally had 4 disks in RAIDz1 (I know) but wanted to expand/change this to the current configuration. So replicated all data off, destroyed original pool and replicated all data back. All is working well.

Before I did this however I wanted to see if there would be any speed improvements from adding additional disks, which I assumed (perhaps incorrectly!) that there would be.

So I installed bonnie++ in a jail and benchmarked several times using the following command:
bonnie++ -u root -s 32G -d /tmp -f -b -n 1 -c 4

And it gave the following results:
736MB/s write
542MB/s rewrite
1544MB/s read
1221 IOPS.

Which was all fine and dandy. However after expanding my disks to the 8 x 2TB RAIDz2 configuration, I am still getting identical results as before. This would lead me to think there is a bottleneck. If it had went slightly up or down I would be fine with it but the results are as near as makes no difference identical.

I'm leaning towards the RAM being quite low which I've known for a while and do probably want to get some more anyways, is there anyway to confirm this suspicion? Any other potential bottlenecks?

This isn't really impacting performance at all and I know its a "benchmark" so not really real world scenario, it was more out of curiosity/learning purposes!

Any help much appreciated!

 

[Chris Moore]

A single vdev is basically limited to the speed of the disks that make up the vdev without regard to the number of drives.
To improve pool performance, you need to add more vdevs, which is why pools constructed from mirror vdevs are generally speaking more performance oriented. Not because mirror vdevs are faster, but because you can have more vdevs.
I am not familiar with how the benchmark utility works, but I think the speed of the processor is a potential limit of real life performance.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Eniqmatic]

A single vdev is basically limited to the speed of the disks that make up the vdev without regard to the number of drives.
To improve pool performance, you need to add more vdevs, which is why pools constructed from mirror vdevs are generally speaking more performance oriented. Not because mirror vdevs are faster, but because you can have more vdevs.
I am not familiar with how the benchmark utility works, but I think the speed of the processor is a potential limit of real life performance.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Thanks for the reply.

OK thanks for the explanation, so would that mean that a single vdev consisting of say 4 disks, 8 disks, 16 disks, 32 disks (which I know isn't a good idea but hypothetically) would all give the same performance? I assumed (I guess incorrectly) that adding more disks would increase performance! Not that I would go back and change the configuration, I would still chose the same as I would rather have the space over mirror vdevs.

Thanks again!

 

[Chris Moore]

There is a small amount of scale improvement in read speed if I recall, but it does not improve iops or write and I have been told that the speed is reduced over a certain number of drives.
We have had someone come to the forum for advice that was dealing with a large system that had been configured with a RAID-Z2 pool of 60 drives in a single vdev. I can't remember the details but they were trying to figure out how to find a faulty drive.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Eniqmatic]

There is a small amount of scale improvement in read speed if I recall, but it does not improve iops or write and I have been told that the speed is reduced over a certain number of drives.
We have had someone come to the forum for advice that was dealing with a large system that had been configured with a RAID-Z2 pool of 60 drives in a single vdev. I can't remember the details but they were trying to figure out how to find a faulty drive.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Interesting!

Is there anything you would recommend to try to get more from the current configuration? More RAM?

 

[kdragon75]

Add more disks and split the vdev into 2 6 drive RAIDz2. Used striped mirrors. What are you using it for?

 

[Eniqmatic]

Add more disks and split the vdev into 2 6 drive RAIDz2. Used striped mirrors. What are you using it for?

Unfortunately I'm using the max disk space of this chassis, but I might do this in the future if I feel like it! There is 2 x 2.5 inch internal drives but thats all the space I have.

Its just used for Plex, CIFS shares and I sometimes use it for ISCSI storage which I do appreciate mirror vdevs would be much better suited for. Like I say, the performance is totally fine for me, just wondered if there was anything I could do without drastic hardware changes!

 

[kdragon75]

Nope, no magic bullets here.

 

[Eniqmatic]

Nope, no magic bullets here.

Cool, thats all I wanted to know, thanks for the input!

 

[Stux]

How many dimms have you got making up your 16GB?

 

[Eniqmatic]

How many dimms have you got making up your 16GB?

4 x 4GB DIMMS, 2 per socket is all, so have space to expand!