Need more SSD drives, more S-ATA ports. And 2.5 bays

[dnilgreb]

For block storage, including zvols for VMs, mirrors are better than raidz.

I see. But better how? What are the advantages with a mirror over a raidz? Will it impact VM performance?

 

[Whattteva]

I see. But better how? What are the advantages with a mirror over a raidz? Will it impact VM performance?

VM's and block storage relies heavily on IOPS or they're going to be painfully slow.

As a general rule, mirrors ALWAYS give better performance no matter what your use case is.

 

[dnilgreb]

Right. I did a little reading, and a way to get a little bigger pool, but not lose IOPS seem to be to add several vdevs in the pool. So, what if the VM pool would consist of totally 4 disks , two (mirrored), in each vdev.

Then, if we assume I am using 240 GB drives, the pool would be 480 GB, be mirrored, and have some oomph, would that be correct? Would that be the way to go?

Seems overkill, but disk cost is irrelevant, since they´re just sitting here anyway.

 

[dnilgreb]

Update:
Everything is ordered and I´m waiting for delivery from Art of Server. I do still have the one question about my new VM pool .
I read a little about block storage, IOPS and so on, and I gathered that the more vdevs there are in the pool, the better IOPS get. So how do/would people handle that? The pool will consist of 4 SSDs, max.

Would you go for redundancy, and put in two mirrored vdevs?

Or

Would you go for 4 single drive vdevs, doubling(?) the IOPS? For backup, there are snapshots, replication and all that, if things would go badly.

Is the difference in IOPS noticable? Max simultaneous users = 3. At most times only 1 (me).
Double or half the storage doesn´t matter much here, performace on the other hand matters a lot!

Thoughts, suggestions?

 

[Etorix]

If the VMs do not hold any critical data and you don't bother to reinstall in case of a failure, you may go with striped single drives.
BUT
SSDs have high I/O to begin with, and 1-3 users should be easy to handle. So better go with striped mirrors and enjoy redundancy.

 

[dnilgreb]

Great,
Then it´s settled. Thanks for the input.

 

[dnilgreb]

Just checked with ebay. It´s another 2 weeks before I receive the LSI card. I´ll leave this thread open, and come back with some results once everything is up and running. Or if I need more help along the way... :)
In the mean time, thank you all for all the feedback.

 

[dnilgreb]

If the VMs do not hold any critical data and you don't bother to reinstall in case of a failure, you may go with striped single drives.
BUT
SSDs have high I/O to begin with, and 1-3 users should be easy to handle. So better go with striped mirrors and enjoy redundancy.

Another question about VM performance. The point of using 4 disks instead of just two is IOPS, and increased performance.

BUT

I had a thought, and checked the speeds of my on board SATA ports. I only have two 6gb/s ports. The rest are 3gb/s.
So, what will give the best performance to my VMs?
Two mirrored disks, in single vdev, connected to said 6gb/s ports, or 4 disks, mirrored in two pairs, all of them connected to 3gb/s ports.
Will it make a noticable difference either way?
All disks and cables support 6gb/s.

Does anyone know?

 

[Whattteva]

I had a thought, and checked the speeds of my on board SATA ports. I only have two 6gb/s ports. The rest are 3gb/s.
So, what will give the best performance to my VMs?
Two mirrored disks, in single vdev, connected to said 6gb/s ports, or 4 disks, mirrored in two pairs, all of them connected to 3gb/s ports.
Will it make a noticable difference either way?
All disks and cables support 6gb/s.

Does anyone know?

Well it really depends on the performance of your individual SSD's. If your SSD's don't exceed 3 Gbps (~358 MiB/s or 375 MB/s) transfer rate, then there is no difference between connecting them to the 3 Gbps port vs the 6 Gbps port.

You would need to benchmark or at least look at the specs of your SSD to determine that.

 

[ChrisRJ]

It should also be added that the biggest difference between SATA SSDs and HDDs is not the linear transfer rate but IOPS. Those are responsible for things being orders of magnitude snappier (if that word exists).

 

[dnilgreb]

Thanks,

I went through the SSD drives I have, and one model has much higher IOPS than all the rest.
It´s the Samsung 840 PRO drive.

Features Sequential Read Speed: Up to 540MB/s

Features Sequential Write Speed: Up to 520MB/s

Random Read Speed Up to 100K IOPS

Random Write Speed Up to 90K IOPS

As it happens, it´s also the only one I have two of.

So, now I can choose.
I either make the VM pool with these two drives, connected to my two 6gb ports. That´s mirrored, in a single vdev.

Or, I put those aside, and use my four other drives, of different brands, connected to 3gb ports. The lowest scoring one would be my Kingston drive

Incompressible Data Transfer 191MB/s Read and 142MB/s Write

IOMETER Maximum Random 4k Read/Write up to 85,000/ up to 43,000 IOPS

That´s 2 vdevs, with two mirrored drives in each.

So, which would be the best move? Slower disks, double the vdevs? Or single vdev, but faster disks?

 

[jgreco]

I see. But better how? What are the advantages with a mirror over a raidz? Will it impact VM performance?

Some differences between RAIDZ and mirrors, and why we use mirrors for block storage

ZFS is a complicated, powerful system. Unfortunately, it isn't actually magic, and there's a lot of opportunity for disappointment if you don't understand what's going on. RAIDZ (including Z2, Z3) is good for storing large sequential files...

www.truenas.com

Now I am wondering if my 600 w PSU will be enugh.

Proper Power Supply Sizing Guidance

I've seen about 1,000 threads like this one where people decide that they can power a dozen hard drives off a 360 watt supply. DO NOT DO THIS. I've seen another 1,000 threads where people decide to buy the cheapest power supply that they can...

www.truenas.com

Also, more generally,

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.truenas.com