Recommended setup

[Dudde]

Our new fileserver will in the beginning start with 8x 6TB WD Red PRO drives.
We will configure them with 2 disk in a mirror, in this case we will get 24TB of storage, and 50% of that will we be leaving untouched for snapshots etc.
So 12TB of storage. This will be the first VDEV and later we want to grow with and additional VDEV pretty much identical to the one above.
How much RAM do you guys think i need, 64GB or 128GB? I understand that a SLOG is per pool, in this case we will start with one pool where we will use 11TB and share 1TB for a customer. Do you guys think a SLOG is required here? The fileserver will be used for storing mostly backups. We will replicate the backups to another server in a different location. It's quite alot of backup jobs from lots of different customers and machines, so the server will more or less constantly read and write data. Would I maby even need an L2ARC?

If i need a SLOG, witch one should you recommend? Do i need two or can we buy let's say an Intel S7310 200GB and partition 100GB for the first pool and later use the remaining 100GB for a future VDEV and pool?
Let me hear how you guys would setup this configuration.

 

[Robert Trevellyan]

This will be the first VDEV ...

What you described is a pool, not a vdev.

64GB or 128GB?

Start with 64GB and upgrade if performance is limited by lack of RAM.

Do you guys think a SLOG is required here?
...
Would I maby even need an L2ARC?

Seems unlikely. If you've maxed out the RAM and read performance is inadequate, consider an L2ARC. If you need sync writes and write performance is inadequate, consider an SLOG.

 

[Arwen]

Like Robert said, a pool is made up of vDevs. In your case, you plan to start with 4 x Mirrored vDevs in your pool.

So, if you need to grow space, another pair of disks, Mirrored, is one way to go. Or you can replace one pair of disks
in a Mirror, (one at a time, wait til re-sync is done), with larger disks, and then that single vDev will grow.

In some ways, using very large disks, like 6TB in a 2 way Mirror is less than optimal. During replacement, the single
remaining disk in the mirror is hit hard. If there is a fault in that remaining disk, then you have data loss. That's why
many of use use RAID-Z2 vDevs. We can loose any 2 disks without data loss. That said, there are specific reasons to
use Mirrored vDevs, like Virtial Machine storage or increased IOPS. Only you can decide.

 

[Dudde]

Sorry, you guys are correct of course. From what information i could find a SLOG would be a good choice of you will run a database or VM's. This is true in my case.

In some ways, using very large disks, like 6TB in a 2 way Mirror is less than optimal. During replacement, the single
remaining disk in the mirror is hit hard. If there is a fault in that remaining disk, then you have data loss. That's why
many of use use RAID-Z2 vDevs.

From what information I could find resilvering a mirrored vdev is much less painful and time consuming then doing it on a RaidZ2 or Z3?
Soruce: http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

 

[Robert Trevellyan]

SLOG would be a good choice of you will run a database or VM's

If your workload is database or block storage for VMs, you probably want sync writes enabled, and that can result in poor performance without a suitable SLOG.

Soruce

That article is full of misinformation. However, mirror vdevs do offer great flexibility and have performance advantages for some workloads.
http://constantin.glez.de/blog/2010/01/home-server-raid-greed-and-why-mirroring-still-best
https://forums.freenas.org/index.ph...d-why-we-use-mirrors-for-block-storage.44068/

 

[Arwen]

...
From what information I could find resilvering a mirrored vdev is much less painful and time consuming then doing it on a RaidZ2 or Z3?
Soruce: http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

On the issue of re-silvering a mirror vDev. Yes, it's less painful to the whole pool. Except you are vulnerable to a
second disk failure taking out your entire pool.

That said, if you have a free disk slot, (which includes a SATA port), then you can perform a ZFS disk replacement
with both original disks in place. This is only useful if one of the original mirror disks is just failing, (bad blocks),
not completely failed. This gives you the best chance of getting a clean re-silver.

For example, say disk 1 has some bad blocks but disk 2 seems okay for now. You add disk 3 as a replacement to
disk 1. During the re-silver process bad blocks are detected in disk 2, but they are different blocks than disk 1. So
you still have 100% coverage of your data. Not the case if you physically removed disk 1 to put in the replacement
disk. This live replacement is something I have wanted for at least 16 years, (not that I really have used it yet).