Phlesher
Dabbler
- Joined
- Jan 9, 2022
- Messages
- 16
Hello all! I have spent quite a bit of time researching to discover the "best practices" around which drive types and quantities I should be purchasing for a new build. I've found a lot of information, but it's scattered across years ranging from 2011 through 2020, and in many cases it's become clear that some information has been outdated by new TrueNAS features and performance improvements, not to mention hardware leaps that have happened even in the past few years.
So, I figured it would be better to ask for some guidance. If there's already a very clear up-to-date post or article somewhere, please feel free to tell me to RTFM. :)
My use case: I would like my TrueNAS running as a VM, and I want to host at least two distinct pools - one for other VM drives (boot + application and associated data) and one for larger storage of media (audio, video, possibly images, possibly other write-once-read-many types of things). I'm planning a 10Gb NIC and would like that to end up being the bottleneck for local file transfer. The goal is to get fast write speed to both pools, with fast random access on the VM pool and fast sequential reads on the bulk storage pool. I have a 24-bay SAS3 backplane I'll be using for hot swappable SAS3 drives (though if there's some good reason for it, I could go SATA3 instead) hosted by a HBA card passed through to the TrueNAS VM. I'll have dual Xeon v4 (24 total cores) with 128 GB ECC to start. So far, I hope all of this is sensible and not too far afield from what most people around here do!
My next thing to figure out is the exact drive configuration I'll start with, which is where things start to get a little fuzzy for me. I understand that there are a few things I'll need:
Boot/OS drive for the VM host -- I'll likely use Proxmox and configure twin SSD's for this purpose, in a ZFS mirror. My understanding is that this mirror can be relatively small, just big enough for Proxmox and whatever stuff it writes, and the TrueNAS VM boot drive and whatever stuff it writes. I'm not clear how big this is exactly... maybe 100 GB is overkill already?
Pool A (VM's, random read/write) -- 2-4 SSD's in a ZFS mirror. I don't think this will need to be very large to start, as I only have plans for a few VM's with modest sizes at the moment. So, I assumed I would get away with perhaps just 2 drives of 1 TB each, to give me a 1 TB mirrored array. I need to decide which specific SSD's to use for best performance and reliability (where if there's contention between the two, I'll choose reliability every time).
Pool B (large storage, sequential read/write) -- 6 SSD's (or should it be spinners?!) in a ZFS RAIDZ2 vdev. My understanding is that this is a good starting configuration that performs well without giving up too much storage space to parity. I might want to start this pool larger... let's say 4 TB per drive to end up with a 16 TB pool size (80% maximum being filled as the guideline leaves me with around 12.8 TB useable space). Again, I need to decide on the specific SSD's (or spinners) to use and want reliability first, performance second.
SLOG -- I'm not clear whether this is necessary, especially if I have SSD's. If I need it, why? If I need it, what considerations should I have about choosing a drive for it? I've read about Optane cards for this purpose, though if I got one, it seems like it would be big enough to act as SLOG, L2ARC, and maybe even the host boot drive too.
L2ARC -- Again, not clear whether this is necessary for my use case, and if so, what drive to use.
Any pointers to the most recent state of play on these issues would be very appreciated!
So, I figured it would be better to ask for some guidance. If there's already a very clear up-to-date post or article somewhere, please feel free to tell me to RTFM. :)
My use case: I would like my TrueNAS running as a VM, and I want to host at least two distinct pools - one for other VM drives (boot + application and associated data) and one for larger storage of media (audio, video, possibly images, possibly other write-once-read-many types of things). I'm planning a 10Gb NIC and would like that to end up being the bottleneck for local file transfer. The goal is to get fast write speed to both pools, with fast random access on the VM pool and fast sequential reads on the bulk storage pool. I have a 24-bay SAS3 backplane I'll be using for hot swappable SAS3 drives (though if there's some good reason for it, I could go SATA3 instead) hosted by a HBA card passed through to the TrueNAS VM. I'll have dual Xeon v4 (24 total cores) with 128 GB ECC to start. So far, I hope all of this is sensible and not too far afield from what most people around here do!
My next thing to figure out is the exact drive configuration I'll start with, which is where things start to get a little fuzzy for me. I understand that there are a few things I'll need:
Boot/OS drive for the VM host -- I'll likely use Proxmox and configure twin SSD's for this purpose, in a ZFS mirror. My understanding is that this mirror can be relatively small, just big enough for Proxmox and whatever stuff it writes, and the TrueNAS VM boot drive and whatever stuff it writes. I'm not clear how big this is exactly... maybe 100 GB is overkill already?
Pool A (VM's, random read/write) -- 2-4 SSD's in a ZFS mirror. I don't think this will need to be very large to start, as I only have plans for a few VM's with modest sizes at the moment. So, I assumed I would get away with perhaps just 2 drives of 1 TB each, to give me a 1 TB mirrored array. I need to decide which specific SSD's to use for best performance and reliability (where if there's contention between the two, I'll choose reliability every time).
Pool B (large storage, sequential read/write) -- 6 SSD's (or should it be spinners?!) in a ZFS RAIDZ2 vdev. My understanding is that this is a good starting configuration that performs well without giving up too much storage space to parity. I might want to start this pool larger... let's say 4 TB per drive to end up with a 16 TB pool size (80% maximum being filled as the guideline leaves me with around 12.8 TB useable space). Again, I need to decide on the specific SSD's (or spinners) to use and want reliability first, performance second.
SLOG -- I'm not clear whether this is necessary, especially if I have SSD's. If I need it, why? If I need it, what considerations should I have about choosing a drive for it? I've read about Optane cards for this purpose, though if I got one, it seems like it would be big enough to act as SLOG, L2ARC, and maybe even the host boot drive too.
L2ARC -- Again, not clear whether this is necessary for my use case, and if so, what drive to use.
Any pointers to the most recent state of play on these issues would be very appreciated!
