Need Pool Setup Advice For 3x NVME M2 Drives

[tsm37]

Hi, I wanted to get your thoughts on how you would setup the pool based on the hardware and use case below:

Hardware

• Motherboard supports 3 x pcie4 nvme m.2 drives, and I have 3 x 2TB pcie4 nvme m.2. drives right now.
• I also have 3 x 1TB crucial SATA SSD drives with DRAM
• Assume no limitations on SATA ports because my mobo has four SATA ports, miniSAS port, and a couple PCIE x16 slots where I can add HBA cards in the future. For what's worth, I'm boot pool is on a single 250 GB SATA SSD.

Use Case

• My personal, family, and certain work files have the highest importance, and I would like these files to be on nvme m2 with the fastest access (I have 10 Gbps setup and I do have external backups). They total 500 GB at this moment and will grow over time.
• I will run a couple window virtual machines; at least one might be on 24/7; I can afford losing these virtual machines and rebuild them as needed.
• In Scale, I will run plex and other applications on TrueCharts

Ask

• For the important files, I can't decide whether to do raidz1 on 3 x 2TB nvme m2 OR do a mirror with 2 x 2TB nvme m2 drives. How would you do it?
• I was told the VMs should be on fast drives. Should I allocate 1 x 2TB nvme m2 just for the windows VMs? I feel that 2TB might be a waste because I will likely have the VMs to write data onto some spinning rusts (I have several WD Red 14TB)
• I was also told the applications should be on fast drives. Should I use the SATA SSD or nvme m2? What redundancy I should be thinking?

 

[john.o]

• For the important files, I can't decide whether to do raidz1 on 3 x 2TB nvme m2 OR do a mirror with 2 x 2TB nvme m2 drives. How would you do it?

I would mirror (2) drives and setup the 3rd as a hot spare since you indicated that your work files are of "highest importance". 3-way mirror would work as well but I think it may be a bit overkill if you have reliable backups. You will have to decide on whether or not the added resiliency is worth the storage capacity hit.

• I was told the VMs should be on fast drives. Should I allocate 1 x 2TB nvme m2 just for the windows VMs? I feel that 2TB might be a waste because I will likely have the VMs to write data onto some spinning rusts (I have several WD Red 14TB)
• I was also told the applications should be on fast drives. Should I use the SATA SSD or nvme m2? What redundancy I should be thinking?

If you are limited to (3) drives, the VM's/Apps can go on the same pool but if it were me, I would buy another 1TB NVME drive and have (2) separate 2-way mirrored pools for both your work files and your VMs/apps.

Just my $0.02

 

[tsm37]

If you are limited to (3) drives, the VM's/Apps can go on the same pool but if it were me, I would buy another 1TB NVME drive and have (2) separate 2-way mirrored pools for both your work files and your VMs/apps.

Thanks for chiming in. For this two separate 2-way mirrored setup, do you mean having two nvme drives in stripe and another two nvme drives mirroring each of the drive in the stripe, essentially achieving the same result like a raidz2?

 

[john.o]

My apologies for the oversight on my part as I misread what you had hardware wise. I mistakenly thought you had a total of (3) 1TB drives but you have (3) 2TB nvme drives and (3) 1TB nvme drives. This actually changes things a bit.

If you are strictly limited to using (3) nvme drives then I would use 2x 2TB in a mirror configuration for your work files and the 3rd 2TB drive will be a cold spare. Since the VMs are not that important, you can just have 1x 1TB drive for your second pool although you will lack redundancy.

Personally, I would just get a PCI-E to M.2 adapter so you can use a 4th nvme drive. If this is possible, I would do this:

- POOL A: 2x 2TB drives in mirror for work files
- POOL B: 2x 1TB drives in mirror for VMs and applications

You are now left with (1) 2TB drive & (1) 1TB drive that you can keep as cold spares in the event of a drive failure.

OR

You could just go with (3x) 2TB drives in a RAIDZ and have everything on (1) pool to keep things simple but I do like the option above of having (2) pools of 2x drives each having a cold spare drive for each pool on standby in the event of a drive failure.

Hope this helps.

 

[tsm37]

Thanks a lot, John. This is very helpful for me to think about and finalize the storage strategy in the coming days.