Truenas Optimization Question

[mountaintime]

Hi all,

I am hoping for some advice. I have been running since Freenas 11 (now 12.0U1.1) a system of:
ASRock X470D4U, Ryzen 3600, 64 Gb ECC Ram, (2) WD Black 512 gb NVME, (6) WD Red 8 TB Sata Drives, (2) generic 128 gb Sata SSD (as Boot Pool), (1) Cold Spare WD Red 8 TB. Intel X550 Dual 10 GBE Ethernet. (I have 10 GBE in my house router, PCs, Security Camera NVR).

I know this is not server grade hardware, but I am still learning, and I have budgets to contend with (I got 4 of the WD Red drives as gifts). It had been rock solid until today as well.

I had setup the two WD blacks in a mirrored pool running VMs, (6) WD Red in a RaidZ2 pool, and the (2) Generic Drives in a mirrored boot pool. Today, for some reason the 6 disk pool took a dump on one drive, and the machine will not resilver with the new drive or the old drives (all drives test perfect on a different PC), it prefers to reboot itself in <5 minutes instead (I've searched the forums, saw different issues, tried several things but think its faster to start over). With the failed drive, I have taken it to another PC and run all the tests, it shows no error.

Currently, my only thought is to back up all the data to another set of drives and start over. I started to think having the mirrored SSD pool is probably not using the resources to the best performance it could be as most of the data will be in the main pool.

With that, I am wondering if there are recommendations on how to optimize performance of the main pool using these resources. I regularly transfer large photo and video assets where saturating the 10Gbe would be preferred. The 6 disk array tested out somewehre around 580-590 mbps if I remember correctly, but when i tested the SSD pool it would saturate the full speed of the connection if the files were large enough.

Since I am about to just start over, is there a better configuration for these resources? or if I added one or two more pieces would that be optimal?

I appreciate your advice.

 

[sretalla]

You could start by sharing the output from zpool status -v and zpool list -v

 

[mountaintime]

Unfortunately the system would not stay running long enough to execute any commands, it looks like I will be starting over. As I said I am learning and made some poor choices by encrypting the pool, and when running a fresh install, they would not re-import. Most items were backed up or can be restored, so its not a total catastrophe.

As I mentioned the System Specs are:
ASRock X470D4U
Ryzen 3600
64 Gb ECC Ram (From MB QVL),
(2) WD Black 512 gb NVME,
(6) WD Red 8 TB Sata Drives,
(2) SP 128 gb Sata SSD (as Mirrored Boot Pool),
(1) Cold Spare WD Red 8 TB.
Intel X550 Dual 10 GBE Ethernet.

Previously I had the SSDs in a mirrored pool, but really was not doing much with them, so I did not benefit from them.

I am wondering if it would be more optimal to use the NVME such as ZIL/Cache or not mirror the boot pool to create faster transfers so that it saturates the 10gbe. Previously the RaidZ2 pool tested somewhere around 500s Mbps, and file transfers for video (25gb-100gb) would slow considerably after a short time.

Best Regards,