Upgrade an existing 120TB Video editing System to TrueNas / Hardware Upgrades

[stvmyr]

Hi there!

I’m currently planing to switch an office infrastructure from a CentOS deployment to TrueNAS Scale.

The system is used to store raw Video footage. Editors will edit from it. The Speeds and latency of the hard drives are sufficient for the work they will do. But the end goal is to get 100% performance out of the existing hardware. The total Capacity after RAID is currently about 90TB. Its about 50 - 60% filled. There is no dedicated caching installed yet.

The Servers are on a 2x10G LAG. All clients (Mac and Windows) are connected with copper based 10G networking. There is a maximum of 10 clients that access a single share. In the worst case 4-6 clients are actively working with large files on the system.


I worked with ZFS before, but not in a „spicy“ environment like that.

The current setup looks like that:
2 Exactly same machines where all files of the first system is manually copied to the second one daily.


The machines looks like that:
Supermicro CSE-826 Chassie
Supermicro X11SSH-F Mainboard with Intel Xeon E3-1230v6
Supermicro Dual 10G X550 Based Network Card
32GB RAM (2x16GB DDR4-2400 Unbuffered)
Avago SAS 9361-4i Hardware RAID Controller
12x10TB HGST Drives in a Single RAID6
2x256GB Micron SATA SSD in RAID1 as boot drive


This is my plan to change the system over to TrueNAS
- Swap out the Hardware RAID controller to an HBA like an SAS 9311-8i with IT-Mode firmware.
- Build two vdevs with 6 drives each in RAID-Z1.
- Upgrade the RAM to 64GiB if necessary.
- Install TrueNAS on the existing RAID1.


So my questions are:
- Is 64 GiB RAM enough for a 120TB RAW Capacity HDD System? (The CPU does not support more than that)
- Is an NVMe based L2ARC a good addition for streaming applications like that?
- Do I run into CPU bottleneck with a single 4-Core CPU
- is there a bottleneck running 12 drives of an SAS Backplane, where the drives are multiplexed to a single SFF-8643 cable?
- what’s you Pro-Tipps for an enviroment like that?


Thanks!
Steven

 

[nabsltd]

- Is 64 GiB RAM enough for a 120TB RAW Capacity HDD System? (The CPU does not support more than that)

This is where I'd be really worried, with no expansion possible. I'd look into the X11SRL-F motherboard and CPU that fits. That would give you up to 1TB of RAM, an on-board M.2 socket, and 36 PCIe lanes across 6 slots...plenty for HBA, NIC, or NVMe cards.

- Do I run into CPU bottleneck with a single 4-Core CPU

Since it has 8 threads, you probably will be fine. If you don't upgrade the motherboard, you could always go for a faster E3-1200v6 series that would be a drop-in replacement.

- is there a bottleneck running 12 drives of an SAS Backplane, where the drives are multiplexed to a single SFF-8643 cable?

You don't say what backplane you have, but the SAS3 version definitely would have no issue, as that would be 48Gbps for 24 drives, allowing 200MB/sec per drive. Plus, you can use both connectors from any of the -8i cards to connect to the backplane and get double the bandwidth if you are using SAS drives. This might not work with SATA drives, since they only have one port.