Hybrid pool configuration for Supermicro 6049P-E1CR45L+

KevinM

Contributor

Sep 10, 2020

We have been using 36-bay Supermicro boxes for a while and are happy with them, but for the latest pair of FreeNAS boxes we are looking at the 6049P-E1CR45L+ due to its larger capacity and optional NVMe storage. My big question is with the hybrid storage pools available with TrueNAS CORE, how much SSD space will I need for a metadata VDEV if I have 5 9-wide RAIDZ2 VDEVs? These systems will primarily be used for Veeam backups but will also be hosting misc NFS and iSCSI storage. Thanks in advance!

Provisional storage layout:
45 x 16tb Nearline SAS
768gb memory
2 x 240gb s4510 SATA drives (OS)
2 x 2tb s4510 NVMe (L2ARC)
2 x 375gb Optane NVMe (SLOG)
2 x 7.6tb NVMe (metadata)

Last edited: Sep 10, 2020

TrueNAS-13-STABLE
U-NAS NSC-810A case
Supermicro X9SCL-F-O motherboard
Intel e3-1230v2 CPU
32 GB ECC RAM
Intel onboard 2-port NIC (management)
Intel 4-port i340-T4 NIC (iSCSI and NFS)
IBM M1015 HBA
16 GB SATA DOM (OS)
6 x WD Red 6 TB SATA drives (RAIDZ2)

Production #1 and backup #2
TrueNAS-13-STABLE
Supermicro E1R36N 36-bay SAS2 chassis
2 x Intel e5-2620v2 CPU
256 GB ECC RAM
1 x 4-port 1 GB Intel i350 NIC (management)
1 x 1-port 10 GB Intel X520-LR1 NIC (replication)
2 x 2-port 10 GB Chelsio T520-CR NIC (iSCSI and NFS)
LSI 9207 HBA
2 x 16 GB Sandisk Cruzer Fit (OS)
2 x 200 GB Intel s3710 SSD (ZIL)
2 x 400 GB Intel s3710 SSD (L2ARC)
36 x 3 TB Seagate SS650 Nearline SAS drives in 6 x 6-wide RAIDZ2

Production #3 and backup #4
TrueNAS-13-STABLE
Supermicro E1CR36L 36-bay SAS3 chassis
2 x Intel e5-2623v3 CPU
256 GB ECC RAM
1 x 2-port Intel X540 (management, gigabit)
1 x 1-port Intel 10 GB X520-LR1 (replication)
2 x 2-port Chelsio 10 GB T520-CR (iSCSI and NFS)
2 x 200 GB Intel s3710 SSD (OS)
400 GB Intel PCIe p3700 NVMe SSD (ZIL)
1.2 TB Intel PCIe p3700 NVMe SSD (L2ARC)
36 x 8 TB HGST Nearline SAS drives in 6 x 6-wide RAIDZ2

Production #5 and backup #6
TrueNAS-13-STABLE
Supermicro E1CR36L 36-bay SAS3 chassis
2 x Intel e5-2609v4 CPU
512 GB ECC RAM
1 x 2-port Intel X540 (onboard, management)
1 x 4-port Intel XL710 10 GB (iSCSI, NFS, replication)
2 x 240 GB Intel s4500 SSD (OS)
2 x 2 TB Intel P4600 NVMe (ZIL and L2ARC)
36 x 12 TB HGST Nearline SAS drives in 6 x 6-wide RAIDZ2

Production #7 and backup #8
TrueNAS-13-STABLE
Supermicro E1CR45L+ 45-bay SAS3 chassis
2 x Intel Xeon Silver 4210R CPU
768 GB ECC RAM
1 x 2-port Intel X540 (onboard, management)
1 x 4-port Intel XL710 10 GB (iSCSI, NFS, replication)
2 x 240 GB Intel s4500 SSD (OS)
2 x 375 GB Intel 4800X NVMe (ZIL)
1 x 4 TB Intel 4510 NVMe (L2ARC)
3 x 7.6 TB Micron 9300 NVMe (Special)
45 x 16 TB Seagate Nearline SAS drives in 5 x 9-wide RAIDZ2

Production #9 and backup #10
TrueNAS-13-STABLE
Supermicro E1CR60L+ 60-bay SAS3 chassis
2 x Intel Xeon Silver 4210R CPU
1024 GB ECC RAM
1 x 2-port Intel X550 (onboard, management)
1 x 4-port Mellanox SFP-28 25 GB (iSCSI, NFS, replication)
2 x 240 GB Intel s4500 SSD (OS)
2 x 400 GB Intel 5800X NVMe (ZIL)
1 x 6.4 TB Micron 7450 NVMe (L2ARC)
3 x 12.8 TB Micron 7450 NVMe (Special)
60 x 22 TB Seagate Nearline SAS drives in 5 x 9-wide RAIDZ2

Important Announcement for The TrueNAS Community.