Layout for 60 SAS drives and server tweaks

[abhijeet1301]

Experts,

I am currently running Refurb Supermicro 6048R-E1CR72L (2 X E5-2683 v3 and DDR4 256GB RAM 10GBE) with 1 pool of VDEV - 5 x 10-wide RAIDz2 (Running as backup repository) and another with 1 X 8 RAIDZ2 with 2 hot spares (Running as NFS datastore for keeping ISO files and such). Currently I am having issues with this server and I am upgrading to Refurb SuperStorage 6049P-E1CR60L (2x Intel Xeon 6132 and 512GB DDR4 RAM 25GBE), I am also getting 6 X Optane SSDPE21K375GA01 for SLOG and L2ARC. All the spinning rust is 60 X WDC WUH721818AL5201. This server will now also run ISCSI to server storage for VMs as well. my question is -
- What would be the best layout for backup target? Currently I backup over 400 VMs daily with twice full backups and rest incremental in a week and I keep last 15 versions available for daily, 4 Weekly, 6 Monthly and 2 Yearly (This is all due to compliance reasons).
- Do I allocate the Optane like 2XL2ARC, 2XSLOG and 2X Metadata?
- What changes needs to be done on BIOS side? I have heard since I am using 2 procs NUMA sometimes causes issues? Do I disable Hyperthreading, VT-d,etc?

 

[Davvo]

What are your requirements? Size? Resilience? Speed?
Also, the metadata special vdev must have the same redundancy of the pool since if you lose it you lose the entire pool; L2ARC and SLOG do not have this requirements (you can have just one since they are striped).

The following thread might be helpful to you.

Storage for 60 disks

Help to choose the optimal solution for creating a storage for 60 disks. supermicro SSG-6049P-E1CR60L server: 2 x Intel CLX 4210R processors (10C/20T); 64 GB of RAM (16 GB x4), and there is no budget to increase the amount of RAM in the next six months or a year. I would like 512 GB of RAM...

www.truenas.com

 

[Etorix]

Also, for the iSCSI part:

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com

- Do I allocate the Optane like 2XL2ARC, 2XSLOG and 2X Metadata?

SLOG is possibly useful for iSCSI.
It is not apparent whether L2ARC would be useful with 1/2 TB RAM. And having a L2ARC barely larger than RAM (if striped!) is… strange.
L2ARC and metadata vdev overlap to some extent, so having both looks like a misallocation of resources.

- What changes needs to be done on BIOS side? I have heard since I am using 2 procs NUMA sometimes causes issues? Do I disable Hyperthreading, VT-d,etc?

I have no clue about advanced BIOS settings for Xeon Scalable. But why do you think that disabling HT or VT-d could help in any way?