Hello FreeNAS-Community!
We are a group of media & it students who are into filming and media creation. Our next project is a replacement of our former 18 TB FreeNAS desktop pc with a futureproof ‘real’ server setup. The old setup was configured with the help of this forum (under another account) years ago. We appreciated the help at that time very much and we thought coming back here would be a good idea.
In a first step we want to install 24 HDD with 8TB each. We got them as a donation and intend to use the whole server as a video archive. For editing purposes, the specific project data is copied to a workstation with SSDs (no live-editing on the storage server). As a link to transfer the data between workstation and server we would like to use one (or later two linked) 10Gbit cards.
Doing the math’s a SAS2-Link (=24Gbits) per 12 HDDs should be sufficient: We thought about using either one HBA with 16xSAS2-Lanes or two with 8xSAS2-Lanes each. Using 4 lanes for every 12HDDs would allow us to connect up to 48 HDDs (with a 16x HBA).
We are a group of media & it students who are into filming and media creation. Our next project is a replacement of our former 18 TB FreeNAS desktop pc with a futureproof ‘real’ server setup. The old setup was configured with the help of this forum (under another account) years ago. We appreciated the help at that time very much and we thought coming back here would be a good idea.
In a first step we want to install 24 HDD with 8TB each. We got them as a donation and intend to use the whole server as a video archive. For editing purposes, the specific project data is copied to a workstation with SSDs (no live-editing on the storage server). As a link to transfer the data between workstation and server we would like to use one (or later two linked) 10Gbit cards.
- Is it wiser to use a 4x Z2 with 6HHDs (4/2) or a 2x Z3 with 12 HDDs (9/3) setup? Following the awesome tool from https://jro.io/r2c2/ the two Z3 pools perform significantly better – but we don’t think that resilvering etc. is considered there. So our question is more practical than theoretical.
- Is there a possibility to link aggregate 2x 10Gbits cards without any router (=direct connect to a workstation with also 2x 10Gbits cards)?
3. Any general recommendation on using multiple smaller chassis or less larger ones?
4. As far as our knowledge reaches a sudden power loss is not affecting the stored data but only files transferred at that moment. So is a dual PSU for the mainboard and storage units and/or a UPS considered as a must?
Doing the math’s a SAS2-Link (=24Gbits) per 12 HDDs should be sufficient: We thought about using either one HBA with 16xSAS2-Lanes or two with 8xSAS2-Lanes each. Using 4 lanes for every 12HDDs would allow us to connect up to 48 HDDs (with a 16x HBA).
5. Are there any reasons or benefits for us to use SAS3-HBAs or SAS3-Expanders in our setup (up to 48 drives)? Is there such a thing as a switch with an external SAS3-Port (between JBOD and mainboard) and internal SAS2-Ports (to the drives/backplane)?
6. Is there a recommendation on (affordable) specific combinations of HBAs and SAS expanders? The SAS 9201-16e is looking good to our eyes so far – but which affordable expanders do you recommend?
Based on this numbers we would need a mainboard with up to 2 PCIe slots (2.0 or better 3.0?) for the HBAs and maybe an additional PCIe if the mainboard doesn’t come with 2x10Gbit ethernet ports. It further needs sufficient ECC RAM slots. In general we like to stay flexible for further upgrades and therefore more slots are appreciated.7. Any recommendations for the mainboard / processor? We are not so much into hardware to be able to choose an appropriate but affordable solution (we don't have to pay for electricity, so energy efficience is not necessarily in our focus at the moment).
8. The rule of thumb for FreeNAS says 1 GB of RAM for every TB of storage. Do you think 64GB of RAM would still be sufficient for 100TB+ storage?
Last but not least: We receive some money from sponsors supporting our build, but we also have to pay parts on our own. Our first aim is to have a reliable setup, but the second is an affordable system. Many thanks to all who help us to achieve this goal. Please feel free to pick the questions you want to answer and give a reference by the number. We will keep you up to date in the build process 