Recommendations on 140TB+ System

[BrainDrain]

Hi;

as longer i read around FreeNAS i get unsecure more and more if its the right product for our purposes...
Usually i build up large Systems using Ubutu LTS or MS Server - but i want to give FreeNAS a try because it looks quit easy to
configurate and maintain...

Actually i have a new Server with this hardware:
2x Intel Xeon E5-2603v4
32GB ECC Reg DDR4 2666
some SSD Bootdrive
16x 12 TB SATA III WD 3,5" 7.2k (512e) - 14 of them running as a RAID 6 on a
Broadcom (LSI/Avago) MegaRAID 9361-8i + Cache Vault = 140TB
10GBit LAN

The Server should run some a Samba Shares for 10-15 People using it for transferring big data between 20 Workstations. The Server should also be an "anker" for a large Backup /Archive solution using bacula or bareos (+LTO Tape Library) on some other server in the same area.

What do you think - is that machine above strong enough to server fast and big data transfer between the workstations and to the a backup-server? And btw - why is Freenas that hungry when it comes to Diskspace <-> RAM?

Thxs!

 

[Chris Moore]

FreeNAS / ZFS uses RAM for cache. ZFS, the file system, needs direct access to the disk space, so that hardware RAID card would need to be replaced with a SAS HBA and ZFS will manage the disks.

 

[BrainDrain]

Thank you... good to know - unfortunately hardware raid has some important features in our setup...

 

[Chris Moore]

Thank you... good to know - unfortunately hardware raid has some important features in our setup...

Like what?

 

[BrainDrain]

Speed/Cache Fault in case of power failure/SSD Caching and that all without using CPU time (also and special on rebuilds)

 

[Chris Moore]

Speed/Cache Fault in case of power failure/SSD Caching and that all without using CPU time (also and special on rebuilds)

I suggest that you might want to learn about ZFS.

 

[BrainDrain]

I will - thanks anyway..

 

[Chris Moore]

I will - thanks anyway..

There are features of ZFS that makes it better than a hardware RAID card. It does take a different hardware set and it is not easy to get high performance on a small system.

 

[Yorick]

@BrainDrain, that’s a big enough decision that time spent to learn what you’d be getting yourself into will pay off tenfold.

Two books helped me a lot: FreeBSD Mastery: ZFS and from the same series, Advanced ZFS.

The first teaches basic concepts.
The second talks about performance optimization and corner use cases.

ZFS is resilient to power loss. It can, unlike a RAID controller, detect data corruption regardless of the source, even when the disks show healthy. It caches in RAM and, when configured to do so, SSD. Caching is for repeat reads, of data or metadata. Size of your dataset and access pattern definitely comes into play here.
For sync writes, it can cache the writes. Async writes, that’s not needed.

There’s a lot to like about ZFS; there’s also a lot to learn. I have to say that just checking “RAID 6” and “write back when BBU healthy” on an LSI controller takes a lot less thinking. It also does nothing for me should the data get corrupted - I better hope I catch it before my backups are too old to do me any good.