First FreeNAS build for storage & video work

[Voltelord]

Hi guys,

I would like to ask you to check my parts and see, if I chose everything correctly and you agree with my hardware picks. Any suggestions for other parts etc. are appreciated. Thank you

This FreeNAS is for home use, media storage etc. but I occasionally also shoot film and create animations with lots of data, that I would then want to access and work on completely on the NAS (Premiere, After Effects, 4K etc.). I have lots of loose HDDs, but have accumulated too much data over the years and want a more professional environment.

Rendering will also happen on the NAS so I can fully work on my main machine.

I wanted to start with 2x8TB and then extend to 8x8TB in the following years. I also plan on using Raid 10.

I guess an SSD Cache would be good for the current project I'd be working on? Haven’t used anything else but NVMes lately and can’t go back to “old” speeds I guess ;)

Does a bigger boot SSD suffice, or would I be better off having something like a cache SSD for that purpose?

I would also consider a 10 Gbit NIC at some point in the future, so I am not restricted to the 125 MB/s bandwidth and can use the SSD and also the pretty fast HGSTs to their full potential.

Anyways this is the list for the standard setup:

Mainboard:
Supermicro Micro ATX DDR4 LGA 1151 Motherboards X11SSM-F-O
https://www.amazon.com/Supermicro-M...531376481&sr=8-1&keywords=Supermicro+X11SSM-F

HDD:
2x HGST Deskstar 8TB (to be upgraded to 4,6,8 in the next couple of years)
https://www.amazon.com/HGST-Desksta...=UTF8&qid=1531377211&sr=8-3&keywords=hgst+8tb

RAM:
1 x Crucial 16GB Single DDR4 (to be upgraded accordingly with HDD to 32,48,64)
https://www.amazon.com/Crucial-Sing...&qid=1531376651&sr=8-4&keywords=ddr4+ecc+16gb

CPU:
Intel Xeon E3-1230 v6
https://www.amazon.com/Intel-Xeon-E...1383851&sr=8-1&keywords=Intel+Xeon+E3-1230+v6

SSD:
Samsung EVO 250 GB
https://www.amazon.com/Samsung-500G...sr=1-1-spons&keywords=samsung+evo+250gb&psc=1

Case:
Any case for a MicroATX board, that has enough room for the final 8 HDDs

PSU:
EVGA SuperNOVA 750 G3
https://www.amazon.com/EVGA-SuperNO...1531378474&sr=1-1&keywords=evga+supernova+750

UPS:
Tripp Lite 1300VA UPS Battery Back Up
https://www.amazon.com/gp/product/B...d8-5e84-b87d-4a65ff94e801&pf_rd_i=764572&th=1

Thanks a lot :)

 

[garm]

Some quick thought
You don’t have enough RAM for a L2ARC and the SSD you have don’t have battery backup so it is unsuitable for SLOG. A SLOG is only for speeding up sync writes so make sure that fits your workflow if you do get one.
But a single mirror vdev of those drives might not give you the expected IOPS. Especially if you are comparing with working locally on NMVe.

The cost of building a system for working on directly is high. It’s a hard threashold to justify. A NVMe will be faster than 10 GbE and setting up a storage server capable of saturating 10 GbE ain’t gonna be cheap.

I suggest you keep reading on ZFS, the forum is full of useful information, and do the math before you start purchasing.

You don’t need a boot drive bigger then 120 GB

Going for DDR3 is proabobly not going to impact performance and cut back on cost. Unless you need some other feature on the newer boards.

 

[Voltelord]

Thank you for pointing me in the right direction garm. I initially thought working on it directly could become expensive and is maybe not worth it, but I still wanted to consider it. I could also just keep working on my main machine and use the NAS as storage only.

Do I really need a battery backup SSD for SLOG, when I have a UPS?

Ok, so from what I’ve read now, it is better to max out RAM before even considering an L2ARC device, correct?

So my workflow would be as follows:

Get only 1 boot drive 120 GB SSD and the 2x8TB HDDs.
Get 32 GB of DDR3 RAM initially.

Set everything up & play around.

See how everything works. Test IOPS.

If I am happy - keep it like that and just add more HDDs and RAM.

After that consider a L2ARC device.

Work on locally on NVMe.

Does this sound good? :)

Thank you very much for your help

 

[IQless]

Do I really need a battery backup SSD for SLOG, when I have a UPS?

(Someone correct me if I'm wrong)

UPS only saves you from power outages or problems from the power grid, but when the machine turns off due to something else... then it won't save you :\
eg. Kernel Panic, ECC memory errors, PSU has gone haywire.

It's in these circumstances that a PLP SSD might save you.

 

[Chris Moore]

I would then want to access and work on completely on the NAS (Premiere, After Effects, 4K etc.)

The need for speed is the problem with this. It will be slower than a local spinning disk if you are on 1Gb Ethernet.

Rendering will also happen on the NAS so I can fully work on my main machine.

Have you investigated how this is supposed to happen?

I guess an SSD Cache would be good for the current project I'd be working on? Haven’t used anything else but NVMes lately and can’t go back to “old” speeds I guess ;)

You can't access the speed of an NVMe disk over 1Gb Ethernet because the network is too slow.

Does a bigger boot SSD suffice, or would I be better off having something like a cache SSD for that purpose?

FreeNAS can actually be installed on an 8GB drive if I recall correctly. I am using a mirrored pair of 40GB spinning disks for the boot pools in both of my home FreeNAS systems. It doesn't serve any purpose to have either a big or fast drive for the boot drive in FreeNAS. The only reason that a SSD is the current 'recommended' boot media is for reliability. A new SSD should last 6 to 10 years if it isn't defective from the start. I have a system right now that is pushing 6 years on the boot drive and it hasn't had any trouble yet, but some models are better than others. Just don't spend extra for big and fast when reliable is what you need.

I would also consider a 10 Gbit NIC at some point in the future, so I am not restricted to the 125 MB/s bandwidth and can use the SSD and also the pretty fast HGSTs to their full potential.

If you don't get 10Gb to the desktop, you will not be able to use the NAS the way you are saying you want to use it but even if you do, you will probably still be disappointed with the speed. It will not be as fast as a NVMe SSD because, even at 10Gb speed, the network could still be a choke point.
Even if the network wasn't a bottleneck, this would be.

HDD:
2x HGST Deskstar 8TB (to be upgraded to 4,6,8 in the next couple of years)

A single vdev (mirror) is going to be limited to the speed of a single spinning disk. That is as fast as you will be able to go, no matter the networking. We have a storage system at work that has around 224 drives (if I recall correctly) and the reason for all the drives is not capacity, they are only about 900GB each; they have that many drives for the speed, because each drive is only able to transfer data at a certain speed. You get more speed by adding more drives. With just some "napkin math", I think you would need (using mirrors) a minimum of 8 drives to come close to filling a 10Gb network pipe, but don't quote me on that.

I initially thought working on it directly could become expensive and is maybe not worth it, but I still wanted to consider it. I could also just keep working on my main machine and use the NAS as storage only.

Working on your system and using the NAS as a backup would make for a smaller initial investment.

Do I really need a battery backup SSD for SLOG, when I have a UPS?

The power loss protection is kind of a misleading name. The protection is for anything that would not allow the system to write the data before a shutdown, not just a power outage. It doesn't happen often, but there are things that can cause a FreeNAS system to crash and the SLOG drive is actually only ever read when there were 'in-flight' writes that have not yet been committed to disk (like after a crash) and the writes are recovered from the SLOG.

Like @garm said, you don't need a SLOG if you are not using NFS or iSCSI.

Ok, so from what I’ve read now, it is better to max out RAM before even considering an L2ARC device, correct?

Yes, you would want to max out your RAM before even considering L2ARC. There are only a few exceptions to this rule. If you are reading the same set of files over and over would be one. Have a look at this video as it explains pretty well:
https://www.youtube.com/watch?v=oDbGj4YJXDw

Get 32 GB of DDR3 RAM initially.

DDR3 / DDR4 depends on the system board ... It must be compatible because it isn't interchangeable. Did you change system boards?

Do you want hardware suggestions, or are you happy with what you have? You could probably save cash money with used gear instead of that new hardware.
However, I would need an actual location before I could make suggestions. In the US, we get deals that the rest of the world usually can't match.
Sorry.

 

[Chris Moore]

UPS only saves you from power outages or problems from the power grid, but when the machine turns off due to something else... then it won't save you :\
eg. Kernel Panic, ECC memory errors, PSU has gone haywire.

It's in these circumstances that a PLP SSD might save you.

That is exactly it. If something besides utility power failure causes an uncontrolled shutdown, the SLOG drive will still have the data that was in-flight so it can be written when the system comes back online.

 

[Jorsher]

- It doesn't look like you selected ECC ram
- Adding a drive here and there isn't going to work the way you're expecting.
- In your case where speed seems paramount, I personally would mirror a couple large M.2 NVMe drives as a "working" drive, and have separate storage for saving the data you aren't working on. I don't know what sort of video bitrate you're working with, but you'd probably be annoyed with 1gbe...it's theoretical max is 125MB/s.