4K Video editing NAS with 10gb speeds is FreeNAS the way to go?

[unpavedkevin]

I have been doing research on solutions for my small video production facility. I am looking to have a NAS that 2 editors can access data from at the same time. I plan on building a server with an e5 xeon and ECC ram, also a dual 10gb ethernet card 16GB of ram, and 8x 4TB enterprise 7200rpm drives. I plan on having both computers attached through SAF+ cables to both editing stations, one being a hackintosh and other a pc.



Is freeNAS a safe, fast, viable option? I was also looking at unraid, but I like the support community of freeNAS. I want to be able to saturate the 10gb speeds on each system. Any thoughts? Is all this possible? Any guides on doing so with my situation? Thanks in advance!

 

[Elliot Dierksen]

At an exceedingly high level, I would be inclined to say yes. Of course, my presence here as a long time FreeNAS user does perhaps imply something less than complete impartiality.

There are some more details that need to be provided. First off as a networking person, I am not real high on the notion of a direct cable connection to each workstation. The would mean you would need a different IP network for each interface and both sides would need to have their addresses statically assigned. I would assume that both FreeNAS and the workstations would have other interfaces that would allow them connectivity to the rest of the world. One of the best things you can do for FreeNAS is give it more RAM which it can use to buffer data. You are certainly within the recommended minimums, but saturating a 10G isn't an easy task. 8 physical drives spreads the data across multiple spinning platters (almost always the slowest component), so that is good. You should probably consider a hot spare just in case. Your willingness to spend some money on the storage causes me to infer that the data has some value. A extra HDD as a CYA wouldn't be too much of an extra investment. You should should really check out some links I am referencing from @Chris Moore's signature for build hardware recommendations, and ZFS build and performance suggestions.

Hardware Requirements http://www.freenas.org/hardware-requirements/

Slideshow explaining VDev, zpool, ZIL and L2ARC https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Testing the benefits of SLOG https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

The ZFS ZIL and SLOG Demystified http://www.freenas.org/blog/zfs-zil-and-slog-demystified/

10 Gig Networking Primer https://forums.freenas.org/index.php?resources/10-gig-networking-primer.42/

 

[unpavedkevin]

Ok, and thank you so much for the helpful thoughts. A lot of what you said though leads me to more questions also due to my lack of experience and your expertise in this field :) I am reading the links you provided though. I couldn't find Chris Moore`s build btw just his profile page.


I made it easy to answer the questions below each line since there is still a lot. hahahah!

1. You said you are not keen on peer to peer connection from NAS to workstation..why? Yes both workstations would also connect to 1gb ethernet port out of the back of the motherboard and then to a comcast cable modem.
2. Would you rather my get a 10gb switch..any why? Budget is tight so a $600 switch is out of the question. Something like a Quanta LB4M is used and cheap though.
3. I am thinking 8x 8TB or 4x 10TB or 8x 6TB something like that in raidZ2. Would one be faster than the other due to the configuration if I stick with 7200rpm drives?
4. Taking your RAM advice I will bump that up to 32GB of RAM. Would also installing a NVME m.2 cache drive benefit me, or a couple of SSDs in raid?
5. Here is the build list so far: correct me if you think I will have problems
6. "You should probably consider a hot spare just in case" whats a hot spare?
7. "A extra HDD as a CYA wouldn't be too much of an extra investment" what do you mean?

I plan on backing up to redundantly to backblaze home vis iscsi method someone recommend if possible to have the drive mounted as a physical drive


SPECS I am thinking:

HGST 10TB or 8TB Hard Drives 7200 RPM Ultrastar He10 SATA 6Gs/s 256MB 3.5
HBA: LSI SAS 9207-8i
PC Workstation: MNPA19-XTR MELLANOX 10GB ETHERNET NETWORK INTERFACE CARD W/CABLES
HACKINTOSH Workstation: Solarflare SFN5122F Nic card
freeNAS Machine: Chelsio s320 Dual Port NIC card
Intel Xeon E5-1428L v2 6-Cores 2.20 GHz 15M Cache 0 GT/s Processor SR1B9 LGA1356 or something used and cheap but powerful enough.
RAM 32GB ECC DDR3
Supermicro motherboard (not sure yet the model)

All your help is greatly appreciated, thanks!

 

[Elliot Dierksen]

1. You said you are not keen on peer to peer connection from NAS to workstation..why?

Some of it is network guy snottiness (if I am completely honest). It is certainly lower cost to do it with Twinax cables (big clunky cables with SFP+ ends soldered on). I guess most of it is thinking that it would be a pain to deal with the static IP on two point point networks. If that doesn't phase you, then it probably isn't a big deal. You could also have the FreeNAS box bridge the two connections together and make it a single IP network. This are addresses #2 & #3.

3. I am thinking 8x 8TB or 4x 10TB or 8x 6TB something like that in raidZ2. Would one be faster than the other due to the configuration if I stick with 7200rpm drives?

If you are going to do RAIDZ2 which uses two drives for parity, 6 or 8 physical drives is the right number. Spreading the data across more platters gives you some better speed. RAM that FreeNAS will allocate for the read buffer is also a big win. It will take everything it can get. Rotational speed on the drives is kind of hard to say. Higher RPM drives definitely generate more heat. I can tell you in my system I have two RAIDZ2 vdevs with 8 x 1TB 7.2k SATA drives, and I can pretty consistently get around 8Gb read performance. I do have 128GB RAM which also helps.

4. Taking your RAM advice I will bump that up to 32GB of RAM. Would also installing a NVME m.2 cache drive benefit me, or a couple of SSDs in raid?

The use case for an SLOG (dedicated ZFS intent log) is when you are using something that does synchronous writes like ESXi accessing FreeNAS via NFS. That is how my ESXi hosts use FreeNAS and adding an NVME SLOG was a major jolt in write performance. It depends how you are accessing FreeNAS, and what you are doing with it. A dedicated L2ARC is a major performance jolt for some very specific cases like reading the same file to image computers. I have never done it because that isn't how I how use mine.

5. Here is the build list so far: correct me if you think I will have problems

Chelsio is definitely10G NIC of choice in FreeNAS-land, but I can't remember if the S320 is still supported in the latest versions. I know ESXi dropped support for it which caused me some grief.

6. "You should probably consider a hot spare just in case" whats a hot spare?
7. "A extra HDD as a CYA wouldn't be too much of an extra investment" what do you mean?

You have an extra drive installed in your system, and you add it to the pool as a spare. That just gives you more protection in the case of multiple drive failures.

I plan on backing up to redundantly to backblaze home vis iscsi method someone recommend if possible to have the drive mounted as a physical drive

I don't know if FreeNAS can mount an iSCSI target, but I know it can mount a remote NFS file system. You can also do rsynch to another system/NAS that supports it. I am just getting started on this myself, so I can't provide too much more insight.

SPECS I am thinking:

HGST 10TB or 8TB Hard Drives 7200 RPM Ultrastar He10 SATA 6Gs/s 256MB 3.5
HBA: LSI SAS 9207-8i
PC Workstation: MNPA19-XTR MELLANOX 10GB ETHERNET NETWORK INTERFACE CARD W/CABLES
HACKINTOSH Workstation: Solarflare SFN5122F Nic card
freeNAS Machine: Chelsio s320 Dual Port NIC card
Intel Xeon E5-1428L v2 6-Cores 2.20 GHz 15M Cache 0 GT/s Processor SR1B9 LGA1356 or something used and cheap but powerful enough.
RAM 32GB ECC DDR3
Supermicro motherboard (not sure yet the model)

The LSI SAS controller is a good choice. I have a 9207-8E for an external enclosure and works very well. That also seems to be the SAS controller of choice here. ECC on the memory is a must IMHO, so good there. Supermicro boards seem to be very popular here, so that seems good as well. Unless you are running jails (using FreeNAS like a kind of Hypervisor), it isn't IMHO terrible processor intensive. Good drive controller, drives, NIC, and enough memory are the most important.

 

[unpavedkevin]

Thank you for the extremely fast response, that took me 45mins to type, you blazed on the typing!

I found a link in regards to the iscsi that might help. I just don`t want to have to use backblaze b2 when I just bought backblaze personal, and its unlimited data! http://thesolving.com/storage/how-to-create-an-iscsi-target-with-freenas/

I am going to move forward with these specs unless you have anything else to add.
I also think nvme would be huge because we will be reading the same video files in a editing project again and again as we work on the cuts. Guessing that would really speed things up.
DDR3 is cheap so maybe that is the way to go. Does the newer generation cpu`s add any benefit for freeNAS?

 

[Elliot Dierksen]

Thank you for the extremely fast response, that took me 45mins to type, you blazed on the typing

You are welcome.

I found a link in regards to the iscsi that might help. I just don`t want to have to use backblaze b2 when I just bought backblaze personal, and its unlimited data!

I am not familiar with backblaze, but having a backup is certainly always a good thing!

I also think nvme would be huge because we will be reading the same video files in a editing project again and again as we work on the cuts. Guessing that would really speed things up.
DDR3 is cheap so maybe that is the way to go. Does the newer generation cpu`s add any benefit for freeNAS?

In that case you would add the NVME as an L2ARC to your pool. The nice thing is that an L2ARC or SLOG can be added or deleted while the pool is mounted, so you can see what gives you the biggest bang for your buck. The only thing I would say on CPU is if you are doing CIFS (which it sounds like you are), a higher clock speed CPU is good because I think Samba is still single threaded. Past that it all sounds good.

 

[unpavedkevin]

Wow you are right Samba is single threaded, that changes everything! I guess I should look for a 4GHZ i5 or something?

 

[Elliot Dierksen]

FYI, the Chelsio T3 should still be fine for FreeNAS. Looked on my 11.1-U5 system, and kernel module still exists.

/boot/kernel/if_cxgb.ko

 

[Elliot Dierksen]

Wow you are right Samba is single threaded, that changes everything! I guess I should look for a 4GHZ i5 or something?

You mentioned an LGA1356 board, but the CPU you referenced is an LGA2011. Your choices in that family are here. https://ark.intel.com/products/series/78582/Intel-Xeon-Processor-E5-v2-Family

FWIW, my main FreeNAS has E5-2660 v2 @ 2.20GHz. My 10G network is strictly for NFS, so I don't have any input on that. That said, reading from disk and sending to a NIC isn't the most intense of CPU operations. I would even have some doubt that your workstations could keep up even if FreeNAS is pushing a sustained 10G.

 

[BigDave]

Just thought I would throw my two cents worth in here.
Before going to the expense of a network upgrade, I would test my current configuration
to verify that your use case (and hardware) saturates the existing 1G interface with consistency.
What I'm trying to say is that in most cases, if IOPs is the important factor, your pool needs
to be configured with mirrored pair volumes. The downside to that type of config is of course
less redundancy. In FreeNAS the RAID2 or RAID3 setup has great levels of redundancy, but
alas is slow as molasses in winter.
So let's assume you need speed rather than safety for working your data. Spinning rust (HDD)
come in various types, the fastest being 10,000+ RPM Enterprise SAS-SATA drives.
These are expensive and generally have less space for storage than your typical modern
> 2TB desktop models spinning at the slower <6,000 RPM.
You probably already know about this, but I just wanted you to know that IMHO the hardware
must be capable of the higher network speeds BEFORE making the investment in 10Gig
equipment. In a nut shell, if your file transfer speeds (over the LAN) can be quadrupled
then have at it. BUT if your hardware is the bottleneck NOW, fix that BEFORE the network upgrade!