NAS Build for small VFX Company. Need input from experienced users.

[Robert Freitag]

Hello,
I own a small VFX company which finally needs a more decent storage solution, over the last days I've investigated quite a lot.
And I think that I'll go with FreeNAS and a Build according to my/our needs. If any of you experienced FreeNAS users could answer me some questions and possibly have a look at my build, that would be much appreciated. Thanks for your Time!
Please keep in mind, it's my first time ever building a NAS. I've read quite a lot before making this post.

My Environment:
Regular 1 Gbit/s network.
A small Render Farm with 5 PC's occasionally pulling bigger amounts of data (<10 Gigabytes) from the NAS (i.e. loading 3D scenes in order to render).
2 Workstations, both need to occasionally access files and sometimes transfer large amounts of data to or from the NAS.

Requirements:
The NAS needs to be able to securely store large amounts of data.
The NAS needs to do drive encryption.
I need to be able to recover all data in case of Hardware Failure.
Optional: I need to be able to use link aggregation in order to provide >1Gbits to multiple computers in the network.

My Questions:
I've seen a lot of FreeNAS tutorials where people are using USB flash drives to store the OS. Aren't USB flash drives kinda vulnerable for storing an OS?
Regarding that, will I be able to recover my data in case of a OS-drive failure? Or (for some reason) OS failure in general?
In case of any other hardware failure, (CPU, RAM etc.) will I be able to just exchange parts and everything runs as it did before?
The MoBo I'm considering has 2 NIC's, will I be able to use the link aggregation features of FreeNas to provide theoretical 2 Gbit/s of bandwidth towards the other computers in the network?

So here is my build, I've mostly followed recommendations from this forum:
Mobo:
ASRock E3C236D2I Intel C236 So.1151 Dual Channel DDR4 Mini-ITX Retail
CPU:
Intel Core i3 6300 2x 3.80GHz So.1151 BOX
RAM (2x, so 32gigs):
16GB Crucial CT16G4WFD8213 DDR4-2133 ECC DIMM CL15 Single
M.2 Drive for OS:
SANDISK Z400s SSD M.2 2242 64GB intern SATA 6Gb/s
HD's (4x):
8000GB WD Red WD80EFZX 5.400U/min 128MB 3.5" (8.9cm) SATA 6Gb/s
PSU:
600 Watt Corsair SF Series SF600 Modular 80+ Gold
Case:
Silverstone DS380 Mini Tower
Case Fans:
Noctua NF-F12 industrialPPC-3000 PWM 120x120x25mm 750-3000 U/min 43.5 dB(A)

Thank you for reading all this and thank you for your input in advance.
-Robert

 

[Stux]

Assuming Raidz2, your data will be safe if the hardware breaks, you lose everything, except 2 drives.

You should still have a backup of the data, to protect yourself from unexplained pool corruption or administrative error.

 

[tvsjr]

IOPS - how often are these systems pulling new data from the central server? If you're planning one RAIDZ2 vdev, you're talking ~100 IOPS. Lots of random seek activity from multiple systems will make performance suck mightily.

What's your use case for drive encryption? Encryption adds overhead and substantial complexity. Make sure you understand exactly why you're running encryption (and ensure running encryption actually addresses the security or compliance use case you have) before you do it.

Link aggregation won't give you >1Gbps from the server to a single client. It uses a hash mechanism to determine which NIC should be used for which stream. Assuming a perfect distribution, 4 gigabit NICs aggregated would give you 1Gbps simultaneously to each of 4 machines, but wouldn't give you anything more than 1Gbps to any one system. If you aren't serving contents to hundreds of systems, link aggregation won't help you... and may actually make things worse by increasing complexity.

 

[SweetAndLow]

USB as boot device is ok solution. The os is completely separate from the data disks and can be reinstalled at any time without data loss.

Encryption is a bad idea you will probably mess it up and lose your keys and never unlock your data.

The best way to not lose data is to always have a backup.

You can replace any hardware parts and not lose data


Link aggregation will not increase throughput. You need 10gig NICs for that. Now if you have 10+ active clients lagg would be helpful.

I would get 6 disks and do raidz2. This would get you ~32TB with 8tb disks. How much storage are you using now?

Sent from my Nexus 5X using Tapatalk

 

[Ericloewe]

The NAS needs to do drive encryption.

Why? That doesn't sound like a typical VFX requirement. It will significantly increase the probability of human error or freak accident locking away your data forever.
It's certainly possible, though.

Optional: I need to be able to use link aggregation in order to provide >1Gbits to multiple computers in the network.

Ehh, that's wishful thinking (IEEE 802 makes this surprisingly user-hostile), but 10Gb switches are getting significantly cheaper.

Bah, xenforo on mobile isn't cooperating, bear with me and the lack of quotes.

Re: USB flash drives. Yes, they suck and are barely tolerable. Since the boot pool contains no important data apart from the config file, which you should always backup, it's not a big problem.
For professional situations, SSDs for boot are the way to go (any cheap SSD from Intel, Crucial, Micron, SanDisk, Toshiba or Samsung).

As for data, yes, it's all separate from the OS (it baffles me that this concept is even questioned, having them intermingled is appalling). And yes, you can swap hardware and everything will work, within reason - NICs may need to be reconfigured and everything has to be compatible (again, even Windows isn't particularly picky about this, so it's not a problem with anything that isn't big iron).

 

[Robert Freitag]

Thanks everyone for answering all those questions! You guys are fantastic!

I would get 6 disks and do raidz2. This would get you ~32TB with 8tb disks. How much storage are you using now?

Thanks for your answer, we are currently using 12 TB on this fancy device: CLICK While sharing it through a windows share... now you know why we need a more proper solution. :X

Why? That doesn't sound like a typical VFX requirement. It will significantly increase the probability of human error or freak accident locking away your data forever.
It's certainly possible, though.

Two of our customers have recently (loosely) asked about the infrastructure and encryption because they want to get a specific IT security certificate by an auditing company in which it states that all their vendors use a certain kind of drive encryption for project related storage.

Ehh, that's wishful thinking (IEEE 802 makes this surprisingly user-hostile), but 10Gb switches are getting significantly cheaper.

I've already thought so! Thanks for the Info though.

As for data, yes, it's all separate from the OS (it baffles me that this concept is even questioned, having them intermingled is appalling). And yes, you can swap hardware and everything will work, within reason - NICs may need to be reconfigured and everything has to be compatible (again, even Windows isn't particularly picky about this, so it's not a problem with anything that isn't big iron).

Thanks! That's good to hear.

Thanks for your answers!

 

[Ericloewe]

Two of our customers have recently (loosely) asked about the infrastructure and encryption because they want to get a specific IT security certificate by an auditing company in which it states that all their vendors use a certain kind of drive encryption for project related storage.

Gotta love bureaucracy. It's not like it's confidential information - unless you're doing VFX for military applications or something of the sort.

 

[Robert Freitag]

Gotta love bureaucracy. It's not like it's confidential information - unless you're doing VFX for military applications or something of the sort.

hmmm, not really actually. I've worked on quite a lot of projects for which I had to sign NDA's with some very sensitive fines for breaking confidentiality.
Imagine a VFX leak for a popular TV series that everyone is talking about...

If you are interested, you can check out our website:
http://www.fridayvfx.com

Thanks again!
-Robert

 

[Ericloewe]

But encryption at rest doesn't help with that, unless people are so obsessed with getting spoilers that they're going through your discarded drives. Are we at that point? I haven't been watching much TV lately.

 

[Robert Freitag]

But encryption at rest doesn't help with that, unless people are so obsessed with getting spoilers that they're going through your discarded drives. Are we at that point? I haven't been watching much TV lately.

Sure, my point just was that VFX sometimes does contain quite confidential information, but you are absolutely right, there is no point about it, just making auditors happy. :)

 

[Stux]

Gotta love bureaucracy. It's not like it's confidential information - unless you're doing VFX for military applications or something of the sort.

Or Hollywood.

can't have non-authorized leaks going to TMZ about an upcoming movie

 

[Ericloewe]

Or Hollywood.

Leaks are bad mmmm'kay

See my point above - confidential is probably the wrong word. Secret enough to warrant the substantial effort of piecing it together from trashed drives from the dumpster. That's only going to happen with state-sponsored effort, and the Chinese are not that desperate for another Michael Bay Transformers movie, they're starting to catch on.

 

[Stux]

See my point above - confidential is probably the wrong word. Secret enough to warrant the substantial effort of piecing it together from trashed drives from the dumpster. That's only going to happen with state-sponsored effort, and the Chinese are not that desperate for another Michael Bay Transformers movie, they're starting to catch on.

Unfortunately, as you know, sometimes the rules are stupid, but you still need to comply with them

To the OP: if you do encrypt your data at rest, make triple sure you have the encryption keys backed up safely.

 

[tvsjr]

Auditors have this thing called "compensating controls". I live in the government world, and we have requirements that say, pretty much, "encrypt data at rest". Encryption at rest addresses three things:
Theft, either of the drive or of the system
Loss, ditto
Improper disposal, also ditto

This makes perfect sense for a laptop. It makes zero sense for a server, because a server is usually, by definition, running 24x7... which means the keys are cached in RAM. There's a 99.999% chance that any compromise will occur over the network.

We've successfully passed audits by deploying compensating controls, such as:
To address theft - housing the equipment in a tier 3/tier 4 data center with extensive physical security controls, additional badge readers on our cages, locked cages within, etc.
To address loss - creating a very detailed procedure by which servers and drives are removed from the secure environment.
To address disposal - creating another detailed procedure to ensure servers and drives are disposed of properly, such as wiping the drives prior to them leaving the confines of the data center cage.

Full-disk encryption on servers is a pain. You can't reboot the system without entering credentials. You take a constant performance hit (even with AES-NI). And it makes it all the much easier to lose your entire storage array/pool if you screw up.

If you go down this path, you need to ensure you have an airtight backup procedure (which will also need to be encrypted, but I digress).

 

[Stux]

If you go down this path, you need to ensure you have an airtight backup procedure (which will also need to be encrypted, but I digress).

Was wondering... how do you safely and securely backup your encryption keys... I don't like the idea of just leaving a couple of USB fobs in a safe.

 

[tvsjr]

Was wondering... how do you safely and securely backup your encryption keys... I don't like the idea of just leaving a couple of USB fobs in a safe.

For most stuff, we use CommVault, which manages the keys internally. For things where that isn't an option, usually the keys are printed out and kept in a very serious fire-proof safe. Ideally, at least two copies in two geographically diverse locations. The number of people with access is minimal, and there's a process for changing the keys every so often.