I might try FreeNAS

[Netdewt]

One thing I haven't seen the addressed is the hot vs. warm storage requirements. Editing photo's will require bandwidth & IOPS. If there's a stable storage archive, you may be able to save some money by running two storage pools. A slow RAIDz2 pool built from conventional disks, and a smaller all SSD pool for your active editing tasks. This would allow you to use small inexpensive SSD's configured to maximize the IOPS, and set up replication tasks to archive to slower bulk storage at night.

I like this idea. It could complicate my offsite backups further.

Currently, I have 2x 14TB single HDD that I use as a long term archive. My business partner does his own thing, but same idea. One is the 'main' drive that I use regularly to access things I need from the past, and the other is an offsite backup. It does not change ever, unless I am prepared to update the offsite backup.

With something like this I could possibly have just 4x 2TB SSDs as 2 mirror vdevs as a fast working zpool. 'Quota' is a new idea to me. I could cap this at 2TB for the zpool and keep 2TB of free space at all times.

Once a job is done, but not yet ready for long term storage (ie I am still bugging my partner to take care of it), it could move to 'purgatory' where it's still accessible and backed up offsite, but not as fast and does not need to be backed up nightly. This could be a single HDD, a single vdev, or I could do another 4x2TB but with HDDs. Perhaps I could even take the 4x 2TB HDDs out of this 5 year old promise pegasus for now. The drives seem fine but the enclosure is going haywire.

8-bay gives you four mirrors for higher performance (especially relevant if you don't go for SSD), or enough for the hot+cold option of @rvassar in one enclosure, with one SSD pool for performance and 4 HDD in RAIDZ2 for security.

I don't understand why I would do a RAIDZ2 with 4 disks. Why not just make a pool of 2 mirrored vdevs?

 

[Netdewt]

See those Samsung Prosumer ( the pro series ) if you load them all the time with 90% of data, there is not enough free space and they will die like tomorrow...

I had not considered this. Definitely noted.

 

[Etorix]

I don't understand why I would do a RAIDZ2 with 4 disks. Why not just make a pool of 2 mirrored vdevs?

Security. A RAIDZ2 can lose any two drives. A stripe of two 2-way mirrors can only lose one drive in each pair; lose two in a pair and the pool is lost.
If your primary NAS is optimised for performance, it would make sense to optimise the second NAS for redundancy.

 

[danb35]

A stripe of two 2-way mirrors can only lose one drive in each pair; lose two in a pair and the pool is lost.

OTOH, the mirror will resilver much more quickly, reducing the time you're without redundancy. Pros and cons. I'd still consider the RAIDZ2 to be safer, but the resilver speed mitigates that a bit.

 

[Netdewt]

I spoke at length with a TrueNAS sales rep today. I learned that the OS is still open source and not 'supported' by them, even if you buy hardware from them. So, while that is still nice to have hardware support I am still considering building a system myself.

Here's a build list from Newegg, if anyone would be willing to look it over to give me feedback. With this case, I could have 4 internal HDDs and 4 hotswap 2.5" SSDs in a 5.25 bay. I'm not sure if the motherboard would let me expand with any additional drives or not. There are 8 sata connections, and the PCIE slots would allow more with another controller, I think.

One big difference perhaps is that this would not have IPMI. I'm not sure if that matters.

This would be approximately $560 cheaper than a TrueNAS XL+.

 

[ECC]

Here's a build list from Newegg, if anyone would be willing to look it over to give me feedback.

I think you picked the wrong RAM. Seems that your board can only handle UDIMM, not registered RAM

 

[Netdewt]

I think you picked the wrong RAM. Seems that your board can only handle UDIMM, not registered RAM

Is UDIMM the same as non-ECC? I think the board takes both?

 

[Patrick M. Hausen]

Is UDIMM the same as non-ECC? I think the board takes both?

UDIMM is unregistered DIMM. As opposed to registered memory. It's the presence or absence of certain auxiliary logic (the register) on the DIMM itself if I remember correctly. Both come in ECC or non-ECC.

 

[Herr_Merlin]

thats correct.
I am not aware of any reg dimms without ECC

 

[Netdewt]

Is 720 TBW acceptable for a $180-200 2TB drive? That's where the Crucial BX500 and the Samsung 870 QVO, IIRC, for around $200. The WD Reds are $280 for over 1300 TBW.

 

[Patrick M. Hausen]

Here's a little bit of simple math to bring those TBW figures into perspective:

How I learned to stop worrying and love SSDs

Hi all, my home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...? So I...

www.truenas.com

Only you can make an educated guess at your write load.

 

[Netdewt]

Good deal, thanks, TBW was a new concept to me.

 

[Herr_Merlin]

Check Out transcend they are usually as cheap but with tlc and ddr cache as well as slc cache... Good deal in europe at least. Plus the 2pb tbw

 

[Patrick M. Hausen]

Check Out transcend

I love to use these as boot devices when I have no redundancy, like in OPNsense or ESXi installations. The TBW for the smaller devices is insane.

 

[Etorix]

Wrong memory! You need ECC UDIMM. RDIMM is for Xeon D/W/Scalable or EPYC.
(To tell modules apart: UDIMM shows complex traces in the middle. Non-ECC UDIMM has chips in multiples of 8; ECC UDIMM in multiples of 9 (8 data + 1 parity). ECC RDIMM has chips in multiples of 9 plus one, which is the register from "Registered". The module you picked has 19 chips and few traces: Your board won't even POST with it.)

Since you're happy with four cores, take an i3-9100F instead of the Xeon E-2124 and save $100. Up to 9th gen., Core i3 handles ECC.

IPMI is for remote management of the motherboard. See for instance Supermicro X11SCH-F, with the same other components

 

[rvassar]

I like this idea. It could complicate my offsite backups further.

Currently, I have 2x 14TB single HDD that I use as a long term archive.

Once a job is done, but not yet ready for long term storage (ie I am still bugging my partner to take care of it), it could move to 'purgatory' where it's still accessible and backed up offsite, but not as fast and does not need to be backed up nightly.

You should take a look at the "Tasks" option in the UI. There are four that you will find interesting with regards to backups:

Rsync Tasks - These can copy data in a dataset from one location to another using the traditional Unix rsync command. For example, replicate your active SSD editing pool to the 14Tb long term storage pool. Rsync only copies files that have changed, so it's relatively quick. These tasks can be made to function "one-way", so that deleting the item on the SSD pool will not automatically delete it from the long term storage pool. This would make your "Purgatory" somewhat automatic. There are some gotchas, but they can be managed.

Periodic Snapshots - These are frozen points in time in the pool. You set a periodic snapshot, and can revert back in time to the previous state. This helps if you accidentally delete a file or entire folder structure. If it exists in a previous snapshot you can "go back in time" and recover it.

Replication Tasks - These implement pool replication, both between pools, as well as between servers running ZFS. It creates a snapshot, and sends the entire contents of it to another pool. I have actually configured ZFS on a Raspberry Pi, and replicated to a external USB drive over the network to effect a backup. It can also be to a VM running on VMware or VirtualBox on your PC, with the USB drive "passed thru" to the VM. It can get complicated, but the point is to avoid attaching and detaching hardware from the NAS. Make the network do the work.

Cloud Sync Tasks - These are used to push data to cloud storage services like AWS S3 / Backblaze B2. I'm not sure if there's a AWS Glacier module. AWS Glacier is the cheapest option for cloud backup. 14Tb would cost about $60/mo., I think. Once setup, the NAS will just push the data up to the cloud over your internet connection on a schedule. Encryption is optional. You just monitor and of course... pay the bills.

 

[Netdewt]

Another existential question: why would someone in my situation want a TrueNAS XL+ or other built-up FreeNAS solution as opposed to a QNAP like this one? What kinds of features would I gain/lose?

QNAP TS-932PX 5 (+4) Bay 4G 64-bit NAS

QNAP TS-932PX 5 +4 Bay 64-bit NAS, Annapurna Labs AL324 64-bit quad-core 1.7GHz, 4GB DDR4 SODIMM RAM, max 16GB, 5 x 3.5 inch and 4 x 2.5 inch drive slots, 2 x 10GbE SFP+ LAN, 2 x 2.5GbE LAN, 3 x USB 3.2 Gen1, hardware encryption, expansion unit support.

qnapdirect.com

 

[jgreco]

Another existential question: why would someone in my situation want a TrueNAS XL+ or other built-up FreeNAS solution as opposed to a QNAP like this one? What kinds of features would I gain/lose?

QNAP TS-932PX 5 (+4) Bay 4G 64-bit NAS

QNAP TS-932PX 5 +4 Bay 64-bit NAS, Annapurna Labs AL324 64-bit quad-core 1.7GHz, 4GB DDR4 SODIMM RAM, max 16GB, 5 x 3.5 inch and 4 x 2.5 inch drive slots, 2 x 10GbE SFP+ LAN, 2 x 2.5GbE LAN, 3 x USB 3.2 Gen1, hardware encryption, expansion unit support.

qnapdirect.com

You gain the ability to be locked into buying another QNAP when your QNAP breaks, paying a price premium if you want to expand beyond the number of bays the unit offers, to be highly limited when it comes to running virtual machines or jails for things like Plex, and to have the craptacular performance that QNAP's and Synology type units are known for.

You can make an equally janky FreeNAS system, of course. But you can also make a screamer...

A 12-bay rackmount QNAP or Synology will typically run $2500-$4000, while you can pick up a basic used 2U 12 drive Supermicro chassis with an X9 board and CPU's off of eBay for $500, outfit it with 256GB of RAM for $400, add a dual 10G card, and have a much better NAS host for $1000.

 

[rvassar]

The little ARM based QNAP boxes are slugs. I had one... Not a bigger one like the one you tagged, one of the entry units. Cheap PSU brick was so noisy it triggered the ARC fault breakers here at my house. It's actually why I ended up here.

 

[Etorix]

To be the Devil's advocate, by going QNAP/Synology you gain a more user-friendly experience of setting up whatever (limited set of) plug-in or VM has been precooked by the vendor.

By going TrueNAS, especially building your own NAS around it, you'll learn a lot more than you expected about server hardware and the intricacies of enterprise storage. Put that as a plus or as a minus as you see fit.