My Objectives, Looking for Guidance on Strategy

[_Will_]

Hello everyone,

Who I am:
-Mechanical Engineer with slightly above average computer knowledge (I didn't even know what NAS/RAID was before I started researching a few weeks ago, but I am extremely interested in learning everything I can)
-I am running Windows 7 on my home computer (I don't have experience with Linux or any other operating systems)
-Although I do not have extensive computer knowledge I did build my own computer and it was an excellent learning experience and I would find building my own NAS rewarding (as long as there are enough advantages for my objectives over QNAP, Synology, etc.).

My Current situation:
-My home computer has a 3TB internal hard drive that I've completely filled up with HD movies, RAW photos, and music.
- I currently back-up my files manually once every few months to an external HDD. This is not sufficient. I looked into CrashPlan and tried the free trial but it was going to take 4 months to upload all of my data.

My Objectives:
- Since my media is already at 3TB and my need for storage will continue to grow I am looking for a solution to get me somewhere in the 8-10TB range with the potential to add more storage in the future (via larger HDD's as technology improves).
- I want to see all of my HDD's as a single drive so RAID seems like the way to go.
- I want an automatic backup solution and everyone states over and over again that RAID (even RAID 6 and RAID 10) is not a backup. If this is the case, what is the best solution to automatically backup 8-10TB of NAS storage?
- Accessing the NAS via the internet from remote locations is required
- I would like to be able to stream movies from remote locations if that is possible
- I am looking to keep all hardware including HDD's under $1,000 if possible.

Questions I have:
If RAID 6 and RAID 10 have redundancy but still aren't backups - should I go with another RAID since my data will be backed up via other methods?
Is FreeNAS right for me?
Is NAS4Free a better solution due to cost or other reasons?
Should I stick to the basics (QNAP, Synology, etc.)?

Please provide what strategy you would use to complete all of the above objectives. I've read through the hardware recommendations, Cyberjock's Guide and ECC information and have learned quite a bit but still don't know what the best solution is for me. Thanks in advance!

 

[Ericloewe]

The budget is low, as it barely covers the HDDs (assuming 6-disk RAIDZ2 with 3TB disks for some ~10TB of usable space).

FreeNAS can definitely work for you:

• Expansion is easy if you replace the drives one at a time with larger ones, or by adding more vdevs
• Some sort of RAID is always the way to go if you value your data, ZFS in particular is a good choice
• Backing up the server is ideally done with a second server. There are other alternatives that have advantages and disadvantages
• Can be done with a VPN
• Again, VPN

As for your questions:

Even with backups, the objective is not having the primary server fail, because restoring everything can be a serious pain in the ass. RAIDZ2 can save you a lot of trouble.
Honestly, I'm not qualified to speak about NAS4Free. You're the one who has to examine the advantages and disadvantages of each solution.
The "basics" aren't much cheaper (if at all) and are nowhere near as capable.

 

[_Will_]

The budget is low, as it barely covers the HDDs (assuming 6-disk RAIDZ2 with 3TB disks for some ~10TB of usable space).

FreeNAS can definitely work for you:

• Expansion is easy if you replace the drives one at a time with larger ones, or by adding more vdevs
• Some sort of RAID is always the way to go if you value your data, ZFS in particular is a good choice
• Backing up the server is ideally done with a second server. There are other alternatives that have advantages and disadvantages
• Can be done with a VPN
• Again, VPN

As for your questions:

Even with backups, the objective is not having the primary server fail, because restoring everything can be a serious pain in the ass. RAIDZ2 can save you a lot of trouble.
Honestly, I'm not qualified to speak about NAS4Free. You're the one who has to examine the advantages and disadvantages of each solution.
The "basics" aren't much cheaper (if at all) and are nowhere near as capable.

Thanks for your response.

What kind of budget do you think I will need to consider in order to purchase hardware, with the exception of HDD's? I will factor those in afterwards. One way to cut down on initial costs might be to settle for 6 or 8TB of usable space and then expand in a few years.

• Maybe I misunderstood, but from reading Cyberjock's Guide, my understanding is that although you can replace the drives with larger drives one at a time using the FreeNAS GUI the zpool usable storage only grows once you've replaced all of the drives in the VDev. Is this correct?
• What are all of the alternatives to backing up to another server? Backing up to another server will be out of my price range at this point.

 

[Ericloewe]

• Maybe I misunderstood, but from reading Cyberjock's Guide, my understanding is that although you can replace the drives with larger drives one at a time using the FreeNAS GUI the zpool usable storage only grows once you've replaced all of the drives in the VDev. Is this correct?

Correct.

• What are all of the alternatives to backing up to another server? Backing up to another server will be out of my price range at this point.

Backing up to loose external drives (ideally connected to a different computer, since FreeNAS can be picky), cloud backups... pretty much anything you can imagine.

What kind of budget do you think I will need to consider in order to purchase hardware, with the exception of HDD's? I will factor those in afterwards. One way to cut down on initial costs might be to settle for 6 or 8TB of usable space and then expand in a few years.

Something in the 600-650ish range gets you a capable server with quality hardware and plenty of potential for future upgrades. Going with AMD may save some bucks, but be prepared to do a lot of research until you find a setup that's known to work (or risk having it not work for some stupid, mysterious reason). An Avoton (Intel Atom) system may also be cheaper, but I haven't actually run the numbers. Shopping around for deals might bring the price down a little bit.

 

[Hyperion]

Mechanical HDD are over, they run down.
Backups are not reliable.
Id make 3 backups of all data.

 

[solarisguy]

Hyperion's suggestion would be very good if there were cheap 10 TiB storage devices (P.S., lookup TiB ;))

Seriously consider that your data keeps growing, so you want to have more than 10TiB and not have to get another server very soon. You likely want to run your server for long enough that it would be cheaper to buy a new one, than upgrade.

I am suggesting that you get a system with six 4TB hard drives in RAID-Z2. That would last until 6TB hard drives are as affordable as 4 TB ones are today. That gives 14.55 TiB, but you had probably learned by now that using a filesystem at >90% utilization is no fun, so you would only store up to 13TiB (ZFS recommendation is more towards 80%).

6 data disks - 2 parity disk = 4 data disks​

4 * 4 TB = 4 * 3.638 TiB = 14.55 TiB​

Then 16GB of ECC RAM. 16GB is to assure stability of the system. ECC is to significantly lower chances of loosing your entire ZFS pool (=all your data) without any chance of recovery. ECC has to be supported by all three: RAM, CPU and the motherboard.

All of the above is ZFS specific and applies to all ZFS implementations. Regardless whether you are choosing FreeNAS, NAS4Free, FreeBSD, Linux or Solaris.

The fact, that someone is alive and well while driving at the speeds twice the posted limit and never stopping at the stop sign, has no bearing on best practices...

Let's deal with backups. NAS = N... A... Storage is where you keep your files. On the other hand, a backup offers protection from natural and man made disasters. More often from the latter ones. A copy in the same building is not considered a valid backup. (At least you are not a business, so no false sense of security that you are backing up to a cloud, that ultimately turns out to be in the data center on the adjacent floor...)

 

[Hyperion]

A backup solution is pretty cheap. Just get an old PC, load FN then run sync.
The main reason HDDs fail is due to fatigue, they run fast and die young.
But if you get a backup machine, say backup once a day, its not under any stress.