Guidance request for New Video Backup Build

[priyanuj]

Hello everyone,
I'm planning to build a NAS for my company to backup RAW video footage. Footages will be 4K 8-bit. Individual size varying from maybe 15-25 GB.

My requirements are:

• At least 30 TB of Final Storage.
• Fast Read/Write Speed as we plan to dump at least 150-200 GB of data every day.

So now my queries:

• Will the following specification for the system works: i5-10400, Compatible Low-Mid Range Mobo with 64 GB Non-ECC RAM, 128 GB Boot SSD, 10 x 8TB HDD
• What is better RAIDZ1, RAIDZ2, or RAIDZ3 and why?
• Help in configuring the disks too.
• Should I change the config and go for ECC RAM? If so, why?
• Is 10G Network required or is it overkill? (I'll be getting separate network cards and switches if it is worth it)
• How long can I use this system before I need a major upgrade?
• Is shutting the system down weekly harmful?

Thanks in advance.

 

[sretalla]

At least 30 TB of Final Storage.
Fast Read/Write Speed as we plan to dump at least 150-200 GB of data every day.

If you want ZFS to continue performing well, you need to leave it a good amount of free space, so consider that when ticking the boxes and setting the target free space here:

ZFS Capacity Calculator - WintelGuy.com

ZFS / RAIDZ Capacity Calculator - evaluets performance of different RAIDZ types and configurations

wintelguy.com

Will the following specification for the system works: i5-10400, Compatible Low-Mid Range Mobo with 64 GB Non-ECC RAM, 128 GB Boot SSD, 10 x 8TB HDD

You will most likely be able to install TrueNAS on this system... some considerations in the following answers.

What is better RAIDZ1, RAIDZ2, or RAIDZ3 and why?

Assuming you want throughput more than IOPS, RAIDZ is a good choice for mixing redundancy with capacity in a fair way.

RAIDZ2 is usually good for most scenarios, but if you're paranoid about data loss, maybe RAIDZ3 is worth the extra disk and small speed penalty (don't forget backups are important too, so don't feel like RAIDZ3 is an excuse not to do any). Avoid RAIDZ1 unless your disks are really small.

Help in configuring the disks too.

Attach the disks in the system. Boot, then under Storage | Pools Create a new pool from the ADD button.

Should I change the config and go for ECC RAM? If so, why?

I'm not sure that CPU supports it, but maybe you should consider as part of your whole plan here: how important is data integrity? if the answer is very, then you want ECC RAM. If you can afford to have the odd bit flip going on here or there, don't bother.

Is 10G Network required or is it overkill?

You may find that you can bottleneck on a 1G LAN, so if ~110MBytes/sec is acceptable, then no need to go to 10G. I'll assume you can do the math for the rest of it.

How long can I use this system before I need a major upgrade?

Again, do some math on your file storage requirements over time. There's no system imposed limits on how long the OS will work for, but expect to upgrade the OS at least a couple of times a year to stay in support.

Is shutting the system down weekly harmful?

Not as such, but remember that maintenance tasks (and maybe also backups) are often best done outside working hours (the biggest contiguous bunch of which are on the weekend).

If you scheduled all the things that need to happen to complete within the working week, I guess there's nothing wrong with that plan. (SMART short and long tests, pool scrubs... these can be long based on data in the pool, backups, any other maintenance tasks).

 

[priyanuj]

If you want ZFS to continue performing well, you need to leave it a good amount of free space, so consider that when ticking the boxes and setting the target free space here:

ZFS Capacity Calculator - WintelGuy.com

ZFS / RAIDZ Capacity Calculator - evaluets performance of different RAIDZ types and configurations

wintelguy.com

You will most likely be able to install TrueNAS on this system... some considerations in the following answers.


Assuming you want throughput more than IOPS, RAIDZ is a good choice for mixing redundancy with capacity in a fair way.

RAIDZ2 is usually good for most scenarios, but if you're paranoid about data loss, maybe RAIDZ3 is worth the extra disk and small speed penalty (don't forget backups are important too, so don't feel like RAIDZ3 is an excuse not to do any). Avoid RAIDZ1 unless your disks are really small.


Attach the disks in the system. Boot, then under Storage | Pools Create a new pool from the ADD button.


I'm not sure that CPU supports it, but maybe you should consider as part of your whole plan here: how important is data integrity? if the answer is very, then you want ECC RAM. If you can afford to have the odd bit flip going on here or there, don't bother.


You may find that you can bottleneck on a 1G LAN, so if ~110MBytes/sec is acceptable, then no need to go to 10G. I'll assume you can do the math for the rest of it.


Again, do some math on your file storage requirements over time. There's no system imposed limits on how long the OS will work for, but expect to upgrade the OS at least a couple of times a year to stay in support.


Not as such, but remember that maintenance tasks (and maybe also backups) are often best done outside working hours (the biggest contiguous bunch of which are on the weekend).

If you scheduled all the things that need to happen to complete within the working week, I guess there's nothing wrong with that plan. (SMART short and long tests, pool scrubs... these can be long based on data in the pool, backups, any other maintenance tasks).

Thanks a lot for the detailed reply above.

One last question, if I create 2 vdevs in the same pool, will the read/write speed increase or decrease and how will it affect the data loss probability?

 

[sretalla]

if I create 2 vdevs in the same pool, will the read/write speed increase or decrease and how will it affect the data loss probability?

Probably not, it may even reduce depending on the VDEV size and type, but you will get double the IOPS (which would help if you have multiple things going on in parallel).

If you changed to having 4 mirrored pairs instead, that's 4 times the IOPS and maybe an increased write or read speed under some conditions, but loss of 2 random disks at the same time can kill your pool if those 2 happen to be in the same VDEV/mirror. (in RAIDZ2 your pool will survive 2 drives failing no matter which 2).

One positive of mirrored VDEVs is easy pool capacity increases with an additional mirrored pair. Capacity increases with RAIDZ implies either an additional VDEV of the same number of disks or replacing all disks in the VDEV with larger ones.

 

[jgreco]

[moderator note: renamed the thread -jg]

 

[Etorix]

Will the following specification for the system works: i5-10400, Compatible Low-Mid Range Mobo with 64 GB Non-ECC RAM, 128 GB Boot SSD, 10 x 8TB HDD

• What is better RAIDZ1, RAIDZ2, or RAIDZ3 and why?
• Help in configuring the disks too.
• Should I change the config and go for ECC RAM? If so, why?

Yes. But if you care about data integrity (willing to use ZFS is a hint…), ECC RAM is recommended because ZFS trusts RAM without checksuming it: If anything bad happens in RAM, only ECC can catch it. The suggestion here would be to "downgrade" to a Core i3-9100F (pure storage does not need an i5 or more) on a server-grade motherboard C242/C246 and ECC RAM, if you can find such components. It may even be cheaper this way.

RAIDZ is not considered secure with drives > 1 TB. If the goal is to offload raw footage to a first NAS, quickly sort it and copy the good shots to another, more secure, storage, then RAIDZ vdevs may be adequate. (Such transfers would be a good reason to invest in 10 GbE.)
If the storage is supposed to be secure, then you want RAIDZ2 (or Z3). A 10-wide RAIDZ2 if capacity is paramount; a stripe of two 5-wide RAIDZ2 for better performance (at the cost of storage space) or for increased flexibilty (easier to later add a third 5-wide vdev than a second 10-wide vdev).

 

[priyanuj]

Probably not, it may even reduce depending on the VDEV size and type, but you will get double the IOPS (which would help if you have multiple things going on in parallel).

If you changed to having 4 mirrored pairs instead, that's 4 times the IOPS and maybe an increased write or read speed under some conditions, but loss of 2 random disks at the same time can kill your pool if those 2 happen to be in the same VDEV/mirror. (in RAIDZ2 your pool will survive 2 drives failing no matter which 2).

One positive of mirrored VDEVs is easy pool capacity increases with an additional mirrored pair. Capacity increases with RAIDZ implies either an additional VDEV of the same number of disks or replacing all disks in the VDEV with larger ones.

Thanks a lot.

 

[priyanuj]

Yes. But if you care about data integrity (willing to use ZFS is a hint…), ECC RAM is recommended because ZFS trusts RAM without checksuming it: If anything bad happens in RAM, only ECC can catch it. The suggestion here would be to "downgrade" to a Core i3-9100F (pure storage does not need an i5 or more) on a server-grade motherboard C242/C246 and ECC RAM, if you can find such components. It may even be cheaper this way.

RAIDZ is not considered secure with drives > 1 TB. If the goal is to offload raw footage to a first NAS, quickly sort it and copy the good shots to another, more secure, storage, then RAIDZ vdevs may be adequate. (Such transfers would be a good reason to invest in 10 GbE.)
If the storage is supposed to be secure, then you want RAIDZ2 (or Z3). A 10-wide RAIDZ2 is capacity is paramount; a stripe of two 5-wide RAIDZ2 for better performance (at the cost of storage space) or for increased flexibilty (easier to later add a third 5-wide vdev than a second 10-wide vdev).

I'm getting a bit confused over here now. If I go with stripe of two 5-wide RAIDZ2 for now and in the future, if I add another 5-wide vdev, then I'll need to create another pool with it to get more storage right?
Or only adding another vdev will increase the overall capacity of my NAS ?

Thanks.

 

[Etorix]

A pool can have any number of vdevs. Vdevs can be added at any time. But storage vdevs should be of the same geometry (no mixing mirrors and raidz or raidz1 and raidz2). Raidz# vdevs cannot be removed once added.
If you create a pool with a single 10-wide raidz2 vdev, the next step to increase capacity is to add a second vdev of 10 drives in raidz2 to the pool. If you create a pool with two 5-wide raidz2, you only need five new drives to add a third vdev later to the same pool. The new drives need not be of the same capacity as the old ones; for instance, a pool of 2*(5*8TB) (48 TB of raw storage in raidz2, minus ZFS overhead) could grow later to 2*(5*8TB)+(5*14TB) (70TB raw).
Of course, if you use different pools you can do as you want, with different vdev geometries for different pools.

 

[priyanuj]

A pool can have any number of vdevs. Vdevs can be added at any time. But storage vdevs should be of the same geometry (no mixing mirrors and raidz or raidz1 and raidz2). Raidz# vdevs cannot be removed once added.
If you create a pool with a single 10-wide raidz2 vdev, the next step to increase capacity is to add a second vdev of 10 drives in raidz2 to the pool. If you create a pool with two 5-wide raidz2, you only need five new drives to add a third vdev later to the same pool. The new drives need not be of the same capacity as the old ones; for instance, a pool of 2*(5*8TB) (48 TB of raw storage in raidz2, minus ZFS overhead) could grow later to 2*(5*8TB)+(5*14TB) (70TB raw).
Of course, if you use different pools you can do as you want, with different vdev geometries for different pools.

Understood. Thanks a lot.