6 disks -- 6 disk RAIDz2 or 3 disk RAIDz1 with backup

[errr]

Hi,
I'm planning on building a server based on a mini ITX board with 6x4Tb disks. My use case is a media server so data is not critical and can be recovered. I initially was set on using RAIDz2 with all 6 disks but the options for mini ITX boards are pretty slim so I ended up getting a Supermicro X11SCV-Q which unfortunately only supports unbuffered non-ECC DDR4 ram. I'm really not all that worried about the potential of corruption at the file level as I said before all data can be recovered and is just movies, but if the ZFS metadata was corrupted and I lost my whole pool that would be fairly catastrophic. My ISP limits downloads to 1Tb/Month so if I used my entire cap each month for just redownloading my content, it would take me at least a year to fully recover. It is for this reason that I'm now considering instead to use a 3 disk RAIDz1 configuration with the other 3 disks serving as a backup. The chances of a RAM error occurring that compromises my entire pool is astronomically low but is it still enough to justify compromising **half** of my storage capacity? I did some quick googling and couldn't find an instance of a complete pool loss due to a RAM error, but maybe those of you who have been on these forums for a while have heard stories.

 

[kherr]

Raid of any type is not a backup ...... and no substitute for an isolated, separate backup. I have original disks of all my movies, and shudder to think of what I'd have to go through to re-rip them all ....... A hardware fault can all of a sudden trash the entire server/pool, or a bad combination of disks dieing ...

I cross my fingers with a backup TN server that is only turned on once/twice a month to backup my main box. I also go 1 step further and have it in a detached garage (my cloud). How much do you really value your time and hassle in event of a crisis ????

In my opinion rz1 is useless because if a disk fails in the process of replacing a disk ..... your dead .... Also use ECC ram if at all possible ...

 

[AlexGG]

I did some quick googling and couldn't find an instance of a complete pool loss due to a RAM error, but maybe those of you who have been on these forums for a while have heard stories.

There is a fair amount of complete pool losses never traced to a definitive cause, often because people don't even bother (if backup is available).

Also, there are other possible common-mode failures, not only faulty RAM bringing down both pools during backup (because when no backup is in progress, there is no activity on the backup pool). Consider power supply blowing up and burning all six disks at the same time. Consider also cryptovirus of some kind encrypting all available network shares, including the backup (if you happen to have a writeable share for backup).

If I was doing this, I would either use two separate PCs, or I would go with a single PC with 6-wide RAIDZ2, but this is derived from my personal weighing up of risks I can think of in five minutes or less.

 

[errr]

Raid of any type is not a backup ...... and no substitute for an isolated, separate backup. I have original disks of all my movies, and shudder to think of what I'd have to go through to re-rip them all ....... A hardware fault can all of a sudden trash the entire server/pool, or a bad combination of disks dieing ...

I cross my fingers with a backup TN server that is only turned on once/twice a month to backup my main box. I also go 1 step further and have it in a detached garage (my cloud). How much do you really value your time and hassle in event of a crisis ????

In my opinion rz1 is useless because if a disk fails in the process of replacing a disk ..... your dead .... Also use ECC ram if at all possible ...

Yeah I understand that raid is not a backup. I guess ill start looking into solutions like yours with RAIDz2 and an isolated backup. Do you think striping with no redundancy would be suffiient for an isolated backup?

 

[errr]

There is a fair amount of complete pool losses never traced to a definitive cause, often because people don't even bother (if backup is available).

This is a good point, I dont think I would investigate much if I had a backup.

If I was doing this, I would either use two separate PCs, or I would go with a single PC with 6-wide RAIDZ2

I think I'm going to do a 6 wide RAIDz2 with a separate pc thats only powered on for backups like @kherr.

 

[Arwen]

@errr Yes, a striped pool can make an effective backup. However, you need to manage your backup pool as well as your main pool. Meaning if your striped backup pool looses a disk, you need to hop on it, and fix it. Because you just lost your backups.

In my case, I am able to fit all my data into a single 8TB disk. (My newer backup disk is 12TB to account for future growth.) For me, rotating through different sets of backup disks allows me to have both more copies of backups. And to have the ability to go back in time if needed.

Plus, I do snapshot the backups, so each backup disk actually contains multiple versions of the backups. I can do this because my source pool is smaller than the 8TB disk. I just check the size of the backup pool before starting a new backup. If the backup pool is getting fullish, I destroy the oldest backup snapshot.

 

[errr]

@errr Yes, a striped pool can make an effective backup. However, you need to manage your backup pool as well as your main pool. Meaning if your striped backup pool looses a disk, you need to hop on it, and fix it. Because you just lost your backups.

In my case, I am able to fit all my data into a single 8TB disk. (My newer backup disk is 12TB to account for future growth.) For me, rotating through different sets of backup disks allows me to have both more copies of backups. And to have the ability to go back in time if needed.

Plus, I do snapshot the backups, so each backup disk actually contains multiple versions of the backups. I can do this because my source pool is smaller than the 8TB disk. I just check the size of the backup pool before starting a new backup. If the backup pool is getting fullish, I destroy the oldest backup snapshot.

With 6x4Tb in RAIDz2 my total usable space is ~16Tb and therefore I need a >16Tb drive to backup my data I’m assuming. For about the same price I could get a single 16Tb drive or two 8Tb drives and then potentially add a third down the line for z1. Seems like that may be a better idea. @Arwen how do you backup your data? Do you have a separate server?

 

[Arwen]

Well, your math is slightly off. Pools should be kept at 80% or less full. Thus, 16TB * .80 = 12.8TB, which would allow you to use a single 14TB disk for a backup pool. And possibly be able to afford a second 14TB disk for another backup pool in a 2 disk rotation.

But, you are right, price concerns DO mater. So a striped pool may be the option due to budget, (I am not following the current price of disks per TB).


In my case, at present I use an external 3.5" disk enclosure, which allows easy swapping of disks. It has USB & eSATA, power brick and fan. The fan was important to me because hours long backups, or ZFS scrubs, can make the disk hot otherwise. See this resource for what I do;

How to: Backup to local disks

I have 2 earlier enclosures listed as well, though I don't use them any more.