Storage configuration & backup

[Keven]

Hi everyone


I’m new to the community. I’m planning a freenas build.

http://ca.pcpartpicker.com/list/fdGwhq

I’m planning to run RAIDZ1 to have some kind of efficiency at the beginning. My plan to be able to grow my storage size and efficiency is to eventually add a backup of my main pool. My backup would not be raid or mirror in order to be easy to add HDD to the backup pool as my storage needs grow. When my main pool is full, I plan to backup this pool to my backup and then I will buy a 5th HDD to the main pool and create a new vdev with all 5 HDD in RAID Z1. After that I would copy back all my data from my backup to my new and more efficient main pool.


If you looked at my build maybe you had notice that my motherboard have only 6 sata port and I will be using 4 at the beginning so for the backup I will only have 2 left. So, is it possible to split the backup in 3 or 4 backup pool, so I only need 1 hard drive connect to the machine at a time to make a partial backup? for example: 1 drive for my movie, 1 drive for my picture… with this method is it possible to also keep the snapshot I plan to make:

1 every day that last 4 weeks

1 every 4 week that last 25 years (like a life time snapshot)


I’m honest to myself and don’t expect my backup system to work with the snapshot, but it would be great.

My must is:

-I can upgrade my main pool one drive at a time while keeping RAIDZ1 redundancy and efficiency.

-keep my archival snapshot


Ps: my backup drive that I will buy eventually will not be online 24/7, I plan to only use them maybe once every 6 month so I expect them to be in top shape when I need them to work for a full backup


If my first plan of connecting 1 backup drive at a time have no way to work, I thought that a barebones PCI-e Sata adapter maybe the solution if nothing else work so I can connect them all at the same time.

 

[Robert Trevellyan]

is it possible to split the backup in 3 or 4 backup pool, so I only need 1 hard drive connect to the machine at a time to make a partial backup? for example: 1 drive for my movie, 1 drive for my picture… with this method is it possible to also keep the snapshot I plan to make:

This sounds like a solution based on backing up individual datasets, which can be done, but there will be significant manual intervention involved. This introduces the weakest link in any backup plan, which is human inertia.

Also, your plan for your backup pool is weak, since during periods when you're expanding your main pool, your data will reside only in one place, with no redundancy.

I thought that a barebones PCI-e Sata adapter maybe the solution if nothing else work so I can connect them all at the same time.

eSATA for external backup is a reasonable approach, if you're careful with hardware selection, e.g. avoid port multipliers.

 

[depasseg]

Since your case holds 10 drives, I'd look into also getting an HBA for the other 4 drive slots instead of an eSATA setup. That way if you wanted, you could house your backup drives and the human factor is removed.

And yes, you can easily configure the replication of one dataset to go to whatever other pool/dataset you want, so that each dataset could go to a different drive/pool/dataset. And while I agree that it's not the best idea, it's certainly better than nothing.

 

[Keven]

This sounds like a solution based on backing up individual datasets, which can be done, but there will be significant manual intervention involved. This introduces the weakest link in any backup plan, which is human inertia.

what kind of manual intervention (is it all doable in the web GUI or not?) this will also keep the snapshot?

Since your case holds 10 drives, I'd look into also getting an HBA for the other 4 drive slots instead of an eSATA setup.

sorry for my unclarity, when I said PCI-e sata I meaned PCI-EXPRESS sata adapter not eSata. I didn't recall "HBA adapter"

As i said a HBA adapter is my last solution because it's not free and i'm kind of already exceeding my budget. the critical data will be also backup on another drive running ntfs just in case something happen to the zfs backup when growing the pool.

 

[Robert Trevellyan]

what kind of manual intervention (is it all doable in the web GUI or not?) this will also keep the snapshot?

If the backup drives are not permanently connected, you will have to initiate the backups manually, which is not well supported in the GUI. ZFS replication preserves snapshots.

I meaned PCI-EXPRESS sata adapter not eSata. I didn't recall "HBA adapter"

If that means keeping the backup drives installed, you will be able to automate the process. The term HBA means 'host bus adapter', which is in contrast to 'RAID card'. Hardware RAID is not what you want. The idea of an HBA is that it gives ZFS direct, full control of the disks.

There are literally dozens of threads about this in these forums.