upgrading storage raid10 or raid z2 or ...

[rwps]

I think I'm clear now. Given:
- reducing possibility of data loss is 1st priority,
- I can live with speed issues of Z2 (compared to mirror), as I understand I'll still be able to stream video/music (the main stress i'd place on the NAS) or am I wrong here does Z2 cripple speed that much with ethe N36L procesor (link to benchmark below)
- I'm limited to 4 drives with the current box (HP proliant N36L)
- I'm not expecting to need more than 3.5TB in the next few years (it's taken me a long time to get to 1.6TB)

I'll go with raid Z2 for the 4 x 2TB

The only question is what's the best method to backup data offsite to the USB HDDs (1x 3TB, 1x 1TB):
- rysync to UFS
- rysnc to ZFS
- ???......

N36L benchmark
http://www.cpubenchmark.net/cpu_lookup.php?cpu=AMD+Athlon+II+Neo+N36L+Dual-Core

 

[cyberjock]

I can't vouch for the best method for backups.. I'm too new with FreeNAS. I know what rsync does, but I've never actually tried to use it. I try not to give advice on things I haven't done myself or at least understand thoroughly.

Personally, I think rsync to ZFS would be best. It seems that in the forums people regularly mention that there is very little that UFS does that ZFS doesn't do better.

I don't buy, use, or recommend AMD CPUs(It's a personal thing that goes way back, don't take my opinion of AMD as a reason to not use them.) but I'd guess that any CPU with 2+ cores will be plenty for your intended use.

 

[rwps]

Thanks noobsauce80, appreciate your analysis & perespective, cheers

 

[louisk]

I think it depends on what the backup destination looks like. If it's another FreeNAS box, the easiest would be ZFS send/recieve of snapshots. If its not going to be FreeNAS, or if you aren't going to have the same disk layout, probably rsync, but then you lose the timeline that snapshots provide, so your backup isn't really over time but simply a binary "yes or no".

 

[rwps]

makes sense, I'll do snaphots on the freenas box itself, and will have to settle for the offsite backup to usb HDDs being through Rsync & effectively a single "snaphot" in time.

 

[rwps]

And now to get the data into the new freenas setup I'm planning on.

Currently all my 1.6TB data is on a 2TB disk in UFS, I have another 3 x 2TB disks. Is there anyway to endup with those 4 disks in raidz2, with the 1.6TB data intact without using another disk?

I'd prefer not to use my 3TB usb hdd to copy the data over to a clean raidz2 setup as that would mean there is only that one single copy of my data on the 3TB usb HDD for the period until the raidz2 has the data on it.

 

[paleoN]

No, unless you used striped mirrors which you decided against. Even then it's not straight forward. You would:

1. Create the first mirror
2. Copy the data from the 2TB UFS to a temp directory you create on the mirror
3. Add second mirror to the pool, destroying the data on the 2TB UFS drive in the process
4. Copy, not move, ALL data to final location in your pool
5. Delete initial temp copied data

I would just use the 3TB drive. Besides isn't there only one copy of the data now?

 

[rwps]

not what I wanted to hear, but what I thought might be the case from my reading.

Handy to know though it can be done with the striped mirrors setup, even though not for me this time.

Happens that at the moment I have at least 3 copies of all data, one in NAS and 2 kept offsite (and a few GB of key personal data in the cloud too). I know disks can fail, I'll back up as much data as I can on to smaller disks I have around so that irreplaceable data (i.e. photos, personal videos, docs) is still offisite & more than one copy. And will live with the risk that the other data (DVDs essentially) is at more risk for that small window, knowing I can replace it from originals.

 

[rwps]

now a permissions issue

I've now got the box setup with raid z2, and managed to master putty to use SSH to then use rsync to copy the data into the raid from the usb hdd I mounted.

Now I'd like to backup some data from the raid to another external HDD. It's a small disk, 1TB, so won't fit all the raid data on it.

I can't see how to use rsync to selectively copy folders across without doing a separate operation for each folder.

So I thought I'd map the 1tb to a cisf share, and then copy through explorer. However a snag there, I can read/write to the drive I've mapped for the raid, however the new mapped drive to the 1TB hdd says I don't have permission. Yet on freenas they both have identical setup:




any thoughts on how best to manage getting data across initially, and also syncing new data to this drive in future from the specific folders it is backing data up from?
Thanks

 

[paleoN]

Now I'd like to backup some data from the raid to another external HDD. It's a small disk, 1TB, so won't fit all the raid data on it.

I can't see how to use rsync to selectively copy folders across without doing a separate operation for each folder.

I know you can exclude folders with rsync. I believe it's something like this:

Code:

rsync -options --exclude 'dir1' --exclude 'dir3/file4.txt' source/ destination/

So I thought I'd map the 1tb to a cisf share, and then copy through explorer. However a snag there, I can read/write to the drive I've mapped for the raid, however the new mapped drive to the 1TB hdd says I don't have permission. Yet on freenas they both have identical setup:

I wouldn't rsync is better. How did you create the folder? It could be a local permissions issue.

any thoughts on how best to manage getting data across initially, and also syncing new data to this drive in future from the specific folders it is backing data up from?
Thanks

Get rsync working ;)

 

[rwps]

I now you can exclude folders with rsync. I believe it's something like this:

Code:

rsync -options --exclude 'dir1' --exclude 'dir3/file4.txt' source/ destination/

Thanks will give that a go

How did you create the folder? It could be a local permissions issue.

the folder I can't access is a newly formatted & mounted UFS volume. So no permissions set on it (by me), hence why I find it a bit odd.

 

[rwps]

rsync command for selective backup

So would this work to do the selective backup where all is copied from "ourdata" to "backup1tb" except "dir1" and the file in "dir3":

Code:

rsync -aPS --exclude 'dir1' --exclude 'dir3/file4.txt' /mnt/ourdata* /mnt/backup1tb 

 

[rwps]

and yes, that line of code for the rsync backup worked a treat, thanks for the tip paleon.

I also figured out what the issue with permissions was, it wasn't the permissions on the share, but the permissions on the storage volume, clicking on the icon to edit permissions next to the volume (on the storage screen) and ticking all the boxes got it working just fine. Still on the learning curve!

For other with this issue the edit permissions icon looks like this:

 

[rwps]

as an aside, I discovered that the interface & the functionality I was used to with Freenas7 has been continued in the nas4free project. And given the only additional thing I wanted was raidz2, i was very pleased to find that nas4free has that too.

So I have now migrated my zzpool over to nas4Free, and with it's inbuilt local rysync capability & file manager I'm finding it more suited to me needs.

Though deviating via freenas8 has certainly helped me learn a good bit about ssh commands. So I am much appreciative of the guidance I've received.