Process of learning

[rogerh]

I don't understand where is the problem. Just imagine you don't have the backup server (let's say it's the server B). Copy the data from the main server (A) to an external drive (can be on a client PC drive, on a USB drive, ...), do your changes to the server, and then copy the data back.

But as he says he can't really make use of the remote replication backup that way (except as a reserve backup in case the one he is using fails). And will almost certainly have to repeat the whole original replication from scratch across the Internet.

There might be some way to do it if he had a portable zfs machine and could physically transfer a replication of his server to the remote machine between routine snapshots.

 

[Bidule0hm]

Euh... Yeah... That's why I said "Just imagine you don't have the backup server", I was suggesting a local copy, not over the WAN.

 

[rogerh]

Euh... Yeah... That's why I said "Just imagine you don't have the backup server", I was suggesting a local copy, not over the WAN.

One of his main objectives was to rebuild the pool and continue incremental remote backups (without having to repeat the original full replication) - I don't think he is doubtful about how to save and reload the pool data apart from this objective. Even if there is a theoretical way to do this, I doubt if it would work in practice.

 

[Ericloewe]

You can always replicate to a second pool on 127.0.0.1

 

[Bidule0hm]

Oh, ok, I understand now. The problem is after this problem is solved to not redo the initial backup. Well, I don't see how to avoid that with a simple method unfortunately.

 

[rogerh]

You can always replicate to a second pool on 127.0.0.1

That of course is the answer, and ideally done using a (or two mirrored) spare SATA ports on the original machine as a 'spare' pool with almost any >43G drives. With slightly greater risk it could be done to mirrored USB drives. The only problem then is to understand precisely how to synchronise the snapshots on the home and remote server when he has replicated the temporary store back to the rejigged main pool. There may be something in the manual about this, but it sounds complicated to me if a significant time has elapsed and they are not both on the same snapshot.

 

[DAXQ]

If the data (aka the shares etc...) on A or the data on B was all I needed I could do just what you guys are describing. What I am trying to do rebuild a snapshot replication for a remote site. Typically - also refereed to as seeding a server in a remote location.

I am thinking it is not really possible - given the way ZFS works (of which my understanding is severely limited).

Server A in Chicago is working great running snap shots daily. Need to get those snap shots backed up (as ZFS & Raid are not backups - they are reliable storage only). So I configure Server A to replicate its snapshots to server B in Dallas City. the first snapshot will need all 100G - subsequent snapshots will only need the changes - and that's really what I am after - the changes only - want to try and speed up the initial 100g seeding of the snapshots.

If I let the replication run its normal course (after properly being set up) it will move 100G of data across a WAN with VPN tunnels which could take a few days.

Rather than doing that it would be nice if I could go to Chicago, physically pull something off server A and take it to Dallas City to get replication started without having to use the WAN for its initial 100G data transfer.

Short of taking a 150lbs server B to Chicago with me, spending the day up there while the replication occurs locally first - I am thinking it cannot be done.

Thinking it would be nice if FreeNAS/ZFS had a mechanism for doing this - as I would expect it to be a pretty common thing during backup prep or even restoration. If things crash in Chicago, and I need to restore the entire snapshot from Dallas City, rather than taking two days for the data to replicate across the WAN & VPN, it would be nice if I could physically drive the data there to get my server rebuilt.

 

[DAXQ]

Sorry I replied before my browser refreshed the posts - I did not see that you guys are getting it, and we agreee it would either be pretty complicated, or not "doable".

Thanks for the confirmation, may just let it ride till I am forced to make a change.

 

[mjws00]

This is baked into ZFS. You can replicate to nearly ANYTHING including a flat file. It is just a stream to STDOUT. Replicate to the temp file or pool on a USB or Disk. Move the disk to the second location. Receive the snapshot on the second server. Replicate the difference across the wire.

You are outside the realm of the GUI, but smack dab in the center of what ZFS is designed to do. Most of the time we see folks 'seed' a large pool during rollout on site BEFORE shipping the box. The GUI can get you through that. That cool little backup/restore they added might also be useful in such a scenario.

Just because you can't do it with two clicks in a GUI doesn't make it complicated or not doable. It is like 5 commands, fire up a few VM's do some reading and go from there. A->B, B->C, A->C (incremental).

For 43GB. Have a helper plug in a USB stick or better yet a hot swap ssd. SSH in do your magic. Send the stick. Done. Not like this is 2 racks of gear and hundreds of TB.

 

[DAXQ]

With that reply I am definately hopeful! Not scared of console at all kinda prefer it - guessing that I am reading the wrong stuff though. I have been reading the 9.3 FreeNAS doc's - fory situation what might be a better guide?

 

[mjws00]

Nothing complicated. Google zfs replication. Even the man page has examples. This is 101 stuff not hardcore. Sorry for the terse reply from the phone. Mav@ did a nice quickly command dump a few days ago helping someone move a data set. Very similar. Unfortunately I'm too slammed to offer much more...

There are a dozen guys could walk you through. Most will hesitate in fear of you screwing up your data. Also it isn't something done every day as this is just not practical for large data pools. We almost always have to temporarily co-locate to seed efficiently. Think 100TB.

 

[DAXQ]

I have been focusing more on freeNas docs - thanks I will do that. My data is no where near the TB range and this whole endevour is to learn it thanks for the pointers and where and what to look for.

 

[DAXQ]

Would a book like solairs 10 ZFS essentials be helpful? Kinda like holding a book

 

[mjws00]

Sorry, no idea on paper books. I would hesitate to mix FreeNAS and Solaris without knowledge of where things differ. They are working on a more flexible replication system for FN and will get there eventually. But for now we gotta use the CLI. There are no destructive commands issued... so risk is minimal imho. I'd break out the commands, but would want to verify first hand using VM's before spewing commands. Someone more intimate with the flags, could just do it on the fly.

Truth is we are talking about 2 commands, not zfs internals. Google 'zfs send receive' or 'zfs incremental send receive' first page will easily cover this. This is a decent blog / example of what is up. http://cuddletech.com/blog/pivot/entry.php?id=984

Here is a nice outline by @mav@ discussing moving a dataset. A couple tweaks, and an incremental flag and you are almost there.

 

[DAXQ]

Awesome was just checking reviews and thinking the same thing - I will check your links and the googling! Thanks for the info and direction.

 

[DAXQ]

Ok that cuddletech link sounds like the hammer needed! Can't wait to get back to work and start playing with it! Thanks a ton!

 

[DAXQ]

In my current situation, I have a full blown FreeNAS Mini (ARC & ZIL and 16g RAM) as my "in production" NAS - it is live and working out well (have learnt since the classes from iXsystems that either my ZIL or ARC may not be used in my mainly CIFS setup, but that is the server currently in production). I built a second server out of an older HP Prolaint server that has been retired. It is a 64 bit system that has 32G ram and 16 146G serial scsi drives that I will keep in my office at a remote site. It will be used as the replicated backup for the Mini - so everything I do on the HP is a work copy of the data. It will suck if I need to rebuild it as original, but certainly wont make me curl up into a ball and cry (and the more times I can do it the more familure I shold get with it). The HP is not ideal with its underlying RAID controller jerry rigged as JBOD, but it seems to be working adequately. I have learnt the hard way that nothing on this set up is "HotSwap". If a drive fails, it will will need to be Offlined in the FreeNAS, server powered off, drive removed from RAID controller and rebuilt, then readded to the FreeNAS system.

After the brilliant info I got over the weekend with this thread, I am feeling more confident that this can be done (and as told is what this stuff was built for), while at the same time I have also realized that this swimming hole is way deeper than I ever would have imagined.

So I have my Mini setup and running nightly snapshots of one dataset.
Those snapshots are replicated nightly from Mini --> HP and it seems to be working great. I have never attempted a restore on the Mini, however if I clone the snapshots on the Proliant - I have access to the data from the Mini.

To rebuild my HP server - I am expecting I will need to:

• Connect a 1TB USB drive to the HP and set it up as a single volume called USB_VOL
• Send last nights successful snapshot from the HP to the USB drive with:
  • zfs send -R volume/replication/userdata@auto-20150414=1w | zfs receive vF USB_VOL
  • offline and unmount the USB_VOL set aside
  • FreeNAS web gui --> detach USB_VOL
• Disable snapshots and replication on the Mini (so no more changes are sent to the HP)
• rebuild the HP the correct way using all drives and add dual mirrored mini USB drives for the FreeNAS OS.
  • so 2 raidz3 of 8 disks each making one Volume
  • 1 mirror of 2 sandisk 16GB usb cruzer fit drives for FreeNAS OS
• Once the server is up and running
• restore the snapshot from the USB drive with
  • connect USB Drive and power on
  • FreeNAS Web Gui --> Import Volume
  • zfs send -R USB_VOL@auto-20150414=1w | zfs receive -vF volume/replication/userdata
  • (trying to keep the volume names the same as previously laid out)
  • FreeNAS Web Gui --> Detach USB_VOL
• Re-enable snapshots and replication on the Mini probably needing to update the remote host key in the replication to get it running again.

Am hoping that should do it. Expect there might be issues with key's

 

[DAXQ]

Say is it possible to just add one of my 16g USB sticks to the install make the OS install a mirror then replace the 1g stick with a second 16g? I dont see the OS volumes in the GUI so thinking if it can be done might have to go CLI to do so?

 

[Ericloewe]

1GB? Are you sure about that? The OS is significantly larger than that.

I recommend you backup the config file and do a new install on the 16GB drives. Then upload the config file and everything should be back the way you left it.

 

[DAXQ]

yes 1 gigabit thumb drive and it's still only taking about 60 percent of the drive even with upgrades surprisingly small for everything that it does thanks for the reply will do as recommended