Process of learning

[rogerh]

yes 1 gigabit thumb drive and it's still only taking about 60 percent of the drive even with upgrades surprisingly small for everything that it does thanks for the reply will do as recommended

Is this FreeNAS 9.3? It must be one of those Chinese USB sticks one hears about where they put in an 8GB chip and get it to declare itself as 1GB!

 

[DAXQ]

I'm pretty sure its Chinese - but yup its a gig and only using about 63% (see screen shot) I dont use jails or anything else so its been fitting fine.

 

[Ericloewe]

Is this FreeNAS 9.3? It must be one of those Chinese USB sticks one hears about where they put in an 8GB chip and get it to declare itself as 1GB!

What's the point of that?

I have heard stories of 128MB devices with hacked firmware that pretended to be much larger and then either silently dropped excess packets or cycled over existing data.

 

[DAXQ]

What's the point of that?

I have heard stories of 128MB devices with hacked firmware that pretended to be much larger and then either silently dropped excess packets or cycled over existing data.

I dont think its fake or misquoting its size, its a gig and says its a gig. Just a cheap 1 gig we typically use for data transfers from PC's to AG equipment. When they stop reading we pitch em.

 

[rogerh]

I dont think its fake or misquoting its size, its a gig and says its a gig. Just a cheap 1 gig we typically use for data transfers from PC's to AG equipment. When they stop reading we pitch em.

I am mystified that FreeNAS 9.3 will fit in 602MB. Is it compressed in some way? I didn't think that was an option.

 

[DAXQ]

I am mystified that FreeNAS 9.3 will fit in 602MB. Is it compressed in some way? I didn't think that was an option.

I assure there is nothing fancy about what I did - I am just learning this system - I used what was laying around before reading the doc's (like most adventurers out there I am sure just to see what it was all about) and just did the defaults for everything [kinda like things Vanilla]. Everything has just been working so I kept going. Now I am to the point where I want to (re)build it correctly.

Does the install you run consume considerably more?

 

[rogerh]

I can't remember how much space the original install took, but with 12 boot environments mine is 5.6GB. And the manual suggests a minimum size of 4GB for the install plus 2 updatess. I do recollect it being over 1GB though! It does seem possible that not everything got installed in your case?

 

[DAXQ]

I can't remember how much space the original install took, but with 12 boot environments mine is 5.6GB. And the manual suggests a minimum size of 4GB for the install plus 2 updatess. I do recollect it being over 1GB though! It does seem possible that not everything got installed in your case?

Not a clue, but it's been working so well I wanted to do it right.

 

[DAXQ]

After installing FreeNAS (and I saw this last time I am sure) there runs a bunch of ........................... and every once in a while a + so it looks like:

.......................................................+...............................................................................................................................................+....+................... etc... scrolling across the screen.

Last time I think I rebooted the server and it booted fine - but would this be that the server is doing?

Never mind just stupid and forgot Generating DH parameters - this is going to take a long time - sorry stupid ?

 

[DAXQ]

Can't make an omelet without breaking a few eggs - here we go:

Disabled all snapshots and replication from FreeNAS Mini
Connected 1TB USB drive to HP and set up as USB_VOL

zfs send -R AdrianVolume/IowaRep/UserData@auto-20150412.2100-1w | zfs receive -vF USB_VOL

Using FreeNAS web GUI → detach USB_VOL
Destroy all volumes (no point in keeping them or having the config reference them) and data
FreeNAS Web Gui → System > General > Save Config
Shutdown
Inserted two 16 mini USB drives
rebooted server - re-installing
reinstalled selecting both mini USB drives so they would run as a mirror.
Web Browse to server
Updated the new install (config would not install versions were different).
Uploaded config
Rebooted
Rebuilt Volume using all Disks (gained a .2 TB)

• AdrianVolume
  • RaidZ3 Disks1 - 8
  • RaidZ3 Disks9 - 16
  • All disks physical Serial Number and bay location documented for swaps.

Attached USB drive with original snapshots
Imported Volume

zfs send -R USB_VOL@auto-20150412.2100-1w | zfs receive -vF AdrianVolume/IowaRep/UserData

Detach USB_VOL
Re-Enable SnapShots on FreeNAS Mini
Re-Checked replication, made sure the Remote ZFS path was correct
Re-Entered the Push Key into Root user on newly rebuilt Pull
Re-entered the new Key generated by the rebuild into the Push Replication job.

Will see if the snapshot creates tonight and if it kicks off the replication!

Even if the replication fails - I still learned how to move data from one server to another in case of needing to rebuild, and learnt more about the ZFS stuff - would say it was a good day!

 

[rogerh]

Sounds good! Hope the replication picks up where it left off!

 

[DAXQ]

Its replicating all new (so that part of it failed), but on a positive note - it is replicating, so I know I got that bit worked out and rebuilding the server was not bad - also a good experience. Also learned about moving data around which is really the main point of being able to restore backups - pulling the data off the servers and getting it from one place to another is critical.

Cant wrap my head around the ability/possibility to seed (unless you can physically get the remote server to the origin) or re-instigate/restart a replication after its had to be rebuilt also seeding the base pool just can't seem to grasp.

I am also experiencing a weird path thing (its trivial, but annoying):
On server A I have FNMini-Pool/UserData
So on server B I build ServerB-Pool/UserData

When I replicate from A -> B using ServerB-Pool/UserData as the Remote ZFS volume/dataset - I end up with ServerB-Pool/UserData/UserData

So instead of the replication storing in ServerB-Pool/UserData as I want, it creates a new dataset.

Is this by design? or am I entering something wrong?
Is there also a way to stop a replication once its begun? or do I just need to let it run?

 

[rogerh]

Did you use the GUI for replication? When I replicate Pool/Dataset1 to Otherpool/backup/Dataset1 I don't get any duplication, I get the contents of the original Dataset1 in the replicated Dataset1. You are using FNMini-Pool/UserData as your regular snapshot task, and replication source, not just FNMini-Pool, aren't you? If you just used the original pool you would get the result you describe. If you are doing the latter it would better to name the destination dataset something along the lines of ServerB-Pool/backup-FNMini. It makes no difference to the result, but it is less confusing

 

[DAXQ]

Did you use the GUI for replication? When I replicate Pool/Dataset1 to Otherpool/backup/Dataset1 I don't get any duplication, I get the contents of the original Dataset1 in the replicated Dataset1. You are using FNMini-Pool/UserData as your regular snapshot task, and replication source, not just FNMini-Pool, aren't you? If you just used the original pool you would get the result you describe. If you are doing the latter it would better to name the destination dataset something along the lines of ServerB-Pool/backup-FNMini. It makes no difference to the result, but it is less confusing

It seems if I use the name (e.g.)

FNmini-Pool/UserData --> replicated to RemoteVolume/IowaRep/UserData = RemoteVolume/IowaRep/UserData/UserData

But if I leave the name off like:
FNmini-Pool/UserData --> replicated to RemoteVolume/IowaRep = RemoteVolume/IowaRep
(so it is putting the data into IowaRep rather than recreating UserData like I would expect.

Really weird, but I can live with it, where ever it starts replicating successfully is where I will leave it.

And yes, I did use the GUI and automatic snapshots with an added replication task. I deleted all the replications and snapshots and will try and just start it over again tonight @9pm - if its still running when I come in, I will disable further snapshots until the first job gets done. Had a tough time deleting some of the snapshots, but eventually got them all removed.

 

[rogerh]

That's very strange - its not what I get. I am replicating to a zfs on Linux pool, not that that should change something like this.

 

[Alvin]

What's the point of that?
I have heard stories of 128MB devices with hacked firmware that pretended to be much larger and then either silently dropped excess packets or cycled over existing data.

Those are true. I have a stick like that. The're sold on Chinese markets and are "made for tourists". :)

 

[DAXQ]

Ill keep messing with it an see what it does tonight, like I said, i'll live with whatever it does as long as I can get to it. Weird things like this are what begin to bring questioning into the mix for something you don't want to be questioning.

 

[rogerh]

Ill keep messing with it an see what it does tonight, like I said, i'll live with whatever it does as long as I can get to it. Weird things like this are what begin to bring questioning into the mix for something you don't want to be questioning.

I think I have sussed what the GUI does. (I haven't looked at the underlying commands - that's why I am using FreeNAS.) Say you have a dataset D1 you want to replicate from PoolA/D1 to PoolB/Bk/D1:

a) if you replicate from PoolA/D1 to PoolB/Bk/D1, having previously created PoolA/Bk/D1 and not ticked the "Initialize remote side for once …" tickbox you get PoolB/Bk/D1/D1/<contents>, as you point out.

b) if you replicate from PoolA/D1 to PoolB/Bk/D1, having previously created PoolA/Bk/D1 and do tick the "Initialize remote side for once …" tickbox you get PoolB/Bk/D1/<contents>, which is I think what you want. (D1 is first destroyed then created again.)

c) if you replicate from PoolA/D1 to PoolB/Bk/ having not created D1 on PoolB and not ticked the "Initialize remote side for once …" tickbox you get PoolB/Bk/<contents>, which, as you observed, is not what you want.

d) I have not tested what happens when you replicate from PoolA/D1 to PoolB/Bk/ and do tick the "Initialize remote side for once …" tickbox, it may be as b) or as c). I'll leave trying that to someone else.


So it is consistent, and if you use method b) you will probably get what you want.

 

[DAXQ]

Thanks for the info, I know for sure that I never ticked "Initializ..." for fear of the "remote data will be lost..." but now hearing your explanation it makes sense and I will not fear in the future!

This last rebuild of the replication is doing exactly what I wanted and I had deleted all snapshots on both servers, deleted replication job, and auto snapshot job (to just rebuild the entire thing). This time I did:

FNmini-Pool/UserData ==> AdrianVolume/IowaRep

Before starting things, I deleted everything (datasets and all)

And I am getting AdrianVolume/IowaRep/UserData on the receiving end as I would hope.

 

[rogerh]

A good outcome, though not necessarily what either of us expected from previous experiments!