Process of learning

[DAXQ]

While learning to set up my FreeNAS system, I have determined that I would like to rebuild my server to get total use of all the drives. When I built it, I had two drives missing, and I now have those drives as well as others for spares.

It is an older HP Proliant with 16 146GB SAS drives and 32G RAM, currently I have:
raidz3 drive 1-7
raidz3 drive 9-15
Both in one volume

Leaving 8 and 16 out of the mix. I would like to rebuild the entire thing rahter thna use the drives as spares if thats even possible, but I have already setup a working replication with another FreeNAS mini at another facility, and would hate to have to redo that two day replication to get seeded. But if that would be the easiest route, it will only build on my learning to have to redo it. Any thought or suggestions appreciated.

 

[Bidule0hm]

It's the only way unfortunately, you can't add single drive to a vdev (but you can add more vdevs to an existing pool).

 

[DAXQ]

Yes I figured as much, was wondering if there might be a way I can try and keep the seed though? I see the replications and the files - just thought there might be a way.

 

[Bidule0hm]

You'll need to put this data somewhere else and then back here, there is no other way.

 

[DAXQ]

That would be perfect, I could hook an external drive easily enough for tempo storage, but is there a process for bringing the data out of freenas? Once the data is out I would likely delete the volumes, reinstall the base on two micro 16g USB drives, recreate the volumes and then bring the seed back in so I can re-enable replication.

 

[Bidule0hm]

No need to reinstall the system if you just want to recreate the pools ;)

The best process AFAIK is to use replication to another ZFS server. But of course you can also use rsync, copy-paste over network, ...

If you want to copy to a USB drive it's best to put it on a client, and not directly on the server.

 

[DAXQ]

I currently have a single USB (1G) drive as the OS drive, and want to replace it with two 16G sandisk micro usb drive(s) with a mirror - so while rebuilding the volumes I was hoping to rectify that little boof as well.

If I:

get all data in .zfs\snapshots off the server and onto an external tempo storage <-- if thats even possible is that all I need?

can I then, save the server configuration off to a thumb or something
destroy everything - the volumes, the os install etc...
insert my two micro USBs and reinstall a clean OS mirror on the two USBs
recreate the volumes using all drives (and with that many drives of 146GB would there be a better setup than what FreeNAS is giving me by default [two raidz3's of 8 drives each]).
restore the server configuration
restore the data
and begin replicating again?

 

[Bidule0hm]

Then, yes, you have a good reason to reinstall ;)

Snapshots doesn't work like that, you can't just copy the .zfs/snapshots to somewhere and copy it back, it's why replication is for.

I think I'll do two RAID-Z3 vdevs of 8 drives but it's my personal choice (and I'm a bit paranoid about data loss) so what I would recommend for a "normal" server would be two RAID-Z2 vdevs of 8 drives ;) I don't recommend to use one vdev of 16 drives, it's really too much (usually we keep the number of drives per vdev to maximum 12).

 

[DAXQ]

I think we have come full circle - the state I'm in now would be very similar to some one needing to seed a server - the data is right there - so how can I get it off the server to re-seed it with the data after I rebuild?

 

[Bidule0hm]

You said you already have replication to another server, right? so use this server to copy the data back.

 

[DAXQ]

Took two days for 43g was hoping to avoid that - otherwise I can just start over create a new replica and let it do its thing. But I cannot believe there is not a way to get the data for seeding.

Rather than replicate again I could drive two hours grab a copy of the data and be back to prime the server. Just seems like I'm missing something. And this is for a small 50g - I couldn't image what someone would do with real data amounts.

 

[Bidule0hm]

2 days for 43 GB? how this is possible? It took me two afternoons for about 2TB and it was a copy from a standard drive (not a SSD) over a gigabit network so nothing fancy.

I think I don't understand where the problem is exactly.

 

[DAXQ]

You must have better internet - rural low speed internet is a terrible thing - which is why I would like to seed.

 

[Bidule0hm]

Oh, I see, the other server is remote. I was talking about LAN...

Well, you don't have another PC at the same place than the server to put the data on temporarily?

 

[DAXQ]

Nothing strong enough for freenas or zfs

 

[Bidule0hm]

No, no need to put FreeNAS on. I was thinking about a simple copy-paste with a share over the network ;)

 

[DAXQ]

OK, so for clarification - lets say I go over to my other facility and grab a copy of my data (using CIFS/NFS etc...)

So I use a client PC not the server directly and access the data on SERVER A via CIFS/NFS/whatever share and just put the 43G onto the external drive.

Now I have a stagnant copy of the data I need (it is in production so will continue to change)

Now I take that drive back to my remote site SERVER B and do what with it so that I can get it into the Replication process over the WAN from A to B? <-- think this is where I am lost in the woods

And if that would work, since I already have successfully replicated snapshots on SERVER B - couldn't I just clone the most recent snapshot & copy it to my external drive (via CIFS/NFS/etc...) and be in the same spot I was in after the drive to my remote facility?

The tricky bit (for me) is what to do with it then - after everything is rebuilt, how do I insert the data on the external drive into a snapshot replication from A to B so that the 43g doesnt need to be replicated across the WAN again, just its changes.

 

[DAXQ]

As I am thinking about this and writing the questions, I am thinking it is probably not possible. If the FreeNAS SanpShots are based on pointers & blocks - and what is replicated is that information (which is why it needs another ZFS file system and how its so small) I don't see how you could every really seed without physically moving the servers to and from a location for the initial seed.

 

[Ericloewe]

For temporary storage, USB HDDs can be a viable solution.

 

[Bidule0hm]

I don't understand where is the problem. Just imagine you don't have the backup server (let's say it's the server B). Copy the data from the main server (A) to an external drive (can be on a client PC drive, on a USB drive, ...), do your changes to the server, and then copy the data back.