System dataset on replication server

[bobpaul]

We're running 2 freenas servers; a primary server that's offering services, and a secondary that has all services disabled (except ssh and SMART). Primary is setup to replicate /mnt/storage to /mnt/storage on secondary. In this configuration, should primary suffer a catastrophic failure, we can restore the primary config on the secondary server, reboot, and resume operation. This worked great up through Freenas 9.2.0

Enter FreeNAS 9.2.1.1
Now there's this mostly undocumented .system volume which lives in /mnt/storage/.system. This is used for samba4 and "leveraged for other purposes"; currently logs and "core files".

The Dilemma
If we upgrade both servers to 9.2.1.1, anything that primary writes to /mnt/storage/.system will be replicated to secondary, replacing whatever secondary may have written to that same path. This means any logs secondary writes will get clobbered every replication cycle, likewise any samba4 or "core files" (though since samba is disabled on secondary, this might not matter. I have no idea what a "core file" is.)

So what's recommended? I believe we have 3 options:

1. Create a second pool on secondary just to hold the .system volume. We only have room for 6 disks and they're all already part of the "storage" pool. I'm sure we'll have to delete and recreate that pool, but is it even possible to have 2 pools on a disk? Can this be done with the GUI, or do we need to create partitions on the disks and create the pools in the partitions? How big should the system pool be?
2. Leave it as is. secondary won't have logs; maybe syslog can help here, but otherwise no harm. I suppose we could point the /var/log symlink to a ramdisk or similar. What are the core files? Are these a concern? Or is it OK if primary's core files are replacing secondary's every 15 minutes?
3. Replicate to a subvolume. By that I mean replicate from /mnt/storage on primary to /mnt/storage/replicated_data on secondary. Now there's no file-clobbering, but it seems like it would also take longer to turn secondary into a replacement for primary should a catastrophic failure occur.

Am I overlooking something? How are other people managing this sort of situation?

 

[Dennis.kulmosen]

You can set a new location of .system under System->Advanced
Do that on Secondary and leave primary as default, you should not have any conflicts during the replication with that setup.

 

[bobpaul]

You can set a new location of .system under System->Advanced

You cannot set a location; you can set a pool to hold it. Since I only have 1 pool (storage), I can't select an alternate location.

So you're advocating for option #1? Delete my entire pool, and somehow create 2 pools across the drives? Do I need to partition the disks for that first?

 

[Dennis.kulmosen]

Ahh. Bugger. I just vagely recalled another post saying that you could do that, and i presumed that you could have a dataset to hold this data.
You could actually just have a USB stick holding this dataset. That would be the easiest setup.
And to keep a backup of this USB have et replicate to a dataset on your main pool with snapshots on.


Sent from my iPad using Tapatalk

 

[bobpaul]

You could actually just have a USB stick holding this dataset. That would be the easiest setup.

Isn't there a concern with the longevity of the USB stick if /var/log is writing to it. We've had issued with /var/log on thumbdrives before (PFSense; had to switch to the CompactFlash version of PFSense which does logs to tmpfs).

Hmm... I guess thumbdrives are cheap and brief downtime in the secondary to switch a thumbdrive would pretty much go unnoticed; worst case this kicks the problem a few months away; best case it works beautifully. Alright, I'll give this a try.


Still seems to me that if putting samba4, core files, and persistent logs on the thumbdrive was a good idea, the FreeNAS developers would have done that instead of logging to tmpfs (in previous versions) and logging to one of your storage pools (current version).

 

[cyberjock]

I wouldn't do a USB stick. Might last 24 hours, might last 24 years.