Just how crucial is the .system dataset?

[Robert Smith]

Contemplating future FreeNAS designs where storage and configuration pools are separated, I am trying to understand how resilient the .system dataset pool needs to be. Is there a need for mirror (or better)? Or is the .system dataset safe to loose, and everything except logs will simply be recreated on reboot?

First of all, to those running FreeNAS Active Directory (Domain Controller); I understand it is stored on the .system dataset. In that case your .system dataset is precious; make sure you choose the most stringent options to safeguard it.​

But, for those who do not run Domain Controller on FreeNAS, I am sure that is most of us, is there anything else on the .system dataset that demands high level of safety?

On a related note; has anyone successfully achieved using something like a couple of SSDs for both mirrored boot device, and mirrored .system dataset?

 

[dlavigne]

Did you find a satisfactory answer for this?

 

[Robert Smith]

Not yet: waiting for somebody else to experiment with this stuff, so I do not have to. LOL

 

[cyberjock]

Well, if the .system dataset becomes inaccessible because the zpool becomes inaccessible while the system is on, the system will fail completely. You'll be on your own to take the risks associated with doing a reboot with a system in some indeterminate state.

Personally, I recommend redundant pools for the .system dataset as it's not worth finding out the hard way that not only did the nonredundant pool go up in flames, but the other pool went up in flames due to an inconsistent ZFS state when you had to hit the power button on the server (and you *will* have to hit the power button on the server when the .system data set goes away). The developers recommend redundant pools for the .system dataset too.

Bottom line, don't try to engineer your way out of a redundant pool for .system. Even if you want to take that small risk of killing some other pool(s), there is no guarantee that some change won't come out next week that will totally negate the discussion we are having now. The FreeNAS manual recommends redundancy for .system, and the developers are going to make their future design decision based on the fact that you wouldn't do something silly like ignore their warnings. To do so may bring for the wrath of the FreeNAS and ZFS Gods, who may smite you.

And to be frank, if someone posted and said "my redundant pool failed after my other pool with my .system dataset crashed and I had to hit the power button" I wouldn't even respond to the post. It would be the kind of thing where I'd think "well, what do you think you'd get for ignoring the recommendations of the developers?"

 

[Robert Smith]

There has to be some tolerance toward hitting the power button, after all most NAS systems do not have redundant power supplies, and even with redundant power supplies the power backplane can still fail and take the whole system down unexpectedly.

Anyway, if a redundant .system dataset is recommended, and redundant boot devices are recommended, then it looks like the next logical step to put the two together.

 

[sef]

Putting the system dataset in with the boot pool (which I have in fact argued for) means that the size requirement goes up significantly -- the minimum size I argued for would be 64g. And you really wouldn't want to use thumb drives any longer, either SSDs or SATA-DOMs only.

As it currently stands, the .system dataset lives in one of your data pools. If your data pool is not redundant, there's not a lot of point to using FreeNAS, honestly, so you should reconsider that if so. But, either way: the .system dataset is as resilient as your data pool is.

 

[cyberjock]

Well, yes, but no.

The boot devices *were* meant to be devices like USBs that have a relatively low number of write cycles. The .system dataset on the other hand keeps logs, reporting data, etc as well as any files that need to have static locations that don't disappear on a reboot. Prior to the .system dataset the reporting data was stored on the boot device (only updated every 2 hours) and all logs were kept on the RAM disk. But since various things (most notably) Samba4 required a more permanent location that wouldn't be lost on shutdown the .system dataset was born. It was just a natural extension to add logs (losing logs on a RAM disk when a system crashes makes troubleshooting difficult).

There may be changes in the future (10.x is going to have a lot of changes). But until then things are what they are.

I wouldn't say redundant boot devices are 'recommended'. None of my machines have redundant boot devices. Most home users likely wouldn't benefit from redundant boot devices. But if a boot device failure would be a significant problem for you and you'd rather wear out 2 devices instead of one for the possibility of avoiding a boot device failure, then go for it. It's all about risk mitigation. Only you can justify for your situation the risks (and rewards) from going with redundant boot devices.

I can restore any system in 15 minutes. As a home user I am totally okay with that potential downtime. ;)

Edit: sef answered while I was writing.

 

[Robert Smith]

Putting the system dataset in with the boot pool (which I have in fact argued for) means that the size requirement goes up significantly -- the minimum size I argued for would be 64g. And you really wouldn't want to use thumb drives any longer, either SSDs or SATA-DOMs only.

As it currently stands, the .system dataset lives in one of your data pools. If your data pool is not redundant, there's not a lot of point to using FreeNAS, honestly, so you should reconsider that if so. But, either way: the .system dataset is as resilient as your data pool is.

Look at this from the other way around: If I am already going to have two SSDs to host the system dataset, then may as well put the boot device on them.

 

[rogerh]

I wonder if having mirrored boot devices not only protects against the inconvenience of server failure but also protects against silent corruption of operating system files which could conceivably damage data before it is discovered? Is this so unlikely as to be not worth considering? Or will ZFS detect this and effectively prevent use of the corrupted files?

 

[sef]

ZFS checksums blocks, and can determine if there is an error that way. However, it cannot repair it; for that, you need redundancy, such as with a mirrored pool.

 

[owensjc]

I run FreeNAS 9-3 as my primary domain controller.
I would like to see the samba4 configuration removed from the system dataset and given it's own location.
I have FreeNAS installed on a mirrored pair of 32GB SSD's so I would like the option of locating the samba config there.
Having the option of locating it on the system drives and/or a datapool would be the best.
Having an option to backup the samba config with the system config would be a great idea to.

 

[sef]

The system dataset is on the data drives.

 

[Bidule0hm]

He wants an option in the GUI to put the Samba conf file (and why not the system dataset) on the OS drive instead of the data drives ;)

 

[owensjc]

He wants an option in the GUI to put the Samba conf file (and why not the system dataset) on the OS drive instead of the data drives ;)

Exactly! 32 & 64GB SSD's are fairly cheap now, so using these as mirrored OS drives there should be plenty of room for the system dataset and the samba config.

 

[solarisguy]

Exactly! 32 & 64GB SSD's are fairly cheap now, so using these as mirrored OS drives there should be plenty of room for the system dataset and the samba config.

Yes!