Advantages of new ZFS-on-boot features (boot scrub, verify install, etc)

[joeschmuck]

Well that seems like semantics to me, be it ZFS or FreeNAS, the FreeNAS OS calls ZFS to perform the task.

My issue is indeed, the bootstrap code, the bootable flag, all the core stuff to making a key bootable. I've never tried to boot my machine with the 'cloned key' in it, without the other. It'd be dandy if it just works, I simply don't know.

Note: I'm not talking about the drive dropping out, what FreeNAS does while in use. I mean a USB key dies, the system is powered off and the bad USB key is removed. Does the other one boot, in full, seamlessly?

Maybe. It depends on your BIOS. Look at it this way, in your BIOS you may need to specifically call out a specific USB device or maybe the option is generic for any USB device.

From the corruption point of view, if the primary boot device is corrupt, odds are things just stop and your mirrored USB device just sits there. This is the argument I've made in the past because odds are more the a user will have a corrupt device than one that isn't recognized at all. With that in mind I push using a single SSD over any number of USB flash devices. Heck, you should be able to use an external USB SSD and gain the benefits like using a SATA SSD.

Hope that helps.

 

[diskdiddler]

I agree with your point but it's still (!) unclear to me from all the discussion about this, does ZFS (FreeNAS) actually, entirely clone a USB key, in full - including boot information? So the disk is marked as a bootable volume?

 

[joeschmuck]

I agree with your point but it's still (!) unclear to me from all the discussion about this, does ZFS (FreeNAS) actually, entirely clone a USB key, in full - including boot information? So the disk is marked as a bootable volume?

Yes.

The mirrored device is bootable from what I understand.

 

[joeschmuck]

Additionally, I have not personally tested this because I believe just making a backup of your configuration file as needed is the best action to take and then you can simply rebuild a new USB device and install your config file, all is good again.

 

[jgreco]

Well that seems like semantics to me, be it ZFS or FreeNAS, the FreeNAS OS calls ZFS to perform the task.

FreeNAS isn't really an OS. It's primarily a management layer built on top of existing available services in FreeBSD.

The question here is both about FreeBSD and ZFS. ZFS does the mirroring. Any component of a mirror holds the entire contents of the mirror.

However, the IBM PC boot sequence is outside of ZFS, and relies on some bits and bobs to get things started. FreeBSD handles that with its boot loader. The boot loader can read a ZFS pool. There's no mirroring of the boot loader, it's merely installed on both disks. You might think of that as "mirroring" since they should in theory be identical, but it isn't actually mirroring.

Now here's the part you have to wrap your head around:

We can't say for *sure* what'll happen when booting in a broken environment. If the BIOS is able and willing to load a valid copy of the boot loader, then FreeBSD will initialize and FreeNAS can run. If the BIOS decides to bail, it won't. This raises significant questions in the case that one boot device is corrupt but sufficiently readable that the BIOS tries to load the boot loader from it, and then fails, or decides to cry that it cannot read the boot device, or whatever.

 

[rogerh]

My understanding was that the boot loader was installed on both mirrored devices when FreNAS was installed to two mirrored devices. So that it should be possible to either remove the bad device or simply aim the bios at the other boot device when one fails at the boot loader stage. Trying to bring this about after adding a second mirrored boot device to an already installed system was the subject of a bug report I seem to have lost track of, and I don't think this was achieved.

However, as everyone says, if the boot loader fails it may be best to reinstall, having saved the config file. The mirrored device will still make it possible to correct errors in the operating system and flag up device failure in most situations, just not cope with device failures affecting the boot loader.

 

[jgreco]

My understanding was that the boot loader was installed on both mirrored devices when FreNAS was installed to two mirrored devices.

I think I just finished saying that...

So that it should be possible to either remove the bad device or simply aim the bios at the other boot device when one fails at the boot loader stage.

Well, that gets at the heart of the problem. Ideally we don't want to have to do that. It'd be nice for it to be automatic. It is, of course, automatic in some specific cases, but this is an issue that is reliant on things FreeNAS has no control over... the preboot environment.

 

[DrKK]

Let me just clarify a bit for diskdiddler.

When the ZFS mirror boot thing came out with 9.3 (I guess it was 9.3), there was a problem in that if you ADDED A SECOND DRIVE AFTER THE FACT for the boot devices (if your installation had two disks from the get go, you were fine), it would faithfully mirror the OS/files to the second device, but it would *NOT* set the proper bootloader. So if the first original usb key was pulled, you would *NOT* be able to boot on the second one.

Some combination of cpuroast and I and whomever filed the bug report on that, which generated some controversy and discussion. In the meantime, we all just manually added the proper zfs bootloader for GRUB2 to the second drive, and it was cool. But that was a while ago, I assume that this is all fixed now, and when you add a second boot device after the fact, the boot loader is properly added.

 

[rogerh]

I think I just finished saying that...

True, my point was that it is not clear that even this happens when you add a mirrored device post-install.

 

[jgreco]

True, my point was that it is not clear that even this happens when you add a mirrored device post-install.

Well, obviously it ought to. I'm amazed that it appears no one developing the feature apparently thought to try. *grumbles* amateur hour....

 

[diskdiddler]

Ok so it sounds like the system is clever enough to put on a bootloader before mirroring (but only after the bug was fixed where it wasn't working properly)
None the less, for all intents and purposes, you're cloning your USB key, regardless of the methodology. That's reassuring.
Thanks.

 

[Lazulitone]

Hello Everyone,

I am new to FreeNas and I've had a server running for 8 months. Currently, I find myself in the scenario where my server is running but my boot drive has died. It really didn't last long. Why does FreeNas, with all of its capabilities, rely on cheap USB drives to boot? It's like building a skyscraper on footings made from pancake batter. I spent about $2000 on the hardware for my NasBox and I use a $20 stick from NewEgg to boot. This is crazy. Big mistake; live and learn. I will be abandoning USB, mirrored or not, and trying out an SSD for my boot drive on the premise that this drive will have better memory that can handle the data cycling. I suggest that FreeNas is a lot harder on these USB sticks than we think.

In the meantime, since I'm in a dicey predicament, how do I get my system fixed considering that it's currently running with a bad boot drive? I simply can't afford to lose all of my data. I use my FreeNas Box extensively for work and I have all of my personal data stored there as well. - I've copied my config file...however, I've heard of people not being able to mount the ZFS pool afterwards anyway. This makes me terrified to boot into an ISO and import an old dataset. I've scoured the forums to discover a way to do this but I've come up empty handed. If a tutorial for this does exist, I apologize for not finding it. It's not from a lac of trying.

Thank you.

 

[depasseg]

If you have your config backed up, just power off, install new boot device, remove old boot device(s), install freenas to new device, give it a basic config (to get on the network), then login and restore your config. If you are super paranoid, you can disconnect all your data drives during the freenas install.

 

[Lazulitone]

Depasseg,

Thank's for your quick response. I am clear on your instructions.

Since I have not had to do this before, I'm naturally hesitant. How does the new OS find the old ZFS data pool? I'm more concerned with the data than the configuration at this point. Will the new OS be able to find the ZFS pool cleanly or is there a chance it won't be able to recover the data? Regarding importing the config file. Does the OS require the config file be from the same version? For example, If I install the latest stable build would it be able to import a config file from 4 builds ago? I fear that my data will be left on the drives without a map to recover it.

Is there a way to make a new boot drive from the currently running OS?

 

[depasseg]

I think it would be best to open a new thread. And you should probably read the newcomers guide.

 

[Bidule0hm]

How does the new OS find the old ZFS data pool?

By uploading the config file or auto-importing the pool on a unconfigured install.

Will the new OS be able to find the ZFS pool cleanly

Yes.

or is there a chance it won't be able to recover the data?

No.

Does the OS require the config file be from the same version? For example, If I install the latest stable build would it be able to import a config file from 4 builds ago?

Yes, but it's recommended to use the same build.

I fear that my data will be left on the drives without a map to recover it.

That's more a problem with hardware RAID than software RAID. Don't worry, your data is here, you'll get it back, the OS is well isolated from the data pools.

 

[Lazulitone]

Depasseg, Bidule0hm,

Thank you for your kind support! I will give it a shot.

 

[danb35]

Why does FreeNas, with all of its capabilities, rely on cheap USB drives to boot?

This might be picking a nit, but if so, it's an important nit to pick. FreeNAS doesn't rely on USB drives, cheap or otherwise, to boot. Many (perhaps most) FreeNAS users rely on cheap USB drives to boot. The manual has given alternatives for a long time, and iXSystems hasn't shipped USB sticks as boot devices in years. Two things have changed in the last couple of years, though, that make USB sticks less of a good choice than they once were:

• SSDs have gotten a lot cheaper, so they're now a viable and inexpensive option for a boot device; and
• Starting a couple of years ago with FreeNAS 9.3, the boot device became a live ZFS pool. Before that, it was a fixed-size UFS filesystem that was loaded into a ramdisk on boot. The only time it was written to was when you made a config change. The ARC still means that the large majority of the OS will be sitting in RAM at any given time, but it isn't guaranteed as it used to be. The increased IO on the boot device isn't compatible with a predictably long lifespan for a USB stick.

 

[Arwen]

@danb35
I would add that;

• USB sticks have become such a commodity that low quality, (and even out right fraud in sizes), has become the norm for the cheaper ones.

Personally I would now test a USB stick with both sizes checks and perhaps 10 full writes and read verifications before putting one into use.

 

[Mirfster]

10 full writes and read verifications

If you did that, then you might have just consumed 1/2 the stick's life capacity... ;)

I vote that the whole USB suggestion be removed altogether IMHO.