Upgrade from 12.0-BETA2.1 to RC1 failing

[coolnodje]

Upgrading thru the GUI seems to be failing.
I don't seem to have a way to understand what is going on as I can't even SSH the machine after a while and it never reboots (I gave it a night).

The only error message I saw was something in the lines of "Not enough space", and it was displayed on the GUI page where I started the upgrade. Quite surprising, I don't understand how this could be the case.
The GUI was still responsive and I was able to restart the upgrade from the GUI.

But then it never finished. I hard reset the machine after a night, restarted the upgrade, but it's not getting any better.

What are my options to understand what fails?

 

[c77dk]

Hmmm, given the "not enough space" error you might want to check for free space on your boot pool - maybe you have a lot of old environments you can delete?

 

[Yorick]

Though old environments should be auto-deleted in that case.

 

[coolnodje]

Yeah I should double check that. It's really only because of this weird error message I had.
I never had to consider space on the boot pool in years of upgrade.

I've had upgrade failing, but they'd run fine the second time after a hard reset.

This is a bit mind boggling.

Still, which logs shall I have a look at to understand precisely what went wrong?

 

[coolnodje]

This is what my my boot pool says:

Filesystem Size Used Avail Capacity Mounted on
freenas-boot/ROOT/FreeNAS-12.0-BETA2.1 4.7G 1.1G 3.6G 23% /

I doubt there could really be a space problem here.

 

[Yorick]

That's a minimum-sized 8GB boot drive? When you go to System->Boot, do you have old environments you can delete?

 

[coolnodje]

I've been able to delete quite a few files from System->Boot, thanks @Yorick! (never paid attention to this menu item before)

It's an 8Gb drive.
Not sure why df reported 4,7GB, now it reports:

Filesystem Size Used Avail Capacity Mounted on
freenas-boot/ROOT/FreeNAS-12.0-BETA2.1 7.1G 1.1G 6.1G 15% /

Still not sure where the boot files were, and why df reports a larger size for the device!
Update are downloaded to /var/db/system/update/

 

[Yorick]

Ah that’s a good one to check as well. You want to make sure your system volume is not on your boot drive. You want that on one of your data pools. That’s in system->general I think? I’m not at my FreeNAS box right now.

 

[coolnodje]

The TrueNas install procedure states that installing on Flash media is preferred.
Doesn'that implies the system volume IS on the boot drive?

 

[Yorick]

"Flash media" here means "SSD", not "USB Flash". Installing on USB Flash has been discouraged for years, these little flash sticks have a tendency to die rather quickly, particularly lately with USB 3.

The type of boot media says nothing about the System Dataset. The System Dataset (System->System Dataset) will be on your first data pool by default. It can be moved to the boot pool, but that is discouraged. Particularly if your boot pool is USB Flash, not SSD Flash, it is extremely discouraged to move your System Dataset there.

Keeping it simple:
- Boot from an SSD of some description. ix prefer it connected via SATA or NVMe. People have had good success with USB-to-SATA-M.2 "stick" enclosures and M.2 SATA 2242 SSDs, though the specific model that's been widely chosen by forum members likely contributes to that success. ix response to that USB enclosure I think was "we don't recommend it, you do you"
- Keep the System Dataset on a data pool

 

[coolnodje]

Thanks, that wasn't immediately clear to me.

I have been operating with a boot pool on USB Flash for years, and it has apparently died.
I'll replace with another USB or Spinning disk, because that's what I have immediately.

My System Dataset has always been on a data pool by default.

Would using an SSD for the boot pool makes operation much faster?

Apart from boot time, I imagine the system (and the web GUI, conspicuously slow in my case) wouldn't benefit from the SSD as they must be running from the System Dataset?

 

[Yorick]

Would using an SSD for the boot pool makes operation much faster?

No. Boot itself will be marginally faster, but how often do you need to boot. It's just about reliability.

The "M.2 SSD inna stick" idea is here: https://www.ixsystems.com/community...my-usb-mirrored-boot-drive.86174/#post-596973

 

[coolnodje]

nice, glad to see there a re small capacity SSDs available under these form factors!

Don't know how large the System Dataset is, but seems like a good idea to put it on SSD to speed things up a bit.

I like the dual mirrored USB sticks idea from the post though. After all my initial USB dongle got me running for 5+ years....
I suppose you make them available through BIOS as failover option.
But does the boot device gets updated with System updates ?

 

[coolnodje]

Or actually use an SSD as boot device and just put the System Dataset on it.
Would that be bad for any reason?

 

[Yorick]

the system dataset doesn’t need ssd speeds. Among other things it holds your daily default config backup. So if your boot drive dies, you can always get back to where you were with a fresh install and that config backup.

Having the system dataset live on a redundant data pool just makes a lot of sense.

 

[Apollo]

the system dataset doesn’t need ssd speeds. Among other things it holds your daily default config backup. So if your boot drive dies, you can always get back to where you were with a fresh install and that config backup.

Having the system dataset live on a redundant data pool just makes a lot of sense.

Problem is, redundancy doesn't exist, or isn't recognized at boot as it is handled by ZFS. So having mirror boot drive doesn't make more sense for booting purposes as UEFI or BOOT doesn't know what is behind the mirror.
Trues hardware RAID wouldn't have such issue.
However, removing the faulty drive or setting boot to look at a specific drive would take care of that.
Otherwise, you won't boot or you will boot but end up nowhere dealing with a crashed kernel.

 

[Yorick]

Hence system dataset on the redundant data pool. I think we are saying the same thing :).

 

[coolnodje]

I've restored my system on a spare USB dongle, waiting for an ordered SSD.
I confirm that TrueNAS-12.0-RC1 can't fit on 8Gb anymore, which feels weird as BETA2 was fitting.

Now the SSD was delivered earlier, and if I could just repeat the install the restore procedure, I was wondering if there was an easy way to backup/restore the boot drive?

This is also because I was surprised to see that the system backup actually doesn't backup anything from the boot drive.
So all my scripts managing fan speed and files stored in root home, and actually other users home have just vanished.

If the boot pool isn't supposed to be modified in the TrueNAS organization, then what would be a good place to store anything coming from the user?
From my understanding the System Dataset is not meant to receive this kind of user data either.

 

[coolnodje]

oups, I just discovered the Boot pool men item.
I understand better now the mirroring option :)
I guess this answers my above questions.

So basically I can just mirror my 64Gb USB key to my new 120Gb SSD.
And hopefully get rid of the original boot pool afterwards and keep only the SSD.

I will lose space but I guess I don't really care about 64Gb at this stage.
And I could probably enlarge the new SSD to its full capacity somehow. Could I?

Then if I want to have a system managed backup of my boot pool, mirroring is THE available option.
Anything aside of this, i need to fend for myself.

 

[joeschmuck]

Not sure I'd mirror the boot pool just to install it to a SSD. I'd backup my configuration files (instruction are in the user manual) and then install FreeNAS/TrueNAS and then restore the configuration files. It's very simple and quick, you will have a clean installation in the end.

Just my two cents.