Really dumb resilvering error that is not warned over anyway

[Koalitech]

Hello everybody.

I've just encountered a really dumb error happening in the process of resilvering (or rebuilding a degraded mirror) that I think it should be mentioned on the manual or anywhere else. I know it is a bit obvious, but could also be a bit hard to get when you don't think about everything on your server.

I have an old PC with:

Intel Core i5-2500 vPro
16 GB or DDR3 RAM (non ECC... YES I KNOW)
2 Seagate 2TB SATA drives
And TrueNAS 12.0-U3

One of those drives (the oldest one), just went offline one day and the next an error popped up saying it was damaged (S.M.A.R.T. error and I/O error), so I got a new one to replace it. I followed all the manual's instructions to get the old one offline, replace the hardware, give the order to "replace" in software, and every single one of the half a dozen times I tried it said "It can't be replaced, new drive is smaller.".

It was NOT smaller, same brand, same capacity, same interface. It had no sense to it. I tried, and tried, read every single page on the manual about it (not really many of them) and got nothing.

When I put the new "tried" drive on a Linux (Ubuntu 20.04) machine, it showed a already created new GPT partition table (ok), two new partitions (right), 1 for swap and 1 for ZFS storage (nice). And then it hit.

PLEASE READ:

If, for whatever reason, you INSTALL TrueNAS/FreeNAS, and LATER, change SWAP allocation settings (THE SIZE OF THE SWAP, explicitly), not only it WON'T change the swap partition already on your drives (which is perfectly understandable and the right thing to do), but it WILL OVERRIDE the setup on your mirror at the resilvering time. Trying to make a swap partition the size on the settings rather than the size on the mirror, causing it, effectively to NOT BE THE RIGHT size to resilver.

I know someone trying to administrate a server should know this and getting it into account, but a single pop up warning a mismatch settings between the actual mirror and the configuration on TrueNAS could be useful. I know this is most caused by my own stupidity than the software itself, but hoping I'm not the only one with such stupidity on the planet, I thought this information could be useful.

Sorry for the wall of text, I thought the whole story would be better to understand the "problem" than a short and fast explanation.

 

[Arwen]

One of the reasons I heard for the swap partition on the data disks, is to compensate for mismatched disk sizes. Where the replacement raw size is a few mega-bytes smaller than the pool disk to be replaced. So, you can adjust the swap size and make it work.

I've experienced this size difference with hardware RAID. In one case, I had to use a disk of almost double in size because I did not have one of suitable size.

In some ways, I wish manufacturers would just standardize on specific sizes. I do know that Sun Microsystems used to buy disks from many different manufacturers, (Seagate, IBM, Hitachi, Fujitsu), and adjust the advertised size as the smallest of them. So, all disks in that size block, were exactly the same size.

 

[Koalitech]

One of the reasons I heard for the swap partition on the data disks, is to compensate for mismatched disk sizes. Where the replacement raw size is a few mega-bytes smaller than the pool disk to be replaced. So, you can adjust the swap size and make it work.

Sure, that is what I ended up doing, changing the swap settings. The thing is, when rebuilding a degraded mirror, FreeNAS/TrueNAS should rebuild the exact same mirror structure that is degraded, not a new one with mismatched settings. That wouldn't fly.

In case of either a change in the mirror settings or a different sized drive, it SHOULD tell you with a pop up, at least. Normally you are a bit, how should I put this?, nervous, about having a mirror raid degraded, putting in danger everything on the drives, to have to think about every single detail on the process. Any bit of help would be nice.