Modifying pool with System Dataset

[DarkDust]

Hi folks,

tl;dr: Want to do modification to pool with System Dataset that aren't support by the GUI. Want to so do without breaking my system a second time.

I've got a pool that had two 1TB disks, mirrored. I wanted it to grow using two 3TB disks. Unfortunately, "attach" is not supported by FreeNAS GUI and I read I had to export it, import in shell, modify it, export again and import in GUI again.

I used the GUI to "Detach" that pool… and the system was broken. GUI stopped. I still had an SSH session open but none of the commands worked any more, not even "ls". Couldn't re-import the pool since "zpool" wasn't found either. Reboot showed nasty error about ld-elf.so.1 not being found and the system didn't come back up again. My guess is that the System Dataset being on that pool is what caused this bug (really, that shouldn't have left me with a completely broken system). Long story short, I reinstalled the system.

While I was at it, I did my modifications to the pool via the shell provided by the installer. To replace the disks in the pool, I did it like this:

Code:

zpool attach mypool gptid/b8b979e8-259f-11e9-bca1-002590e891fc /dev/da0
zpool attach mypool gptid/7c2763b1-5778-0e4b-9141-814812e238fa /dev/da6

After resilvering finished, I detached the two 1TB devices and now the pool is 3TB. Great. Finished re-installation and set the system up again.

Now that my system is up and running again, I notice that these devices seem to lack a GPT. "glabel status" doesn't list them and "gpart list /dev/da0" doesn't work either. Here's what "zpool status hel2" is showing me:

Code:

[root@hub ~]# zpool status hel2
  pool: hel2
 state: ONLINE
  scan: resilvered 1.51T in 0 days 02:30:23 with 0 errors on Sat Feb  2 12:33:59 2019
config:

        NAME        STATE     READ WRITE CKSUM
        hel2        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da6     ONLINE       0     0     0

errors: No known data errors

Since I've got a spare 3TB disk I figured I could do a little dance to get GPT back: attach third disk to the pool, this time with GPT, detach one disk, add GPT to that disk and reattach, detach the second "raw" disk. Et voila, two disks with GPT.

BUT. I don't like breaking my system again. So how can I modify this pool, that has the System Dataset, without breaking FreeNAS again?

If there's no good way of doing that, are there any downsides to having the disks attached "raw" without GPT?

 

[Chris Moore]

tl;dr: You should have come and come and asked questions first, because what you read was wrong and it caused you all kinds of problems that you didn't need to go through and the problems are not done yet.

The answer you should have found, to grow your pool by replacing drives, you replace one drive at a time until all the drives in a given vdev are replaced with larger drives, then the pool will autoexpand.
Here is where that is in the manual, which you should totally read:
https://www.ixsystems.com/documentation/freenas/11.2/storage.html#replacing-disks-to-grow-a-pool

I've got a pool that had two 1TB disks, mirrored. I wanted it to grow using two 3TB disks. Unfortunately, "attach" is not supported by FreeNAS GUI and I read I had to export it, import in shell, modify it, export again and import in GUI again.

Attach is the wrong thing to look for. You should have asked first.

While I was at it, I did my modifications to the pool via the shell provided by the installer. To replace the disks in the pool, I did it like this:

Code:
zpool attach mypool gptid/b8b979e8-259f-11e9-bca1-002590e891fc /dev/da0
zpool attach mypool gptid/7c2763b1-5778-0e4b-9141-814812e238fa /dev/da6

This was completely unnecessary and wrong.

Now that my system is up and running again, I notice that these devices seem to lack a GPT. "glabel status" doesn't list them and "gpart list /dev/da0" doesn't work either. Here's what "zpool status hel2" is showing me:

What you did can be done manually, even in FreeNAS, but the way you went about it is completely wrong and you ended up with no swap partition because that is something the middleware handles when you do it from the GUI, the way you should have done.

Since I've got a spare 3TB disk I figured I could do a little dance to get GPT back: attach third disk to the pool, this time with GPT, detach one disk, add GPT to that disk and reattach, detach the second "raw" disk. Et voila, two disks with GPT.

It might work, but it might not because what you have done is dedicate the entire disk to ZFS where FreeNAS normally creates a partition for swap and another partition for ZFS. If you replace these disks FROM THE GUI, it might be able to fix what you have broken.

If there's no good way of doing that, are there any downsides to having the disks attached "raw" without GPT?

The downside, you probably have no system swap space.

 

[Chris Moore]

PS. You should also read all of this:

Updated Forum Rules 12/5/18
https://forums.freenas.org/index.php?threads/updated-forum-rules-12-5-18.45124/

and Did you read the manual?
http://doc.freenas.org/11/freenas.html

 

[DarkDust]

Hi Chris,

first of all thanks for answer and yes, I did read a lot of the manual in the past few days but I missed the "how to grow a pool" section. Since I intend to turn a different 2-mirror pool into a permanent 3-mirror (a different topic I'll now be sure to research more in-depth and ask about here) I did search for that and the answers were "can't do that with UI". I only grew pools with attach/detach in the past so it didn't occur to me "replace" would be an option here.

So to get to disks with GPT, you recommend that I replace one of these disks with the spare one in the UI and then do the same with the remaining disk and the then-spare?

According to swapinfo and gmirror, the swap partitions are on different disks right now. Thanks for making me aware of this, seems to be fine.