Replace/Upgrade Boot & System Dataset Drive

[Electr0]

Hi All,

I currently have two 250GB SSDs (ada0, ada1) installed in my server. Each with three partitions:

• ada0p1 (260M) - EFI
• ada0p2 (29G) - Boot Pool
• ada0p3 (203G) - System Dataset Pool
  
  
• ada1p1 (260M) - EFI
• ada1p2 (29G) - Boot Pool
• ada1p3 (203G) - System Dataset Pool

Both the Boot Pool and System Dataset Pool exist on the same SSD which is Mirrored accross the the two SSDs.
I want to replace both drives with 500GB SSDs.

What is the best way to go about this?

Is there a way in which I can Resilver the entire drive (all partitions) at once? Or will I have to Resilver each Boot and System Dataset partition individually?
If the latter, how do I do that? Do I need to create each (empty) partition individually first, then resilver the partitions?

Furthermore, what do I do about the EFI (adaXp1) partition? Do I need to also copy/mirror that somehow?

Thanks in advance.

 

[winnielinnie]

I want to replace both drives with 500GB SSDs.

Why do you want to do this? You won't be able to make use of the extra space on the boot drives / boot-pool.

It cannot be used for shares, user datasets, or general storage.

It houses the boot loader, operating system, logs, System Dataset (optional), and a 16 GiB swap (optional). Nothing else.

 

[Electr0]

Why do you want to do this?

Because I want to increase (double) the partition size and therefore available capacity for System Dataset Pool.

I was under the impression that for "normal" mirrored pools, with one partition, you can expand their size by replacing one drive at a time and resilvering. Once you have resilvered the second dive (in a two drive mirror) the partition will expand to make use of the new additional space.

That's what I was hoping to achieve, specifically for the System Dataset pool.

---

Putting a little more though into it, could I just...:

1. Remove one of the 250GB drives
2. Replace it with a 500GB drive
3. dd the other 250GB drive to the new 500GB drive
4. Use gpart to expand the System Dataset partition
5. zfs attach the 500GB Boot and Dataset partions (? and resilver)
6. Repeat for the second drive

 

[Patrick M. Hausen]

Keep in mind that partitioning your boot drives is an unsupported configuration. That being said you can achieve your goal like this:

Code:

# 1st drive
zpool scrub boot-pool
zpool detach boot-pool ada1p2
zpool scrub <system dataset pool>
zpool detach <system dataset pool> <id of partition on ada1>
# power down
# replace ada1 with larger drive
# boot up
gpart backup ada0 | gpart restore ada1
gpart resize -i3 ada1
dd if=/dev/ada0p1 of=/dev/ada1p1
zpool attach boot-pool ada0p2 ada1p2
zpool attach <system dataset pool> <id of partition on ada0> <id of partition on ada1>
# let the resilver finish

# 2nd drive
zpool detach boot-pool ada0p2
zpool detach <system dataset pool> <id of partition on ada0>
# power down
# replace ada0 with larger drive
# boot up
gpart backup ada1 | gpart restore ada0
gpart resize -i3 ada0
dd if=/dev/ada1p1 of=/dev/ada0p1
zpool attach boot-pool ada1p2 ada0p2
zpool attach <system dataset pool> <id of partition on ada1> <id of partition on ada0>
# let the resilver finish

Make sure you are using the gptid devices and not e.g. /dev/ada0p3 for your system dataset pool. The boot-pool is the only pool in CORE using absolute device names.

HTH,
Patrick

 

[Electr0]

Keep in mind that partitioning your boot drives is an unsupported configuration. That being said you can achieve your goal like this:

Yup, I know this is an unsupported configuration, that's why I thought it best to ask before I go tinkering.

Thanks so much Patrick, that's exactly what I was looking for. In the same time as you posted I edited my second post and it seems like I was on the right track. I'll follow your suggestions.

 

[winnielinnie]

Because I want to increase (double) the partition size and therefore available capacity for System Dataset Pool.

I think you're confusing some of the terms.

The System Dataset is a small, special (iXsystems) dataset that needs only minimal space.

Your storage pool(s) are where you actual store and share user data.

The System Dataset can live inside one of your storage pools or on your boot-pool.

Your boot-pool cannot (and should not) be used to store and share user data.



The GUI is a bit misleading in that it labels your main storage pool as "System Dataset Pool".

The reason for this "label" is simply to remind you "Hey, this is where your (small) System Dataset lives."


EDIT: Did not realize this was a custom (unsupported) configuration from the start. (See posts further below.)

To drive this point home, this is how much space is being used up by my System Dataset (i.e, hidden ".system" dataset):

Code:

NAME                  AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
meta-pool/.system       260G   539M        0B   7.24M             0B       532M

My entire ".system" dataset only uses up 539MB.

 

[Electr0]

I think you're confusing some of the terms.

Ohh, perhaps I am...

These are my current "Data" pools:

And this is my current Boot-Pool:

Both the boot-pool and the (System Dataset Pool) are on the same mirrored SSDs (an officially "unsupported" configuration).

It's the space for the (System Dataset Pool) that I want to expand.

 

[Patrick M. Hausen]

Referring to it as the "system data set pool" is what confused @winnielinnie, apparently. It's just a regular data pool on the major part of your boot SSDs.

 

[Electr0]

Referring to it as the "system data set pool" is what confused @winnielinnie, apparently.

Ahh okay, got it.

It's just a regular data pool on the major part of your boot SSDs

Yes exactly, that's how I've set it up.

Thanks again for your help!

 

[winnielinnie]

All good here, and some hasty assumption on my part. I didn't realize that you did a custom (unsupported) configuration when you installed TrueNAS.

Since by default, the TrueNAS installer will not "divvy up" the boot-pool nor allow you to do so in the GUI later.

I guess this goes back to "Why can't we use our existing SSDs to break off a useable SSD data pool?"

 

[Electr0]

No worries mate.

Yeah I'm kind of raw-dogging it at the moment.
And from memory it took a bit of effort to get it this way, being unsupported and all.
But I though it was better than running it off two mirrored USBs

 

[winnielinnie]

Yeah I'm kind of raw-dogging it at the moment.
And from memory it took a bit of effort to get it this way, being unsupported and all.

No worries. For future reference, it helps to make a disclaimer that this was initially created via a custom/unsupported method.


Because when I saw this...

ada0p3 (203G) - System Dataset Pool

...I thought you made a mistake in your post and that you were implying "The boot-pool is also where I located my System Dataset."

I didn't think you actually created a separate partition. (Thought you wrote "ada0p3" as a rough-hand way to try to explain it.) But now I see you literally created an actual extra partition.

 

[Electr0]

No worries. For future reference, it helps to make a disclaimer that this was initially created via a custom/unsupported method.

Will do, thanks. I actually forgot it was unsupported until Patrick called it out.

But now I see you literally created an actual extra partition.

Haha yes, I am a mad man. ‍

 

[Patrick M. Hausen]

Repeating myself, just make sure to refer to the partitions in your "ssd" pool via their GPTIDs, e.g. gptid/8d299abe-e22e-11ea-9ee7-ac1f6b76641c. Only use /dev/ada0p2 and /dev/ada1p2 for the boot pool.

You can find the IDs by e.g. gpart list ada0 and looking for the rawuuid field.

That will keep the middleware and the UI happy, e.g. exporting/importing via UI etc.

 

[Electr0]

Yup, got it!

Thanks :)

 

[Electr0]

Worked like a charm! Thanks @Patrick M. Hausen

Just has to zpool set autoexpand=on <pool name> and then zpool online -e gptid/xxxx each drive.