Create zfs mirror by adding a drive?

TheShoeGuy

Cadet
Joined
Sep 10, 2013
Messages
2
Hi there,
I run my own business and have been using FreeNAS on an HP Proliant Microserver for over a year but it just sits there. The factory HDD suddenly died and I had no solution. I use FreeNAS for Windows CIFS fileshares and to date there's no easy way to get Acronis/Crashplan or any other regular backup solutions to work. I'm fairly technical but FreeNAS is my first Linux based adventure. Thus, all the forums are very confusing.

So.
I have an exiting zfs drive.
I simply want to mirror it on a second drive. (I know a mirror is not a backup)

How to I set up the second drive as a mirror?
Do I have to copy all the data to a third drive and then setup the zfs mirror on the two drives, then copy the data back?
How about auto import on the second drive?
help please, you wonderful wonderful people..
 
Last edited by a moderator:

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Let's assume ada0 is your existing disk, ada1 is the new one, tank is the pool name.
  1. gpart create -s gpt /dev/ada1
  2. gpart add -i 1 -b 128 -t freebsd-swap -s 2g /dev/ada1
  3. gpart add -i 2 -t freebsd-zfs /dev/ada1
  4. Run zpool status and note the gptid of the existing disk
  5. Run glabel status and find the gptid of the newly created partition. It is the gptid associated with ada1p2.
  6. zpool attach tank /dev/gptid/[gptid_of_the_existing_disk] /dev/gptid/[gptid_of_the_new_partition]
Test this first in a VM to verify that you understand everything.

Edit: Updated the steps to create the default 2GB swap partition.
 

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
This should work (but it will not create the default 2GB swap partition):
What's the benefit to this. You might as well as simply use the whole disk. The commands you were looking for:
Code:
gpart create -s gpt ada1

gpart add -b 128 -i 1 -t freebsd-swap -s 2G ada1

gpart add -i 2 -t freebsd-zfs ada1
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
What's the benefit to this. You might as well as simply use the whole disk.
Two reasons:
  1. I wanted to use GPTIDs as they are safer when disks move around.
  2. I wanted to keep the instructions simple. I saw this in notifier.py and I did not want to bother TheShoeGuy with the arithmetic:
Code:
# Calculate swap size.
swapgb = swapsize
swapsize = swapsize * 1024 * 1024 * 2
# Round up to nearest whole integral multiple of 128 and subtract by 34
# so next partition starts at mutiple of 128.
swapsize = ((swapsize+127)/128)*128
<snip>
"gpart add -b 128 -t freebsd-swap -s %d %s" % (swapsize, devname)


However, as I read it again now (it was late night yesterday) the code is NOP. If you multiply something by 1024, it will always be also multiple of 128 and the strange -34 happens only in a comment :).

So, yes paleoN is correct, you can insert this as step 1.5:
gpart add -b 128 -t freebsd-swap -s 2G /dev/ada1
Later in gpart list look for ada1p2 (the ada1 partition that has type: freebsd-zfs)
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Cool beans. I've always wondered where you get the command for the swapspace creation. Now I have it. :)

And I'm betting the code to create the gpt partition for ZFS is in there too!
 

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
I wanted to use GPTIDs as they are safer when disks move around.
Well, ZFS manages that just fine. I suppose it is helpful to the GUI to use GPTIDs. I wired down all of my disk devices anyway.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Yes, ZFS is fine, but the GUI can get confused without GPTIDs.
 

James Snell

Explorer
Joined
Jul 25, 2013
Messages
50
Thanks, this helped me. I followed the steps given and now I can see my newly added drive is presently reslivering. Perfect. For completeness, I'm quoting the steps, with 1.5 inserted (as #2 below), as I could have ended up missing 1.5, and others may end up missing it.. Thanks a lot Dusan!

----------------------------------------------------------------------
Dusan & James' rad guide to
ZFS: Adding a drive to create a mirror
of another in an existing ZFS pool
(circa FreeNAS 9.1.1)

Here we're adding a drive labelled /dev/ada1 - Your exact drive name obviously matters vastly. Make sure you know the correct name for the drive you mean to add to your pool!

  1. gpart create -s gpt /dev/ada1
  2. gpart add -b 128 -t freebsd-swap -s 2G /dev/ada1
  3. gpart add -t freebsd-zfs /dev/ada1
  4. Run zpool status and note the gptid of the existing disk
  5. Run gpart list and find the gptid of the newly created partition. It is the rawuuid field. In this example it would be rawuuid of ada1p1
  6. zpool attach tank /dev/gptid/[gptid_of_the_existing_disk] /dev/gptid/[gptid_of_the_new_partition]


Upon completion of these steps, running zpool status should return a screen similar to mine, note the "resilver" comments in the output:
[root@nas2] ~# zpool status
pool: store0
state: ONLINE
status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Mon Nov 4 18:17:24 2013
66.7G scanned out of 1.48T at 77.3M/s, 5h19m to go​
66.7G resilvered, 4.41% done​
config:

NAME STATE READ WRITE CKSUM​
store0 ONLINE 0 0 0​
mirror-0 ONLINE 0 0 0​
gptid/99a9f243-1953-11e3-ae07-0013208ac865 ONLINE 0 0 0​
gptid/b7d84b57-45b7-11e3-9774-74d02b92b560 ONLINE 0 0 1.12K (resilvering)​
logs​
gptid/0025ee3a-1b65-11e3-8c5f-000fb546f3b2 ONLINE 0 0 0​

errors: No known data errors


Note in this case, I have a drive for a zfs slog - I expect most won't have the logs section I do.. ^_^
 

CRServers

Cadet
Joined
Feb 12, 2014
Messages
3
Hello to all,

We have a FreeNAS (FreeNAS-8.3.1) server running and it has one identical unused disk that was installed from the beginning.
We would like to add it as a mirror to the main disk as it is explained here.

The main disk is very busy at this time serving shared storage to our Xenserver pool so I would like not to stop service to the Xenserver pool.

So wanting to be careful before diving into the task, I have a couple of questions:
1- Can this procedure be performed without service interruption?
2- Do I have to reboot the box afterwards?
3- Assuming all commands are executed correctly and procedure above is followed exactly. What could be the risk of performing the task?

Thanks very much for your feedback, and please excuse me if some of my questions are a little dumb ;)

Regards,

Rodrigo
 

CRServers

Cadet
Joined
Feb 12, 2014
Messages
3
this procedure was performed and ended successfully with no issues.
No need to reboot
No service interruptions
Thanks to all for the good work.
Regards,
 

Pontiac75

Dabbler
Joined
Aug 9, 2014
Messages
18
Thank you so much for this guide. I'm a newbie to FreeNAS and had a ZFS volume on a 320GB WD Scorpio Blue, with a spare 320 Scorpio Blue formatted to NTFS sitting waiting to be mirrored.

I copied my media files over the network to the ZFS single volume array and then plugged in the NTFS drive and quick wiped it before following this guide.

zpool status in shell shows that the drives are silvering and at about 50% complete! Thank you.

A couple of questions, though:

When silvering is finished will both drives show up in the GUI as a mirror? Will I be able to control them both via GUI, or will the newest addition be hidden and autonomously reflect whatever I add/delete on my main zpool?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
They should show up as mirrors right now. Unfortunately the commands above are not accurate for the latest versions of FreeNAS. But too late now...
 

Pontiac75

Dabbler
Joined
Aug 9, 2014
Messages
18
You may be right. All my files are there on the first drive (which I called array1) but the second drive does show up in "View Disks".

The NAS has been left on while I slept and the "zpool status" command now shows this:

[root@freenas ~]# zpool status
pool: array1
state: ONLINE
scan: resilvered 196G in 1h2m with 0 errors on Sat Aug 9 23:36:07 2014
config:

NAME STATE READ WRITE CKS
UM
array1 ONLINE 0 0
0
mirror-0 ONLINE 0 0
0
gptid/282513a4-1cd1-11e4-8e08-74d435e0f216 ONLINE 0 0
0
gptid/518ca9fb-200c-11e4-80d7-74d435e0f216 ONLINE 0 0
0

errors: No known data errors
[root@freenas ~]#

So, is my configuration OK in case one of the drives fails?

All I want is a mirrored pair and when either drive starts to fail I'll replace it and resilver. Are changes to array1 currently being mirrored on the second drive?

Thanks.

PS: I've just watched 5 mins of HD 1080 movie streamed from the NAS, then closed my media player. The Reporting>Disk I/O section in the GUI shows roughly equal demand on the graphs for ada0 and ada1 drives. So the mirror must be working at improving my read speed, I think.
 
Last edited:

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
I'm fairly technical but FreeNas is my first Linux based adventure. Thus, all the forums are very confusing.

So first problem I notice is right there. FreeNAS is not Linux based. FreeNAS is BSD based. If you're looking around the internet for tips keep that in mind. There are plenty of similarities, but there are also some very key differences, so you should make sure to make yourself aware of the differences between Linux and BSD.

Trust me, I've fubarred stuff before due to a reverse misunderstanding (thinking DD-WRT was BSD based for some reason when that is Linux based)

So, is my configuration OK in case one of the drives fails?

Yes, that does look to be fine.
 

Pontiac75

Dabbler
Joined
Aug 9, 2014
Messages
18
So if my zpool staus is OK, then am I safe if one of the drives fail?

I initially set up FreeNAS with one drive full of data and have added the second drive as a mirror rather than the accepted way of starting off with two drives in a mirror array.

If either drive fails I won't lose my data, will I? The plan in the event of failure is obviously to detach the failing drive and add a new one of equal or greater capacity and then resilver.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
So if my zpool staus is OK, then am I safe if one of the drives fail?

I initially set up FreeNAS with one drive full of data and have added the second drive as a mirror rather than the accepted way of starting off with two drives in a mirror array.

If either drive fails I won't lose my data, will I? The plan in the event of failure is obviously to detach the failing drive and add a new one of equal or greater capacity and then resilver.

You can always cold test it by booting with only one of the drives detached (in this case the original drive since we already know it at least contains the data, you really want to test if the new drive is also containing the data) and see if the pools shows as degraded but still accessible.
 

Pontiac75

Dabbler
Joined
Aug 9, 2014
Messages
18
Duh! Sometimes the obvious eludes me!

Should I perform a software detach, or simply power off and unplug the drive from its SATA port without telling FreeNAS first? If I just pulled the drive whilst powered off would this simulate a power failure on the drive controller?
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
Duh! Sometimes the obvious eludes me!

Should I perform a software detach, or simply power off and unplug the drive from its SATA port without telling FreeNAS first? If I just pulled the drive whilst powered off would this simulate a power failure on the drive controller?

Just power off and unplug. That should be simulation enough for the purpose of your test. You just want to test to see if the data is on the new disk properly, so your only concern is taking the other drive out of the equation.
 

Pontiac75

Dabbler
Joined
Aug 9, 2014
Messages
18
I powered off and unplugged the original drive, just leaving the mirror and on power on I got an email and a warning message in the GUI saying that the pool was degraded.

All of my files were still there on the mirror. :)

I then powered off, re-attached the drive and powered on again. It has resilvered but now I have a warning message in the GUI telling me I should do a "zpool clear" command and it appears that the resilvering has introduced an error checksum:

Click Image to Enlarge Free Image Hosting

I'm guessing that this checksum error is due to my testing. Is this the case?

Is it safe to do "zpool clear" to get rid of the warning message or will that erase my data?
 
Top