Suggestions - Changing HDDs in Existing ZFS Home Server

Status
Not open for further replies.

feleven

Dabbler
Joined
Feb 17, 2014
Messages
39
I've been running a FreeNAS-ZFS 24/7 home server for a couple years, adding/upgrading drives as required, which today consists of a pair of volumes: primary storage FN-RZ2 using RAIDZ2, and backup storage FN-BK as two mirrored drives. Pictures of my current HDD setup are attached.

All drives are WD-RED models. Backup is done nightly via rsync task in cron. This is primarily a Plex server, serving movies, music and photos to my home network. All of the media is separately backed up on an unpowered standalone USB HDD attached to my Mac desktop elsewhere on the network, so losing server data is undesirable mostly due to the nuisance of reconfiguring the Plex app and the time taken to reload the data files.

I'd make use of Snapshots instead of, or as well as, rsync, but I can't quite grasp how they work (separate question, I realize).

I'm at the point where I want to replace the primary 6 x 2TB HDDs with 4 x 4TB HDDs. While I'll get about the same useable space with the new setup, I'll benefit from approx 5 times the MTTDL with the fewer number of larger HDDs, according to a currently popular online calculator. Fewer drives also reduces heat dissipation in my drive chassis to an extent.

Problem 1: two of the 4TB HDDs mentioned above are currently being used in my mirrored backup, so I have some dancing to do to free them up for use as primary storage.

Problem 2: not entirely sure about my options (if any) to move the primary storage from the six 2TB to the four 4TB HDDs.

I have a pair of 3TB HDDs currently unused, so my tentative plan was to:

1) replace the two 4TB FN-BK HDDs with the two unused 3TB HDDs. My current backup uses just under 1.9TB, so this should work (for now). The long term plan is to replace them with 6TB HDDs.

Q: can I just "replace" each 4TB HDD with a 3TB HDD, as if replacing failed drives? The process will take a while, I know, but at least the system doesn't need to be shut down. If this isn't the most efficient/effective method, could someone give me a nudge in the right direction? I have a drive-cloner dock, so its also possible to simply clone the two 4TB to the two 3TB (one at a time) offline, although this plan makes me nervous (UUID issues? dunno).

2) with the two 4TB HDDs freed from backup service, I now have four x 4TB HDDs for primary storage. If I understand ZFS rules correctly, I can't decrease the HDD count in zpool FN-RZ2. Do I have this correct? I understand a zpool is made up of a number of vdev's, and that vdev's are made up of block devices, and in turn block devices can consist of HDDs in various configurations (mirror, RAIDZx).

Q: I think what I have to do is export the data, offline and remove all six 2TB drives, install the four 4TB drives, set them up as RAIDZ2, and then restore them from the backup drives ... is this correct? Is there a better/more efficient/more rational way to go about this?

I think I can dope this process out on my own by moving slowly and re-reading the manual as I go. But any workflow ideas offered would sure be appreciated. I'd rather not bugger things up, or take the scenic route to completion. So - any suggestions are welcome.

FN-Volumes.jpg


FN-ViewDisks.jpg
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457

feleven

Dabbler
Joined
Feb 17, 2014
Messages
39
So, I can't go smaller, even on a simple mirrored pair? I guess that makes sense - each drive is supposed to be a clone of the other, which would include parity under ZFS. I was sort of hoping that the parity would pertain only to written data blocks, not to the unassigned blocks. Nuts, that puts a crimp in things. My thinking kind of depended on freeing up those two 4TB backup drives as a first step.

The following seems it might be a reasonable alternative approach (assuming the rsync doesn't worry about destination volume size as long as its big enough):

1) install new 3TB drives in chassis (two drive slots are empty)
2) create new backup volume FN-BK1 using the two 3TB drives
3) rsync existing FN-RZ2 volume to new FN-BK1 volume, scrub FN-BK1
3a) what's the best way to test a backup - actually DO a full restore then spot check the restored data for validity (play a movie, look at photos, etc)?
4) once FN-BK1 is known good, remove the original FN-BK backup volume
3) rename FN-BK1 to FN-BK (to keep things simple)
4) detach and remove the original FN-RZ2 volume/drives, put aside in case of SNAFU
5) physically install the four 4TB HDDs into the newly vacated slots
6) create a new FN-RZ2 primary volume with the 4TB drives
7) rsync from FN-BK to the new FN-RZ2 primary by reversing source and destination in the rsync task)

Does this sound do-able? Or is there a better way to do get the same result?

[ Edit: Can I shell / cp the original FN-BK 4TB drives to the new FN-BK1 3TB drives? or is that even slower than rsync for creating the smaller backup volume? ]
 
Last edited:

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Or is there a better way to do get the same result?
Usually, the recommended approach is to replicate a snapshot from the old pool to the new. The main benefit of this is that it preserves the full filesystem structure, including child datasets and snapshots.

In your case, the benefit is less obvious because your filesystem structure is so simple. The only child datasets are associated with jails, and you say you aren't using snapshots.
Can I shell / cp the original FN-BK 4TB drives to the new FN-BK1 3TB drives?
Yes, but rsync can resume an interrupted transfer, so that might save you some time in the long run.
 

feleven

Dabbler
Joined
Feb 17, 2014
Messages
39
Usually, the recommended approach is to replicate a snapshot from the old pool to the new. The main benefit of this is that it preserves the full filesystem structure, including child datasets and snapshots.

In your case, the benefit is less obvious because your filesystem structure is so simple. The only child datasets are associated with jails, and you say you aren't using snapshots.

I might use Snapshots if I understood how they work. Explanations I've found by Googling and RTFM always seem to assume I know what they are and how they work. But having had some experience with halted cp processes before, I definitely prefer the rsync ability to resume a transfer - its saved my butt a couple times.

Appreciate you taking the time to reply, every little bit helps :)

[ Edit: Found a good source of background info on Snapshots at a FreeBSD site: ]
<https://www.freebsd.org/doc/handbook/zfs-zfs.html>
 
Last edited:

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
At the most basic level, a snapshot is just a way of preventing storage blocks from being reused when data is modified or deleted. It's something that copy-on-write filesystems like ZFS can do very cheaply, so they do it.
 

feleven

Dabbler
Joined
Feb 17, 2014
Messages
39
Ok ... finally found time this afternoon to replace the backup drives, freeing two 4TB drives to join two new 4TB to form my new primary 4x4TB RZ2 zpool. In summary, what I want to do is:

6 x 2TB RZ2 pool --> 4 x 4TB RZ2 pool

I realize I can just offline and replace 2TB drives with 4TB drives one at a time (or even two at a time if I want to live dangerously - I don't). But it dawns on me that eventually I'll have a pool with 4 x 4TB + 2 x 2TB ... how do I "get rid of" the last two 2TB drives? Can I somehow split them off, leaving the 4x4TB in RZ2 configuration?

Or do I have to create a new 4 x 4TB RZ2 pool, then sync the existing 6x2TB pool to the new pool?

Or is it just safer/faster/smarter to export / offline the whole existing 6 x 2TB RZ2 pool, then rsync the new 4 x 4TB RZ2 pool from the backup pool (essentially doing a recover operation from my backups, as if I'd lost the original 6x2TB pool)?

Rod
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
Can I somehow split them off, leaving the 4x4TB in RZ2 configuration?
No, you can't. If you want to end up with a four-disk pool, and for that pool to be healthy, you'll need to create a new pool with the four disks.

If you have room to connect 10 disks at once (6-disk existing pool + 4 disk new pool), do that. Create new pool. Transfer data from old pool to new pool however you want (I'd use ZFS replication). Offline old pool. (optionally, rename new pool to old pool's name).
 

feleven

Dabbler
Joined
Feb 17, 2014
Messages
39
Thx, danb35 ... that was the conclusion I reached after reading the Solaris website info on ZFS management (great place for a detailed ZFS education), but I was really hoping I missed something.

I'll look into replication as an alternative to rsync-ing - never hurts to learn about options :) . I have room to temporarily install all 10 drives, but they'll be cheek-to-jowl ... I'll have to watch temperatures carefully and make sure my fans are doing their job.
 
Status
Not open for further replies.
Top