Backing up large pool to esata enclosure or other ways?

Status
Not open for further replies.

Tofu_Guy

Cadet
Joined
Jul 10, 2015
Messages
4
I'm planning on building a FreeNAS server for my home that will have one pool of mirror vdevs. Initially, the pool will consist of 4 drives of 6TB each (2 mirror vdevs), so 12TB usable data. It will grow in the future. I'll probably add another mirror vdev to the pool when I run out of space.

My question is: How can I back up the entire 12TB to external hard drives?

Here's what I'm thinking would be best. Correct me if any of this doesn't seem right, because I am a newbie at this. This is going to be my first FreeNAS build...

What I'm thinking of doing is buying two external esata hard drive enclosures (JBOD). Each enclosure would be used to connect directly with and backup the FreeNAS machine. Each enclosure would contain a ZFS filesystem, so that I can use scrub and ZFS replication. I would connect them, perform the backup, and then disconnect them and take them off-site.

Each esata enclosure would contain exactly the same number of terabytes as the main FreeNAS system. So if my FreeNAS system had 12TB of usable hard drive space, then the backup esata enclosure would contain 12TB of usable hard drive space also. That way, I don't have the issue of having to manually split up my data onto more than one hard drive individually.

So the plan would be: I plug in the first enclosure, backup everything to it, disconnect it, and then take that enclosure off-site (to my work office). Then a week later, I run another backup using the second esata enclosure that's at home still. Then I take that 2nd enclosure to my office (off-site). And when I return home, I take the first esata enclosure back home with me. This cycle repeats once a week or so. One enclosure stays at home, while the other is off-site, and I rotate them. I can maybe even leave the esata enclosure plugged into the main FreeNAS machine all the time while it's at home, receiving regular backup snapshots in real-time.

Does this sound reasonable? Do esata enclosures work with FreeNAS? Do they work well? I guess it just depends if your motherboard has an esata port on it?

I'm not sure how ZFS replication would work in this case, or if I'd need to use rsync or something instead. I think I'd prefer using ZFS snapshots.

This should make recovery easy. If I lose my pool, I could replace the drives in my main FreeNAS box and then copy from one of the esata enclosures to my FreeNAS box. In the worst case scenario, I lose a week of data, which is acceptable.

I considered backing it up using individual USB attached hard drives, but people generally say that that ZFS isn't reliable with USB attached hard drives.

Also, using single hard drives (as opposed to some kind of JBOD enclosure which contained multiple hard drives) means that I would have to manually split up my backup. In other words, if I want to back up 12TB, I can't use a single, 6TB hard drive to do that. Even if it was possible with compression, my pool could grow eventually to 24TB. I just don't see a way forward with individual external hard drives. I'd much rather use a RAID enclosure with esata.

What do you think? Is that something people do? If not, what's a better way? Are there any technical issues that I'm not seeing which will make this not work like I want it to?

Using Crashplan or some off-site backup service seems wrong for this purpose. It's too much data. Ends up costing money. And I'd rather not use them as my main backup method. Though, I might use Crashplan to backup really important parts of my data like my pictures and home videos.

Thanks for the help.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
I think you have a sound plan. esata is generally well supported. And the plan to use ZFS on them with snapshots and replication is a great way to go. Replication works at the dataset level, so if you wanted, and your dataset sizes supported it, you could replicate each dataset (or multiple smaller) to it's own single drive.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
whoa whoa whoa.

He's not talking about eSATA, he's talking about eSATA with port multiplication. i.e., one eSATA connection to several drives in a cage.

eSATA itself is a slam dunk---there's no difference really between that and a SATA connection for the purposes of the filesystem. But the port multiplier is a non-trivial complexifying factor, and I would make sure I asked a BSD or FreeNAS expert how well that is supported. Maybe @cyberjock or @jgreco can opine?
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Oops. Missed that. Yeah, port multipliers suck.
 

Tofu_Guy

Cadet
Joined
Jul 10, 2015
Messages
4
Thanks for the feedback.

One thing about this eSATA idea of mine: I would probably use no redundancy in the eSATA enclosure itself. So, two hard drives, each 6TB, giving a total of 12TB usable disk space. No mirroring, no RAID. This would be one of two backup eSATA enclosures, so if a disk goes, it's not an issue.

Given that description, does port multiplication over eSATA still have the same issues? What issues would I run into in general? And does this still apply to FreeNAS 10?

A popular strategy for handling backups seems to be to build a whole other working FreeNAS box and have automatic replication in real-time over to it.

The problem with that is that you still need to backup your stuff off-site. That extra FreeNAS box is on-site, unless you have a nice friend who wouldn't mind letting you eat some of his bandwidth, and your upload bandwidth is reasonable (mine isn't great). It also means a large extra cost for building a second system and the hassle of maintaining it. And again, you still need to backup to something you can take with you off-site. Actually, you would still need to make two things to take off-site (one stays at home while the other is off-site, and then you rotate).

So building one extra FreeNAS system doesn't solve the problem.

I guess you could make 3 total FreeNAS boxes and take two of them off-site, rotating like I mentioned with my eSATA enclosure idea. But that seems like a bad idea. FreeNAS wants to remain up all the time, not powered on and off repeatedly. It would also mean a phenomenal extra cost.

So I guess if eSATA enclosures (port multipliers) don't work, I'm stuck with maybe using eSATA on just one individual drive externally. No enclosure. Like depasseg mentioned, I could limit the dataset size to the max size of an external disk (say, 6TB), and then backup each dataset in my pool to individual external disks, one at a time. It's a little more work, and it does impose an arbitrary limit on dataset size, but it's workable.

Is that the best alternative?

Thanks again!
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Last edited:

Tofu_Guy

Cadet
Joined
Jul 10, 2015
Messages
4
Oh I think I see what you're saying, Robert Trevellyan. So, get an eSATA card to insert into the motherboard. And the eSATA card could have, say, 2 eSATA ports on it. Connect up those two eSATA ports to two eSATA drives (through a dual-port eSATA hard drive dock like you posted). And that way, FreeNAS can see both drives simultaneously, without going through a port multiplier. You'd have two eSATA cables connecting the dock to the FreeNAS box.

Do I have that right?

While I'll be starting with just 2 mirror vdevs (4 drives), in the future I may have a max of 4 mirror vdevs in this FreeNAS box (so 8 drives total) . With this configuration, I would need an eSATA card that would give me 4 eSATA ports in order to future-proof it. Either that or get 2 cards with 2 eSATA ports each?

Does anyone have any recommendations for an eSATA card that FreeNAS would work well with?

I guess I should avoid any card having to do with RAID, right? I've heard some RAID controllers say they have a JBOD mode but are actually doing RAID 0 with individual disks instead, which FreeNAS would not like.

Thanks again. You guys are great!
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Do I have that right?
Yes.

Please note, a case can be made for not being so concerned about port multipliers with a periodic backup task. The idea would be to make the backup, then immediately run a scrub to verify it. This is a very different from relying on a port multiplier to handle live data 24/7.

Here are some cards to consider: http://www.amazon.com/dp/B003GS8VA4/?tag=ozlp-20
I haven't used these myself, because my RC-219 cards work just fine, but those are discontinued.

Another method I've used successfully is an internal HBA (e.g. a HighPoint Rocket 640L [*]) connected to an eSATA bracket, but then you have to be extra conservative about cable length.

[*] doesn't support hot-swap, so you have to restart after connecting the drives.
 

Tofu_Guy

Cadet
Joined
Jul 10, 2015
Messages
4
Good info, Robert.

Yes, I would definitely want to hot-swap, rather than having to reboot all the time just to backup. Now to shop for eSATA cards.

I think this solution will end up being cheaper and maybe better than my original idea. I don't need two drive enclosures, in other words. I'd just need a hard drive dock, and then pop in and pop out the drives as needed. Take the drives off-site and rotate. Nice.

Thanks again!
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
But the port multiplier is a non-trivial complexifying factor, and I would make sure I asked a BSD or FreeNAS expert how well that is supported. Maybe @cyberjock or @jgreco can opine?
Not exactly what you're looking for, but I can help:

Intel, of all people, doesn't support port multipliers. Their SATA controllers won't generally outright refuse to work, but experiences are rather negative.
Port multipliers are also universally crap. So crappy that WD decided to roll their own controller firmware for a hybrid 1TB HDD/120GB SSD instead of using a port multiplier to present two independent devices.

Also, those external enclosures tend to be Easy-bake ovens for drives, powered by PSU worthy of a back-alley Shenzhen wandering salesman.
 
Status
Not open for further replies.
Top