Is FreeNAS suitable for this type of usage?

[pirateghost]

Instead of adding to that mess, I decided to make a more manageable solution from the ground up.

So you're essentially giving up that 'mess' to build a whole new server, to prevent making the mess bigger, and want to manage your new server in the same manner as the old mess...thinking in terms of the same old methods in which you used with your current configuration.

 

[Honey]

Let me ask you this first.

Why are you leaving your current solution with your hodgepodge of disks?

Going from a mix of various disks, I want to standardize. One NAS populated with similar high quality proven disks.
I hope to move a lot of the files from the desktop to the NAS, while streamlining the desktop with fewer but newer/faster drives as well. Not all at once, but as a process.
Ideally, I'd like to get rid of all the external drives for anything other than backup.

So you're essentially giving up that 'mess' to build a whole new server, to prevent making the mess bigger, and want to manage your new server in the same manner as the old mess...thinking in terms of the same old methods in which you used with your current configuration.

I think I get what you're saying. Keep in mind, you have a lot of experience with FreeNAS. I have very little. Something that is obvious to you, may not be to me.
That said, I am not at all against having 10 drives mapped from the NAS, from an organizational point of view.

I realize that in order to go the RAIDZ way, I must add drives together and manage them collectively as one whole. I resist this and it grates the eyes of those on this board, because not doing this, is an inefficient way of using ZFS and FreeNAS.
As I implied previously, I like simplicity. 9 separate drives, shared individually. One drive dies, one drive is replaced. This is simple. 6+3 drives connected interdependently, requiring careful management or everything from all drives is lost. That is not simple.
I'd love to do RAIDZ, but it worries me that it's so easy to screw up. I don't mind really solid solutions, that even a baby with a chainsaw would have a hard time screwing up - even if they're not as elegant or if they take a little longer. I know if one drive dies, with my way, that it's dead. But I can handle that (get back to me on that FreeNAS going bonkers over losing a single drive in a pool thing, please). Not sure I can handle losing everything on all drives, because I pushed a wrong button out of order.

 

[pirateghost]

I have run FreeNAS for years and never lost an entire pool. I lost plenty of data prior to using FreeNAS when I thought I could manage all the individual drives independently...

What careful management are you referring to? You configure your server one time, and let it handle itself. When you get an email about a disk about to fail, you just replace it and move along about your day. No data recovery required.

I think you're feeding too much off the idea that you are going to lose everything randomly. If you perpetually stay in this "I'm too much of a noob to do that" mentality, you will never get to a point to be comfortable with your server.

If you aren't willing to learn a little bit about the system you're building, and the OS that it runs, then you should stick to what you know.

When something goes wrong, what are you going to do about it? Learn on the fly while your data is inaccessible?

I don't see how you think managing 9 separate drives on 9 separate pools is EASIER than running a pool of vdevs. What services are you going to run on this server? How do you think recovering from a pool loss is going to be easy? This is more than just drive letters and mapped network shares.

It's your server, do what you want, but given your current desire to ignore the warnings of the collective here, I don't think FreeNAS is appropriate for this configuration. At all.

 

[depasseg]

If you perpetually stay in this "I'm too much of a noob to do that" mentality, you will never get to a point to be comfortable with your server.

I think the fact that the OP desktop has 24TB and is backed up kills the "I'm too much of a noob". You are in the big leagues now.

It's your server, do what you want, but given your current desire to ignore the warnings of the collective here, I don't think FreeNAS is appropriate for this configuration. At all.

And I think that even though you could create 9 separate pools, shares, settings, etc, that isn't the recommended approach for freenas. But honestly, I don't think anything else is either.

And I really don't understand the "I can lose one drive at a time, but I can't lose everything all at once" concept.

 

[Honey]

What careful management are you referring to? You configure your server one time, and let it handle itself.

You're the one who mentioned that managing 10 pools would be ridiculous. I thought it was set and forget.

I think you're feeding too much off the idea that you are going to lose everything randomly.

I am more worried about things popping out of the blue and kicking me over the shin. Like the scenario you mentioned, where losing a single disk in a single disk pool would trip up FreeNAS to the point of having to break out the command line and start talking straight to FreeBSD. Irk!

If you perpetually stay in this "I'm too much of a noob to do that" mentality, you will never get to a point to be comfortable with your server.

True. I'm not saying I'm too noob to do it. I'm saying I'm a newbie now and currently do not have time for the skills needed to deal with the scenario you brought up.

If you aren't willing to learn a little bit about the system you're building, and the OS that it runs, then you should stick to what you know.

Very true. I am learning of course. Enough to use the system as I want, at least that is what I hope. With advice from the web and forum, like here, I realize I knew less than I thought. So it is time to re-evaluate and I am thankful for that.

When something goes wrong, what are you going to do about it? Learn on the fly while your data is inaccessible?

That's usually how it goes. Does everyone practice disaster scenarios before actually using FreeNAS? Practically none of the guides or videos I've seen have talked about this. So it doesn't seem like something high on the collective hive mind.

I don't see how you think managing 9 separate drives on 9 separate pools is EASIER than running a pool of vdevs. What services are you going to run on this server? How do you think recovering from a pool loss is going to be easy? This is more than just drive letters and mapped network shares.

You're the expert here. From my perspective... Erasing the pool. Removing the drive. Add a new drive. Create a new pool. Done.
I do not plan on running any services, plugins or whatever on the NAS.

It's your server, do what you want, but given your current desire to ignore the warnings of the collective here, I don't think FreeNAS is appropriate for this configuration. At all.

You may very well be correct.

And I think that even though you could create 9 separate pools, shares, settings, etc, that isn't the recommended approach for freenas. But honestly, I don't think anything else is either.

And I really don't understand the "I can lose one drive at a time, but I can't lose everything all at once" concept.

Well, it's mostly a case of not having to download 25+ TB of data if everything is lost, vs only having to download 4 TB for a single drive. For all intents and purposes, time is the key here. I'd rather not spend a month to recreate my system. A single drive takes a few days.

 

[Ericloewe]

You're the one who mentioned that managing 10 pools would be ridiculous. I thought it was set and forget.

10 pools is. One pool isn't.

I am more worried about things popping out of the blue and kicking me over the shin. Like the scenario you mentioned, where losing a single disk in a single disk pool would trip up FreeNAS to the point of having to break out the command line and start talking straight to FreeBSD. Irk!

Well, that won't happen in a properly-managed RAIDZ2 pool. By that I mean "regular SMART tests and scrubs are working, email alerts setup and prompt action to remedy drive failures".

That's usually how it goes. Does everyone practice disaster scenarios before actually using FreeNAS? Practically none of the guides or videos I've seen have talked about this. So it doesn't seem like something high on the collective hive mind.

That's standard engineering practice. We tend to practice what we preach (or at least try - sometimes stuff doesn't work out), and making sure your server correctly handles common scenarios is essential to trusting it.
As for guides and videos, much of what is it there is painfully wrong. If there's a contradiction between some outside guide and a forum sticky, trust the forum.

You're the expert here. From my perspective... Erasing the pool. Removing the drive. Add a new drive. Create a new pool. Done.

You're missing crucial, non-trivial steps:

• Determine what data needs to be copied
• Determine where additional copies can be found
• copy data over to new drive

This procedure is just begging to be done by an entity designed to treat massive amounts of data - namely the server itself. These steps are fully automated by ZFS, in a proper pool (RAIDZ vdevs or mirrors).

I do not plan on running any services, plugins or whatever on the NAS.

Here's a big problem: How on earth do you get data onto the server without a file sharing service? Sneakernet?
Services aren't a weird, niche thing, they're fundamental to the operation of the server.
At the very least, you need CIFS or NFS or something similar.

Well, it's mostly a case of not having to download 25+ TB of data if everything is lost, vs only having to download 4 TB for a single drive. For all intents and purposes, time is the key here. I'd rather not spend a month to recreate my system. A single drive takes a few days.

With a properly-maintained pool, you should never, ever lose data. Nothing is foolproof, obviously, but the expected value of downtime for a proper configuration is "the ten minutes it takes you to replace the drive and turn on the server again". Then you just click the button and ZFS will handle the rest while still being available.
You do not lose days, much less months.

Losing a pool is like a plane crash. People work damn hard to make sure it won't happen and the hard work pays off by making flying the safest way to travel. The only difference is that digital data can be trivially backed up, whereas people can't just submit to a quantum photocopying.
Losing a disk is like losing an engine on a plane. It happens and it's unavoidable. No big deal, there's more that can keep things going safely for a while. If planes crashed every time an engine failed, we'd all be screwed.


tl;dr - You're supposed to build a reliable system from unreliable parts by using redundancy, not a small mountain of unreliable systems cobbled together in an ad-hoc fashion.

 

[jgreco]

Well, it's mostly a case of not having to download 25+ TB of data if everything is lost, vs only having to download 4 TB for a single drive. For all intents and purposes, time is the key here. I'd rather not spend a month to recreate my system. A single drive takes a few days.

That's a totally screwed up perspective. You think it is all right for it to take any more of your time than the five minutes it takes to slot in a new hard disk?

A properly designed FreeNAS system should never lose data. You're building reliability by designing a system that has redundancy. You can go out to very high levels of redundancy and the thing will keep purring away serving your data up.

The moment you say something like "only having to download 4TB" ... you're totally screwed, because it means you lost data. Disk space is cheap, technology is fantastic, you can make technology work for you, or you might as well just go back to storing things on QIC tape and praying the drive can read the tape....

 

[Honey]

Okay, so say I will go the route of a RAIDZx setup.

WD Reds keep being mentioned as good for NAS, but their URE is only 10^14. As far as I can read, this is unsuitable for any kind of parity raid with large capacity drives.
Is there something special about ZFS that somehow sidesteps this vulnerability?

 

[depasseg]

Before anyone answers that I would like to understand what you think a URE of 10^14 means and the vulnerability that it brings. And have you read anything about how ZFS works? I don't mean any disrespect. I'm trying to understand where you are coming from.

Sent from my Nexus 9 using Tapatalk

 

[Ericloewe]

URE is only 10^14.

1. That spec is close to meaningless
2. Here's how the math works:

P[data loss in a single operation]=P(URE)^(1+parity level) * P(same sector being affected)^parity level

Parity level is 2 for a fully functional RAIDZ2 vdev, but let's go with 1, for a degraded RAIDZ2 vdev missing an entire drive.
P(same sector) is roughly P(URE/(capacity/4kB)).

You'd then multiply by the number of read operations performed, which will be larger for larger drives.

 

[jgreco]

Okay, so say I will go the route of a RAIDZx setup.

WD Reds keep being mentioned as good for NAS, but their URE is only 10^14. As far as I can read, this is unsuitable for any kind of parity raid with large capacity drives.

So you'd suggest something like, oh, say, the Western Digital Se "Data Center" grade drives?

Be my guest. You want that "15" in there. Fine. But please do notice that what it says is actually "< 10 in 10^15", which is a fancy math way of saying "maybe just slightly better than 10^14 but definitely not 10^15".

Is there something special about ZFS that somehow sidesteps this vulnerability?

Yeah, it's called "reality".

See, an unrecoverable read error isn't a problem for ZFS when redundancy is available. ZFS just wanders over and gets the redundant data. So what you really want, with ZFS, is to do something like RAIDZ2. Then when your disk dies, you lose one full level of redundancy. And when you're trying to rebuild that failed disk, and you hit a URE, ZFS says "aw crap, I gotta do some work", and recovers that block from redundancy. The success of that recovery operation is then dependent on that one block being readable-without-error on the remaining disks, but with a 10^14 error rate, the likelihood of further read errors is vanishingly small. And if that worries you, then there's always RAIDZ3.

Part of this is I think you've read a bunch of stuff that may have been referring to RAID5. The hardware RAID5 guys do indeed have an issue with URE's, so RAID6 was invented, which reduces that problem for them. ZFS does that with RAIDZ2. But we can go one better with RAIDZ3.

But the other thing is that you've been merrily storing your data on disks without any sort of checksum validation, and it is perfectly possible you've already experienced a data-corrupting URE event at some point without even realizing it. The entire damage of a URE is the loss of a single block of data, so what I'm going to say at this point is that it strikes me as just a bit ridiculous that you'd come in here and get all OCD about URE levels when you're doing such a poor job of curating your existing data. ZFS will identify problems for you. If it cannot resolve them, it will even tell you which file(s) are damaged/unrecoverable. That's something most other systems just can't do for you.

 

[joeschmuck]

Wow, this thread has gone crazy. The simple answer here is that FreeNAS, and several other NAS software solutions do an excellent job of managing a large storage of data and they can be customized to meet most if not all needs of the user needs. FreeNAS is just a very good product which has a fairly simple to use GUI with a very fast throughput. I would suggest you try FreeNAS out and test the GUI and see how you like it. This could be in a virtual machine or on some old hardware that at least has 6GB RAM and a single hard drive (again just to test it out). If you like it, invest in the proper hardware to make a solid FreeNAS machine and perform the proper initial setup. With respect to the proper hardware, I would not use a USB Flash drive as the boot device if hands off is what you are looking for just because USB Flash drives do tend to fail, I'd stick with a small SSD (the cheaper the better) as this will outlive any USB Flash drive. And there are a lot of threads here discussing recommended hardware. If your usage is only as a basic NAS then you can buy a minimal hardware system. If you want to transcode and stream video content then a slightly faster CPU will be required. The thing I'd stay away from are drive encryption, just don't do it if you want an easy to replace drive solution.

I am more worried about things popping out of the blue and kicking me over the shin. Like the scenario you mentioned, where losing a single disk in a single disk pool would trip up FreeNAS to the point of having to break out the command line and start talking straight to FreeBSD. Irk!

FYI, if a single disk failed in a single disk pool, FreeNAS would be fine with it (it wouldn't crash) but your data would be all gone. No need to go into the command line, your data is gone. Okay, it is possible to do some data recovery if the drive wasn't physically dead but I don't believe FreeNAS would be the software of choice and data recovery is not a certainty at all. Also, a single drive pool is not a proper setup. Please keep in mind we are talking about a proper setup, anything less is a risk.

One of the benefits not mentioned is that if the computer FreeNAS is running on breaks, you can take all your hard drives to any other machine (meeting the minimum requirements of course), plug them in and you have access to all your data. This is a very cool feature to have.

Test it out and then if you have any more questions, ask. I don't want to force you into a NAS solution that you are not comfortable with, which is what this thread is feeling like.

 

[jgreco]

I don't want to force you into a NAS solution that you are not comfortable with, which is what this thread is feeling like.

I think part of the process of trying something new has to be that you allow people to rattle your cage when you're doing something that's clearly comfortable but way suboptimal. Otherwise there's much less value in the trying something new.

 

[Honey]

Alright, so I've been doing a bit more reading. I understand I cannot add disks over time, in order to build up a vdev, only replace existing disks.
I can add new disks as additional vdevs, but it would be "stupid" or dangerous, since I gain no redundancy by having several striped vdevs in a zspool.
Basically, if I want to use FreeNAS "appropriately" I need to get the amount of drives I want the final NAS to consist of and add them all at once. If they're smaller drives, I can replace them one-by-one as time goes on with bigger drives. But, new capacity will not be realized until the last drive is changed.
Sound good so far?

I had some misconceptions about how RAIDZ with ZFS worked. I did believe that encountering errors during resilvering would doom the entire process, as it does with traditional RAID. I am very happy that is not the case. A few corrupt files can always be restored from remote backup.

When I acquire the necessary drives to build the initial RAIDZ, I'll make sure to get a small SSD to host the OS. Thanks for that.
As far as non-hdd hardware goes, I'm quite happy with what I have, though the MB does not support IPMI. I wish I had known about that earlier.

I've gotten a better idea of what I can do with FreeNAS. Suddenly it feels like I'd do myself a disservice by ignoring all the cool features it offers.
Thanks again for the feedback and rattling.

 

[nojohnny101]

wow this thread has gotten crazy.

Alright, so I've been doing a bit more reading. I understand I cannot add disks over time, in order to build up a vdev, only replace existing disks.
I can add new disks as additional vdevs, but it would be "stupid" or dangerous, since I gain no redundancy by having several striped vdevs in a zspool.
Basically, if I want to use FreeNAS "appropriately" I need to get the amount of drives I want the final NAS to consist of and add them all at once. If they're smaller drives, I can replace them one-by-one as time goes on with bigger drives. But, new capacity will not be realized until the last drive is changed.
Sound good so far?

you're finally getting it! correct. of course there are caveats to everything. you don't always have to start with the number of disks you want the final FreeNAS to have forever and ever. Me for an example. I started with 4 x 2TB RAIDz2 but then my storage needs grew so I bought 6 x 3TB drives and created a new pool (RAIDz2) and them moved all my data to that new one. Then I used the old 2TB drives to build a backup FreeNAS box. So yes you can not expand a vdev once it is created but that doesn't stop you from creating one later and them moving everything over.

-----

I am going to say the following because I'm trying to help you, don't take it personally!
-> some of the most veteran people in this forum (no me, i'm a newbie still) have replied to you. it would behoove you to read what they have typed, digest it, understand it, and then ask intelligent questions back. almost all of what you have posted could have been answered in the manual. after you have read the manual, and you don't understand something, you can ask specific questions (believe me, no on expects you to understand everything in the manual). but don't post vague question and ramblings that ignore what was previously written as if you didn't read it. it can be frustrating for people trying to help.

sounds like you are on your way though. FreeNAS and Raidz2 or greater will manage and safeguard your data better than any hodgepodge of 24TB of data can spread across who knows how may external drives. yikes!

 

[danb35]

I can add new disks as additional vdevs, but it would be "stupid" or dangerous, since I gain no redundancy by having several striped vdevs in a zspool.

It's not at all stupid or dangerous to add new disks as additional vdevs, as long as the new vdev has adequate redundancy. For example, you have six disks in a RAIDZ2 vdev, and you add six more in another RAIDZ2 vdev. That's perfectly fine. What's dangerous is when you have your six disks in RAIDZ2 and want to add one disk. The GUI will let you do that (though you'll have to jump through some hoops to do so), but that means that your entire pool depends on that one disk. When it fails, you'll lose all your data.