Is FreeNAS suitable for this type of usage?

[Honey]

I need to be able to add hard drives to the NAS, that each get their own network share. No raid, parity drives, backup or similar. Backup will be performed regularly through upload to a backup service.

I have installed FreeNAS, so far with only one 4TB drive, but with space for more. It can max out my Gb connection when receiving and almost when sending, so I am quite happy about the speed.

But is it overkill? Do I still get the benefits of ZFS file system by not using raid of any kind and using each drive in its own pool? Is the 1GB of ram per TB still applicable in my scenario? I have 16GB so far and if I add 5 more 4TB drives I'll have 24 TB of total storage.

Should I continue to use FreeNAS or is something else more appropriate for my limited use?

 

[pirateghost]

Sounds like FreeNAS is not what you need. Look at openmediavault

 

[nojohnny101]

@Honey it would help if you provide us with more information.
- what is the primary usage case of the box?
- how many users is it serving?
- what type of environment?
- is it going to be accessed remotely?

the whole reason people choose freenas is because of the benefits of ZFS. One of the selling points of FreeNAS is that the software raid is so good. Not using Raidz2 and above (note: raidz1, which you seem to be using is NOT recommended) is like buying a ferrari and never driving it fast (rough analogy).

 

[depasseg]

But is it overkill?

I don't think it's overkill, but it might not be a great fit.

Do I still get the benefits of ZFS file system by not using raid of any kind and using each drive in its own pool?

Obviously, you don't get all the benefits. FreeNAS will be able to detect errors, but will be unable to fix them (unless you set copies=2, which will halve your available space on a single drive).

Is the 1GB of ram per TB still applicable in my scenario? I have 16GB so far and if I add 5 more 4TB drives I'll have 24 TB of total storage.

For basic file sharing, I think this is an acceptable ratio. The 1GB per TB is more of a guide for the lower end. As capacity grows it's less strict.

Should I continue to use FreeNAS or is something else more appropriate for my limited use?

I don't know if you are going to find something that really meets your needs. Can you provide some explanation why each drive must be it's own pool and share? Maybe there is a better way to architect the solution.

 

[Honey]

Okay, I'll try to fill in the blanks.

- The primary use of the box is file storage. Mostly movies, photos and other media.
- It will serve the household only, though its primary function will be to extend the storage available to my desktop.
- Remote access is not necessary.

The reason I want to keep each drive in its own pool - and this may rest on misunderstanding the concept - is that I don't want a zpool of multiple drives to fail because one drive fails. Especially since I am not planning on striping across multiple drives.
For reasons of organization, it actually suits me as well, keeping things separated.

Would it be possible to have the benefits of parity across multiple drives, without actually doing raid, but storing each file completely on its own drive only?
It seems replication is a tricky affair at best and I don't need the speed performance benefits of raid, so I'm trying to play it as safe as possible - while maximizing storage. I'll still have offsite backup frequently scheduled, so a loss of data will never be entirely catastrophic, plus very important files will also live both on my desktop, on multiple drives, as well as the NAS.

 

[Jailer]

The reason I want to keep each drive in its own pool - and this may rest on misunderstanding the concept - is that I don't want a zpool of multiple drives to fail because one drive fails.

Then add enough drives for parity to mitigate the potential. RAIDZ2 or even RAIDZ3 would be the choice you are looking for.

For reasons of organization, it actually suits me as well, keeping things separated.

That's what datasets are for.

You really should read the docs as well as do some research on what ZFS is and what it can do. You can do what you intend to do and keep your data safe from when a drive fails if you set things up correctly from the start.

 

[joeschmuck]

@Honey

FreeNAS can do basically what you are asking but you need to shift the way you think about storage. I understand keeping things separate and you can do that, but not really to a single drive safely. As others have suggested you should look at the docs and guides here, it will help you some. I'll try to explain it in a nut shell...

You create a pool of many hard drives, in order to have drive failure protection you basically add 1 additional hard drive for each drive failure you want to be able to handle. So here on this site we promote pretty heavily to have 2 extra drives (RAIDZ2) and this gives you the ability to have 2 drives fail while all your data remains safe. If you had a third drive failure, well all that data is gone. And this is fine because if you are a smart person, you would get an email from the NAS stating there was a drive failure and then you would purchase a replacement right away and get it replaced before a second drive failed.

As for separating your data, well you would create what is called a dataset and you can even denote how much space this dataset can have, like maybe you have one called "Movies" and you want it to have 20TB of storage, whilst you have another data set called "Computer Backups" and you can give it the default of unlimited storage. These datasets can be shared as mapped drive letters on your computers, that is what I have done. Additionally if yo wanted to you could assign only certain people to access these datasets but I don't do that in my home environment, I just hope that my dad doesn't start deleting files.

Which brings me to my last thing... Always have a backup of your important data on a different media, could be DVD or an external USB hard drive, but it's just the smart thing to do. I wouldn't put all my movies on other media, especially if I had purchased them and just ripped them for my own ease of entertainment.

As for RAM usage, you need 8GB RAM for the normal configuration. 16GB RAM for if you are going to use a lot of plugins/jails. Don't worry about the 1GB per 1TB of storage, that is for a true corporate optimized file system and only works up to a certain extent, home users don't really need to worry about it unless you are doing some stuff which requires lots of RAM. iSCSI comes to mind where more RAM is required to work better. What you haven't said was what your hardware is and you must use ECC RAM when using a ZFS file system or you risk data loss.

Also, you cannot just blindly add drives to a pool, ensure you know what you are doing as it's very easy to set yourself up for failure if you do not read the user guide. Ask if you are unsure before you add.

 

[pirateghost]

The reason I want to keep each drive in its own pool - and this may rest on misunderstanding the concept - is that I don't want a zpool of multiple drives to fail because one drive fails. Especially since I am not planning on striping across multiple drives.
For reasons of organization, it actually suits me as well, keeping things separated.

This is definitely a misunderstanding of what ZFS actually is and does.

You put disks into VDEVs, and your VDEV is where your redundancy/parity lives. Your pool is comprised of VDEVs. You really should look at a RAIDZ2 configuration for your proposed setup, if you truly care about the data being stored. If you simply do not care about the data being stored, then there is NO benefit for you to run FreeNAS. The overhead required for the system outweighs your needs. A simple Synology would suffice for what you are describing.

Would it be possible to have the benefits of parity across multiple drives, without actually doing raid, but storing each file completely on its own drive only?

That is in no way even similar to parity or RAID/ZFS. That is just a bunch of disks with data on them. If you lose a drive, you lose everything on that drive.

I don't need the speed performance benefits of raid, so I'm trying to play it as safe as possible - while maximizing storage.

Again, you need to take a moment to read and understand what RAID and in the case of FreeNAS, what ZFS is and does. A RAIDZx configuration is not about speed.

 

[Honey]

I have off site backup of all my desktop data thus far. I would do the same with the NAS.
At present, I don't have the time to get deep into freebsd command line or learn the intricacies of data recovery (assuming it works as advertised). It's also part of the reason why I want to keep things simple. It's not really a disaster for me if a drive dies. I'll just repopulate it from the off site backup. Drives die so infrequently, it's usually a rare annoyance.
It's also rather pricey setting aside multiple 10^15 drives, simply for the purpose of speedy recovery. It wouldn't take more than a few days to restore a full 4 TB drive from off site backup. So the question is whether I'd rather have many TB more of storage, or the 2-3 days. Since a lot of the content is non-mission-critical, it is OK for it to restore in the background. Mission critical stuff already is mirrored.

If even adding a drive to a pool can be fraught with danger, it would be behoove me to respect my current limitations and use the software from that point of view. I'm fairly certain I can add new drives as a separate pools and share out separately, without too much peril.
Even RAIDZ has limitations according to the ZFS Best Practices document. I'm not savvy enough, in this field, to really utilize and care for it, so it doesn't break unintentionally. My mistake if I had the idea it also came with a performance boost.

As far as overhead is concerned, you're correct. A simple Synology could have sufficed, except for the fact that they're more expensive to buy and populate than building my own. Offer less flexibility in expansion. Cannot be repurposed later, etc, etc. My current case has space for 10 drives, though I could expand to 15-20 with icybox. Plus a commercial NAS is not easily fixed if something breaks. There are also the usual suspects in customer reviews with horror stories about how their commercial NAS suddenly died for no apparent reason. That's why I built my own.

I have read about ZFS and RAIDZ, but obviously I'm still very much a newbie when it comes to understanding - and more importantly: properly utilizing - it all. Likely colored by preconceived notions of how things work on other platforms. Yes, I know it's a common mistake and I made it.
My question has been sufficiently answered and I thank everyone who has participated. It's entirely possible I'll do a RAIDZx setup down the road - use FreeNAS properly, as it were - but for now, I'll use this advanced tool as a simple hammer. I apologize for that :)

 

[depasseg]

Can I ask you what ZFS benefits would you like to take advantage of? To me, there are 2 groups:
1. Data reliability (whatever I write to disk will be written properly, and won't get corrupted either now or in the future)
2. Data Availability (My data will be available when I need it
3 (There is probably also maintainability, but that's very straightforward with most NAS solutions).

So for #1, the fact that I use ZFS, whether on 1 drive or thousands, gives me that assurance. Now if I'm only using 1 drive, and I only have 1 copy of the data, then ZFS will tell me when a file is corrupt, but might not be able to correct it. If I use 1 disk, and tell ZFS to make 2 copies, then it's very likely that the file is recoverable.

For #2, this means that I can physically have a drive fail (RAIDZ1) or 2 drives (RAIDZ2) or 3 drives (RAIDZ3) and my data is still there and available.

To me it sounds like #2 might not be that important to you. Is #1? Is there something else?

And if you are going to put 10-20 drives in single instances, you are going to get to the point where you are spending more time and effort backing up and recovering than it would have taken to just use the ZFS RAID feature.

 

[Honey]

#1 is certainly very important. I can imagine nobody relishes the idea of their data being corrupted.
#2 is less important. Of course, ideally, I want all data available all the time. But I can accept that some data might be gone for a short while, in case of complete drive failure.

With one drive per pool, if a file does end up being corrupt, my off site backup should be able to restore a functioning file - if it ever was in the first place.

However, I agree that ZFS RAID is awesome. Right now the first barrier to use is startup costs. I don't want to use low quality drives; several are needed for it to be viable. The second, and arguably more important, barrier is the learning curve necessary to utilize FreeNAS properly - and not destroy absolutely everything due to a newbie mistake.
I did a lot of research before settling on this path. Some of it is a blur by now. I do recall people having problems with data recovery via RAIDZ. Either it was too fiddly or it simply broke in some way. I imagine it probably works the way it's supposed to, but that particular topic is not one I have researched very much in depth. Mostly because I quickly decided it wasn't something I was interested in, due to aforementioned technical issues. Also that very helpful powerpoint that is pinned on this forum. It says "If one has no interest in learning FreeBSD command line, one should probably find another solution." I paraphrase of course. It also solidifies the decision to not get into deep waters :)

 

[pirateghost]

#1 is certainly very important. I can imagine nobody relishes the idea of their data being corrupted.
#2 is less important. Of course, ideally, I want all data available all the time. But I can accept that some data might be gone for a short while, in case of complete drive failure.

With one drive per pool, if a file does end up being corrupt, my off site backup should be able to restore a functioning file - if it ever was in the first place.

However, I agree that ZFS RAID is awesome. Right now the first barrier to use is startup costs. I don't want to use low quality drives; several are needed for it to be viable. The second, and arguably more important, barrier is the learning curve necessary to utilize FreeNAS properly - and not destroy absolutely everything due to a newbie mistake.
I did a lot of research before settling on this path. Some of it is a blur by now. I do recall people having problems with data recovery via RAIDZ. Either it was too fiddly or it simply broke in some way. I imagine it probably works the way it's supposed to, but that particular topic is not one I have researched very much in depth. Mostly because I quickly decided it wasn't something I was interested in, due to aforementioned technical issues. Also that very helpful powerpoint that is pinned on this forum. It says "If one has no interest in learning FreeBSD command line, one should probably find another solution." I paraphrase of course. It also solidifies the decision to not get into deep waters :)

All the more reason to look at other solutions/projects as FreeNAS is ZFS only.

Your idea that you will easily manage 10+ pools (as you alluded to wanting many drives) is laughable. The idea of managing that much data in individual pools is insane and too time consuming to be efficient or feasible.

I don't understand the desire to skirt around utilizing the benefits of FreeNAS for your proposed system. You gain basically nothing with this method and are wasting overhead resources and space for no gain. Even a single disk vdev 'stripe' is going to have disk overhead space consumed by the system.

If you really just want to manage individual disks as file shares, why not just install windows or any Linux distro and use that? What benefit do you perceive that you are gaining by using FreeNAS?

 

[depasseg]

However, I agree that ZFS RAID is awesome. Right now the first barrier to use is startup costs. I don't want to use low quality drives; several are needed for it to be viable. The second, and arguably more important, barrier is the learning curve necessary to utilize FreeNAS properly - and not destroy absolutely everything due to a newbie mistake.
I did a lot of research before settling on this path. Some of it is a blur by now. I do recall people having problems with data recovery via RAIDZ. Either it was too fiddly or it simply broke in some way.

You are really confusing me. Either you have backups and can easily recover (in which case data recovery isn't an issue), or you don't and availabilty is important. The stories you've probably read about data recovery are likely from people using things like a HW RAID set of JBOD Drives instead of direct attached (IT mode HBA). Or from people who didn't have backups and the failed pool was their last hope of data recovery.

If you can afford to lose each individual drive and feel fine recovering from backups, then you can certainly handle dealing with RAID Z(something).

As for startup costs, you don't need really expensive drives. In fact, since you have room for 10, you have options, since a single disk doesn't have to be a certain size. You can start with a single mirror of 2 disks, or a 10 disk disk RAID Z(1,2, or 3) or anywhere in between. [Remember in RAIDZ1, 1 drive can fail without data loss, RAIDZ2 2 drives and RAIDZ3, 3 drives].

What are your storage sizing needs? There are a lot of folks who can help you get to where you need to go.

 

[depasseg]

I don't understand the desire to skirt around utilizing the benefits of FreeNAS for your proposed system.

My guess is he is worried by the horror stories of people who have lost everything with an improper RAIDZ setup. But that's just a hunch.

 

[depasseg]

At present, I don't have the time to get deep into freebsd command line or learn the intricacies of data recovery (assuming it works as advertised).

You realize that if a drive in RAID dies, you use the GUI to replace the failed disk with a new one. It's a couple clicks. And the system rebuilds on the new disk in the backgroud. You aren't doing CLI level data recovery.

 

[pirateghost]

You realize that if a drive in RAID dies, you use the GUI to replace the failed disk with a new one. It's a couple clicks. And the system rebuilds on the new disk in the backgroud. You aren't doing CLI level data recovery.

And if a single disk pool fails, imagine the trouble that will present itself from all the services relying on that single disk vdev pool....

I foresee someone learning some command line stuff when their first pool fails.

 

[depasseg]

Good point. Losing a pool is a bigger pain than losing a single drive.

 

[Honey]

My main interest in FreeNAS was the ability to run off a thumb drive - which wouldn't get destroyed by constant writing. Windows is straight out for that purpose. Possibly a linux distro, but I'm not sure which would fit the bill.
ZFS's ability to identify corrupt files is a giant bonus of course.

So you guys are saying that if I run with a single drive in a pool, and that drive fails, FreeNAS will vomit all over itself and be unable to cope without delving into the underlying guts via the CLI?

About my storage needs, basically massive. I accumulate a lot of media files and they take up a lot of space. Right now I have about 25 TB in my desktop. Mostly a mix of various drives, internal and external. Instead of adding to that mess, I decided to make a more manageable solution from the ground up.

 

[Honey]

Also, forgive my ignorance, but I thought FreeNAS was more or less "set and forget". Why would it be a massive effort having 10 separate pools?

 

[pirateghost]

Also, forgive my ignorance, but I thought FreeNAS was more or less "set and forget". Why would it be a massive effort having 10 separate pools?

Let me ask you this first.

Why are you leaving your current solution with your hodgepodge of disks?