RAID 1 - Hardware or Software? UDF or ZFS

[BravoYankee17]

Hi,

I would like to build a NAS with 3*3 TB HD, 2 configured as RAID 1 and the third updated with Rsync.
My question is related to RAID configuration. Have I to buy a MB with RAID controller or is better to delegate RAID control to FreeNAS, i.e. have I to built an hardware RAID (through MB controller) or software RAID (through FreeNAS)?
Is it better to use UDF file system or ZFS? Exist RAID 1 with ZFS file system?

Thanks to all for your kind reply.

 

[ProtoSD]

You don't need a hardware RAID controller with ZFS, it's better because if your controller dies, you don't need to rely on finding an exact match to recover your data.

So to answer your questions, use ZFS with raidZ1, no RAID controller.

 

[matthewowen01]

Why not use all 3 in a ZFS mirror?

 

[BravoYankee17]

Why not use all 3 in a ZFS mirror?

Sorry, this is not clear to me. What is the purpose of 3 HD configured as ZFS mirror? How does it ZFS mirror works and which are the benefits and the disadvantages? I googled for more information but I didn't catch the point.

Thanks for your explanations.

 

[matthewowen01]

it sounds like from your original post, you want 2 to be mirrored, then a 3rd that is synced with rsync to have the same contents.

if that's the case, why not just have all the drives mirrored under zfs and skip the rsync replication.

if we ignore how we do it, what do you want to accomplish?

 

[globus999]

You don't need a hardware RAID controller with ZFS, it's better because if your controller dies, you don't need to rely on finding an exact match to recover your data.

So to answer your questions, use ZFS with raidZ1, no RAID controller.

Don't want to start a religious war here, but the alternative answer (based on my personal experience and googling) is as follows:

Hardware RAID, particularly mirroring works OK most of the time. With a few exceptions most hardware RAID *mirrors* (please note that I said *mirrors* and NOT raid0 or raid2 or... raidx) do not add any extra info to HDDs, particularly if they are same model, same capacity, same brand. So, yes, should one of the HDDs fails you will be able to simply unplug it and replugit into a different controller and everything will work OK. So, hardware RAID for mirrors is OK.

The issue with hardware RAID is the physical limitation of the number of HDDs that it can handle. In contrast ZFS can, theoretically, handle an unlimited number of HDDs. However, for all intent and purposes, most people use only 2HDDs for mirroring, so you are covered with a hardware RAID.

Hardware RAIDs used to be quite expensive, but now days are quite cheap, many mobos came with them integrated in it, so the cost is zero.

ZFS, on the other hand, is supposed to be the uber-file-system. However it has a number of flaws. It is software-based, which means that anything that affects the OS has the potential to affect the HDD's that ZFS is handling. Is notoriously picky with hardware. The current FreeNAS 8.0 implementation uses a *very* obsolete version. Buyer beware. In addition, it is not as resilient as hardware mirroring to power outages, disc changes or upgrades. Just google these issues and you will find plenty of problems and dissatisfied people. Lastly there are no recovery tools for crashed ZFS hdds. For example there are UFS or NTFS data recovery tools galore, however, for ZFS, zilch.

So, IMHO, for mirroring 2 HDDs I would go with hardware and not software. Now, if we are talking RAID X then there are other considerations.

 

[BravoYankee17]

it sounds like from your original post, you want 2 to be mirrored, then a 3rd that is synced with rsync to have the same contents.

if that's the case, why not just have all the drives mirrored under zfs and skip the rsync replication.

if we ignore how we do it, what do you want to accomplish?

Your suggestion is clear to me, thanks.
My concern is that if I delete a file it is deleted also in the RAID volume (after some time, ok) but if I have a second HD syncronized with Rsync I have some more time to restore a copy of the wrongly deleted file. One other way to save me from this error is to have two to sets of backup, i.e. one running every even days and the second running each odd days.

Have I reached to explain my point?

 

[ProtoSD]

BravoYankee17 - This is where snapshots come in handy. Each snapshot is like a new 'Recycle Bin' in Windows. It holds onto any changes/deletions since it was created. The bad part is that your disk space gets filled up with them because I believe they must stay on the disk you are creating them on, although you could copy the data contained in that snapshot and delete it with a cron job, snapshots can be very large....

But other than that I understand why you want to configure things like you suggested.

 

[BravoYankee17]

BravoYankee17 - This is where snapshots come in handy. Each snapshot is like a new 'Recycle Bin' in Windows. It holds onto any changes/deletions since it was created. The bad part is that your disk space gets filled up with them because I believe they must stay on the disk you are creating them on, although you could copy the data contained in that snapshot and delete it with a cron job, snapshots can be very large....

But other than that I understand why you want to configure things like you suggested.

If I will reach this weekend I'll do some test with old PIII 256 MB RAM and 2*320 GB IDE HD.
I am really new of FreeNAS and I would like to understand as much as possibile before to spend 800 € in a new system.
The following is the configuration that I am looking for:

INTEL - Processore Pentium G850 (Sandy Bridge) Dual Core GPU integrata HD 2000 2,9 Ghz Socket LGA 1155 Boxato
ASUS - Motherboard P8H61-M LE Rev. B3 socket LGA 1155 chipset Intel H61 Micro-ATX
KINGSTON - Memoria Dimm HyperX 8Gb (2x4Gb) ddr3 1333MHz Non-ECC CL7
WESTERN DIGITAL - Caviar Green Power 3 Tb Sata III 6 Gb / s buffer 64 Mb 7200 rpm
BE QUIET! - Be Quiet! Fan Silent Wings USC - 120mm
COOLERMASTER - Case Elite 310 Middle Tower Blu / Bianco
ANTEC - Alimentatore EarthWatts 430 Watt Continui Ecologico e con funzione di risparmio energetico
DIGITUS - SCHEDA RETE PCI EXPRESS 10/100/1000MBIT

I think it could be a very fast NAS...what is your feeling?

 

[ProtoSD]

I think you will have some problems with your old test system that has 256MB of RAM. This might be possible with FreeNAS .7, but with FreeNAS 8, if you are using ZFS, you should have a minimum of 4GB. Maybe for testing you could try 2GB, but you might have problems.

I'm not the best to give feedback about your ideas for your new system. Maybe some others can make better suggestions. It sounds like could be a nice system, but I'm not sure if the motherboard is compatible.

Anyone else care to comment?

 

[BravoYankee17]

You are absolutely right regarding the ZFS RAID with only 256 MB RAM, in fact my idea is to start with a less demanding software RAID not ZFS.
You are also right regarding the MB compatibility, I didn't check this aspect taking into account that I will not have an embedded RAID controller.

Anyway I don't this this is compatible, the chipset is too new.

I have a serious doubt if the better is to build a tailored system or if I would have to buy a branded system (like Synology).

 

[matthewowen01]

Don't want to start a religious war here

lol

Hardware RAID, particularly mirroring works OK most of the time. With a few exceptions most hardware RAID *mirrors* (please note that I said *mirrors* and NOT raid0 or raid2 or... raidx) do not add any extra info to HDDs, particularly if they are same model, same capacity, same brand. So, yes, should one of the HDDs fails you will be able to simply unplug it and replugit into a different controller and everything will work OK. So, hardware RAID for mirrors is OK.

The issue with hardware RAID is the physical limitation of the number of HDDs that it can handle. In contrast ZFS can, theoretically, handle an unlimited number of HDDs. However, for all intent and purposes, most people use only 2HDDs for mirroring, so you are covered with a hardware RAID.

Hardware RAIDs used to be quite expensive, but now days are quite cheap, many mobos came with them integrated in it, so the cost is zero.

ZFS, on the other hand, is supposed to be the uber-file-system. However it has a number of flaws. It is software-based, which means that anything that affects the OS has the potential to affect the HDD's that ZFS is handling. Is notoriously picky with hardware. The current FreeNAS 8.0 implementation uses a *very* obsolete version. Buyer beware. In addition, it is not as resilient as hardware mirroring to power outages, disc changes or upgrades. Just google these issues and you will find plenty of problems and dissatisfied people. Lastly there are no recovery tools for crashed ZFS hdds. For example there are UFS or NTFS data recovery tools galore, however, for ZFS, zilch.

So, IMHO, for mirroring 2 HDDs I would go with hardware and not software. Now, if we are talking RAID X then there are other considerations.

the problem with hardware raid vs zfs is the lack of checksums. if i loose a drive cleanly, we're good either way, but if a drive suffers bit rot or one drive drops you write some, then it comes back up, recyncing is a pain. it's hard to tell which drive has the correct data for a hardware raid. zfs is easy with the checksums. if you google zfs data recovery, there are plenty of good guides.

You are absolutely right regarding the ZFS RAID with only 256 MB RAM, in fact my idea is to start with a less demanding software RAID not ZFS.
You are also right regarding the MB compatibility, I didn't check this aspect taking into account that I will not have an embedded RAID controller.

Anyway I don't this this is compatible, the chipset is too new.

I have a serious doubt if the better is to build a tailored system or if I would have to buy a branded system (like Synology).

zfs needs memory otherwise it will be excruciatingly painful to run. the system doesn't need to be of the last 3 generations, but you need something more than that, but if you want to try UFS that should be ok i think.

 

[globus999]

the problem with hardware raid vs zfs is the lack of checksums. if i loose a drive cleanly, we're good either way, but if a drive suffers bit rot or one drive drops you write some, then it comes back up, recyncing is a pain. it's hard to tell which drive has the correct data for a hardware raid. zfs is easy with the checksums. if you google zfs data recovery, there are plenty of good guides.

Well.... I would agree to disagree. If you check the research that was done on different FS specifically on failures that mimic bit rot you will find that ZFS does not fare that well (surprisingly enough NTFS did quite well). As a matter of fact, the researches posited a number of improvements for ZFS precisely for bit rot. In addition, many hardware raids today do perform checksums so (I/O that is), it is not that much of a problem today. Wrt re-synching, I have personally gone through a number of such events in low-end to high-end systems and found that the process is -usually- quick and painless. Actually I find ZFS re-silvering being a complete PITA. But that's just me.

zfs needs memory otherwise it will be excruciatingly painful to run. the system doesn't need to be of the last 3 generations, but you need something more than that, but if you want to try UFS that should be ok i think.

My crappy "test" system is actually running PIII on 2Gb of memory OK. However, and this is a big however, I have crappy HDDs attached (40Gb to 250Gb). On those hdds it runs quite well.... providing ZFS is tweaked by hand. Out-of-the-box crashes under medium load. Once tweaked runs smoothly (but slowly).

 

[matthewowen01]

Well.... I would agree to disagree.

i don't agree to disagree ;)

If you check the research that was done on different FS specifically on failures that mimic bit rot you will find that ZFS does not fare that well (surprisingly enough NTFS did quite well). As a matter of fact, the researches posited a number of improvements for ZFS precisely for bit rot. In addition, many hardware raids today do perform checksums so (I/O that is), it is not that much of a problem today. Wrt re-synching, I have personally gone through a number of such events in low-end to high-end systems and found that the process is -usually- quick and painless. Actually I find ZFS re-silvering being a complete PITA. But that's just me.

I'd like to read the research if you can find the links. I like the resilvering process because it only does the data. if i have to replace a 2 TB drive on a nearly empty pool, it's very fast, on a hardware raid which is agnostic to the data, it has to do all 2 TB. my only experience with hardware raid was with a 3ware. Performance was pretty lousy for raid 5 and when ever a disk dropped (once a quarter or so), it took hours to rebuild.

My crappy "test" system is actually running PIII on 2Gb of memory OK. However, and this is a big however, I have crappy HDDs attached (40Gb to 250Gb). On those hdds it runs quite well.... providing ZFS is tweaked by hand. Out-of-the-box crashes under medium load. Once tweaked runs smoothly (but slowly).

memory is more important than the cpu for zfs. i can't wait until i get 16 GB on my file server. it's nice to hear how 'low' we can go though.

 

[ProtoSD]

globus999, I can understand you've had your share of negative experiences with ZFS, but is your point here to steer as many people as possible away from it and from FreeNAS, or is there something about it that compels you keep using it despite all the flaws you've pointed out in these forums? I'm not trying to provoke 'a religious war' with you either. It's good for people to be aware of potential problems, but every filesystem has it's weak points. You made it extremely obvious you don't like ZFS, and FreeNAS, so please tell us, why are you sticking with it?

 

[Daisuke]

I think you will have some problems with your old test system that has 256MB of RAM. This might be possible with FreeNAS .7, but with FreeNAS 8, if you are using ZFS, you should have a minimum of 4GB. Maybe for testing you could try 2GB, but you might have problems.

+1 on the memory size, ZFS needs a minimum of 4GB RAM to offer decent performance. 2GB won't cut it. Also I would install 8.0.1 Beta4 to have all the ZFS bug fixes. I watch every day the Announcements forum to see if we are getting closer to 8.0.1 Gold. :)

 

[Daisuke]

I have a serious doubt if the better is to build a tailored system or if I would have to buy a branded system (like Synology).

WOW, you want to spend nearly 1,000$ for a system that does not even has the disks included in that price? I've built a top of the line NAS for $800, with 6 x 2TB disks included. Not to mention the ridiculously slow transfer speeds you get in Synology...

 

[BravoYankee17]

WOW, you want to spend nearly 1,000$ for a system that does not even has the disks included in that price? I've built a top of the line NAS for $800, with 6 x 2TB disks included. Not to mention the ridiculously slow transfer speeds you get in Synology...

Actually I found a DS710+ less than 450 €, with 2*WESTERN DIGITAL - Caviar Green Power 3 Tb Sata III 6 Gb / s buffer 64 Mb 7200 rpm at 132 € each I will have a NAS with less than 750,00 € and a LAN RAID 1 continuos transfer rate little less than 95 MB/s on a Gigabit LAN capable of a theorical 125 MB/s maximum transfer rate.

I am perfectly in accordance with you if you say that I can not upgrade this system but, considering that the FreeNAS 8 version is still without iTunes server or Emule download capability and considering that this system has to be configured (and I am completely a newbie in FreeeNAS) I am not sure if I will reach the same performance and the same usability targets than a branded NAS.
This is my point.

Which is your transfer rate with your monsterNAS ?

I am of course available to discuss and to change my above mentioned statements.

 

[ProtoSD]

BravoYankee17 - I can understand your desire to get up and running with something that's *maybe* already stable, and the fact that you're not comfortable with FreeNAS and the learning curve... but.... I'd rather have something that's open source and know that I can add or customize features on my own and not be forced into some manufacturer's idea of what they think is important to you. Here's what I'd suggest. Pick your hardware as if you are going to run FreeNAS 8, start out with FreeNAS .7 which has the features you want and is more established and stable. Then wait until FreeNAS 8 gets up to version 8.1 and all the bells and whistles are working, and then upgrade. I've never used FreeNAS .7, but I know the userbase is pretty established and the support is there if you need help.

 

[BravoYankee17]

Protosd, I think I'll follow your suggestion, I will start using FreeNAS 7 in my old PIII (512 MB RAM and not 256 MB, however) and I will learn as much as possible on the various configuration.
If I perform a full installation (i.e. in a dedicated small HD) is it possible to upgrade to FreeNAS 8 without lose data or configurations?