My needs are few, my questions are many. (First FreeNAS build)

[TremorAcePV]

Hello,

I want a very simple home NAS setup that will utilize two 4TB Seagate 7200RPM HDD's in RAID1 (or FreeNAS's equivalent) for redundancy. 4TB is enough for my household as is.

Let's say money is tight and I want to buy one 4TB drive now and another later. I imagine putting it into the server in RAID1 would not be difficult (since it's the equivalent of copy/pasting a drive to another drive).

I tried researching this, and from what I read, cyberjock mentioned that adding a single drive to a pool was like playing with fire (great Moore's law reference). However, he was talking about a set up much larger than what I need, and I could be wrong in understanding what he meant due to my lack of knowledge in regards to ZFS, FreeNAS, and the like, as I've also read that it's very easy to grow a pool.

As for my hardware, I already have a moderate PC I built last Christmas. I've researched recommended hardware a bit as well on the forums, and from what I've seen, ECC is recommended for ZFS, getting server grade equipment is usually also recommended as the "premium" for it isn't that much more compared to the benefits, Realtek NICs can be crap or fine but aren't to be trusted, and low power is a major plus for 24/7 (Mine will be).

However, I already have a completely working computer that appears to be sufficient to run what I need.

Relevant Specs:

• AMD Phenom II X4 Black Edition 3.4GHz default, boosts to 4.0GHz.
  Yes, I know that it's a power hog when compared to server grade stuff, but this is my problem: I'm split between saving power/better efficiency over the long term by buying a system that is made for a FreeNAS server and using a system I already have that is capable (although not recommended) of utilizing FreeNAS.
• 6 SATA II ports on the Motherboard.
  More than enough for my needs.
• 8GB of Kingston RAM (4x2GB DDR3 DIMMS)
  From my reading, 1GB RAM for 1TB storage is the rule of thumb. The manual page recommends at least 8GB overall while other places say the minimum is 6GB. I was originally intending to turn an older system (DDR2 memory, for instance) into this server, but I didn't realize the RAM requirement would be this high. It's not ECC (obviously), but that goes back to my problem of either using what I have or investing the money in new hardware to optimize my system. I would also say my data isn't "vital" enough to warrant ECC being "necessary". Just media in general with system images and backups of games (my internet is painfully slow).
• It uses a Realtek onboard NIC atm, but I can easily buy and install an Intel NIC and will do so if I ultimately decide to go with this system as my NAS. Still considering it.
• 750w 80 plus Gold PSU
  Overkill, from what I've read, but it being Gold certified is a good thing.
• It's currently in a Mid-tower case, but I'd likely put it in a smaller one once it's set up.
• SanDisk 8GB USB drive My boot drive, and I have 2 extras.

I see the many drawbacks of using this (no ECC, high power usage, overkill PSU), but the positive of simply having it all ready and not having to buy new hardware (aside from the NIC) is a big one as I'd like to spend that money on Steam sales (Summer sale is imminent) and other hardware for gaming (Mechanical Keyboard).

Now, I have a few questions:

1. Are there any positives/negatives I am missing in regards to the choice between using current hardware or buying new hardware for each side, considering what hardware I currently have?
2. Will 8GB RAM, in your opinion/experience, be sufficient for my needs?
3. Would an SSD for cache aid in system performance? (This may be covered in the manual, but I may have missed it.)
4. Should I go with UFS rather than ZFS if I decide to use my currently owned hardware?
5. Are there hidden difficulties that I've yet to learn about in simply changing a single drive set up to a dual, RAID1 style setup with ZFS? (From the 3rd paragraph.)
6. If you had these simple needs, with your knowledge/experience of FreeNAS, what setup would you build (i.e. hardware used)? (So I can consider how far my current hardware strays from what would normally be used)

Feel free to answer/not answer any of the questions.

Advice, suggestions, comments, and opinions welcome.

Thanks,
Vitalius (The V in my name).

 

[TremorAcePV]

Is this the wrong place for a post like this?

 

[russnas]

im not sure but if your running 24/7 if so you should get efficient server hardware

otherwise

dont worry about ecc and server grade equipment if its just for media, domestic, you need a practical setup availability ,affordability etc, i have consumer hardware but im only running for media, not running 24/7.

try underclock your cpu, disable cores, you wont need so much power, and get at lease 16gb ram

i have one drive 4tb in one pool and another 4tb backup, data is backed up on usb drives too. im not storing all my data on it since i dont access it frequently or at all,

i perfer zfs,i haven't tried with a ssd drive as its just a media box
http://constantin.glez.de/blog/2010/07/solaris-zfs-synchronous-writes-and-zil-explained

sorry in a rush will read again later, many posts dont get a reply, you will have to keep researching otherwise,

 

[TremorAcePV]

Ok,

Will do. I figured going all out would be pointless overall.

I didn't even consider underclocking/disabling cores. How many cores would FreeNAS be able to use? In other words, how many should I disable?

16GB? Has it increased from 1GB for every 1TB to 2GB for every 1TB? Or is that just for overhead to be safe?

Alright, thanks for your time. I figured that's how it would be. This is kind of a "niche" forum from what I've seen.

im not sure but if your running 24/7 if so you should get efficient server hardware

otherwise

dont worry about ecc and server grade equipment if its just for media, domestic, you need a practical setup availability ,affordability etc, i have consumer hardware but im only running for media, not running 24/7.

try underclock your cpu, disable cores, you wont need so much power, and get at lease 16gb ram

i have one drive 4tb in one pool and another 4tb backup, data is backed up on usb drives too. im not storing all my data on it since i dont access it frequently or at all,

i perfer zfs,i haven't tried with a ssd drive as its just a media box
http://constantin.glez.de/blog/2010/07/solaris-zfs-synchronous-writes-and-zil-explained

sorry in a rush will read again later, many posts dont get a reply, you will have to keep researching otherwise,

 

[russnas]

i havent tried any lower than 16gb,,
Zfs ram requirements will depend on your file system i/o load, types/sizes of files, types and rates of file system ops, etc
you will be fine with 8gb transferring data to zfs may take longer.

i got a intel g2020 dual core similar to the e8400, clocked at 1.6ghz , average 25% used, you have an unlocked multiplier try 1 - 2ghz dual,after you transferred everything,see how 1080p movies or general copying once you underclocked,

if possible get a power plug watt reader and see how much its drawing, i get 43watts average.

 

[Caesar]

I have 8GB and performance meets my needs. I use my NAS to serve my two xbmc media centers. 1080p streams without issue. I have always ran my PCs 24/7 for years now. Most PC's have a single drive and work just fine for day to day. Now I do believe in backups and there is some risk running just one drive but I personally do not see a big problem with you getting one drive for now and then adding another for redundancy later. Just make sure later is within a few months and not years.

I don't think you need an ssd at all.

zfs has more features than ufs and you have the power to run so I would use zfs.

I have never config'd a mirror in freenas so I am not sure what all is involved. I believe you can add a disk and start a resliver and your data will not be deleted but you would have to double check that.

Here are my specs for comparison to yours

CPU - AMD A4-3400
MB - MSI A55M-P33
RAM - 8GB
Hard drive - (3 x 500GB ST3500418AS) (3 x 2TB WD20EFRX)
Vdev1 - Raidz (3 x 500GB)
Vdev2 - Raidz (3 x 2TB)

 

[jgreco]

dont worry about ecc and server grade equipment if its just for media, domestic, you need a practical setup availability ,affordability etc, i have consumer hardware but im only running for media, not running 24/7

This is simply wrong; worry about ECC unless you don't value your data and don't mind if undetected memory corruption causes ZFS to shred your pool during a scrub.

 

[TremorAcePV]

Thank you for all the help guys. This makes it easier for me to decide.

i havent tried any lower than 16gb,,
Zfs ram requirements will depend on your file system i/o load, types/sizes of files, types and rates of file system ops, etc
you will be fine with 8gb transferring data to zfs may take longer.

i got a intel g2020 dual core similar to the e8400, clocked at 1.6ghz , average 25% used, you have an unlocked multiplier try 1 - 2ghz dual,after you transferred everything,see how 1080p movies or general copying once you underclocked,

if possible get a power plug watt reader and see how much its drawing, i get 43watts average.

I'm fine with backups/transfers taking longer. I have many things to do in the meantime. As long as I could stream a movie (1080p24) if I wanted, that'd be perfect. That only takes like 5MB/s at worst and less than 1MB/s at best, so I'm not worried about it.

That's true. I really can't believe I didn't think about locking cores and lowering the clock speed of my CPU. I'll definitely do that. So dual core with a 1-2GHz speed is good for ZFS. Awesome.

I have 8GB and performance meets my needs. I use my NAS to serve my two xbmc media centers. 1080p streams without issue. I have always ran my PCs 24/7 for years now. Most PC's have a single drive and work just fine for day to day. Now I do believe in backups and there is some risk running just one drive but I personally do not see a big problem with you getting one drive for now and then adding another for redundancy later. Just make sure later is within a few months and not years.

I don't think you need an ssd at all.

zfs has more features than ufs and you have the power to run so I would use zfs.

I have never config'd a mirror in freenas so I am not sure what all is involved. I believe you can add a disk and start a resliver and your data will not be deleted but you would have to double check that.

Here are my specs for comparison to yours

CPU - AMD A4-3400
MB - MSI A55M-P33
RAM - 8GB
Hard drive - (3 x 500GB ST3500418AS) (3 x 2TB WD20EFRX)
Vdev1 - Raidz (3 x 500GB)
Vdev2 - Raidz (3 x 2TB)

You have just under my total (final) size, so yep. If it works for you, it will likely work for me. Cool. Will do.

Figured about the SSD.

Gotcha.

Great, that helps a lot, thanks.

This is simply wrong; worry about ECC unless you don't value your data and don't mind if undetected memory corruption causes ZFS to shred your pool during a scrub.

I'm willing to guess that is what happened to you and likely why you advocate ECC so much, though I could be wrong.

Hmm, so if I don't use ECC, it's possible that there will be an error and it will basically destroy all my data with other corrupted data. At least that's what it sounds like what would happen based on how a "scrub" was explained on the FreeNAS feature page.

... If I'm only doing RAID 1, as there's only 2 drives (or will be) at most, why would a scrub be necessary? It's simple replication.

I could understand if my setup were more RAID5/6-like, but it's not. If I ever upgraded to that much memory and redundancy, I could understand getting ECC, sever-grade stuff, and the like.

Also, my data isn't important enough for me to be bothered by losing it. Media (movies, games, pictures, and programs) are about it.

The worst thing that would happen if my current data drive failed would be that I would have to buy a new one and I'd have to re-download it all. And my internet connection is relatively horrible (375KB/s down, 75KB/s up), so that's what I'm avoiding.

I'm more interested in FreeNAS for it's ability as a NAS, and it's functionality as a remote server. Being able to access it from the internet would be nice to put files on it, or to get files from it. Redundancy is a bonus.

I understand getting a file off the server with 75KB/s would be slow and painful, but that doesn't limit my intended use for it.

 

[jgreco]

I'm willing to guess that is what happened to you and likely why you advocate ECC so much, though I could be wrong.

Nope. I do servers professionally, and I don't cut any corner that I don't actually choose to cut. ECC everywhere that supports it, basically only some laptops and maybe some Atom gear that doesn't, all I can think of. But we have seen people come through here who have had bad non-ECC memory and ended up with a shredded pool. Draw your own conclusions, or feel free to be the next example! ;-)

Hmm, so if I don't use ECC, it's possible that there will be an error and it will basically destroy all my data with other corrupted data. At least that's what it sounds like what would happen based on how a "scrub" was explained on the FreeNAS feature page.

Well, you see, that's the thing. It's not that likely. I recently compared it to driving around in a car without a seatbelt. You usually don't need them. You usually don't get hurt by not wearing them. Even in a catastrophic crash, you are not guaranteed to be saved by them. So ... do you wear them? Why (or why not?)

I figure that if you're willing to spend a thousand dollars on a NAS device that supports ZFS, there are only a few realistic subclassifications:

1) You really care about your data and you've got something like a bunch of disks in RAIDZ3.

2) You're just looking for convenience in managing a lot of storage, but don't value the data too much.

If you're user class 2, then I'll politely tell you that failure to ECC is a mistake and let you be on your way. If you're user class 1, I may bang on and on for a while to make sure you understand.

... If I'm only doing RAID 1, as there's only 2 drives (or will be) at most, why would a scrub be necessary? It's simple replication.

Because there's no "only". A RAID5 shouldn't have errors either. But you know what, sometimes it does.

The real purpose of the scrub is to proactively go looking to recognize problems that could possibly be detected. See, contemporary SATA hardware already supports SMART tests, so you can merely schedule and monitor those and be fairly certain that you're going to get warnings of failing/failed drives. But it is unclear what each manufacturer actually implements for a long test. There's no reason you can't go around reading it all and verifying consistency yourself though. Enterprise RAID controllers do this all the time; LSI calls it "patrol read" and you can be sure that all us retentive types use stuff like this. ZFS is kind of special; since it checksums data, it can scrub even a nonredundant pool in a meaningful way.

Because, you know, sometimes, undetectable disk errors do pop up. It is rare. But ZFS is designed to love your data, and it'll notice it, and fix it if possible. But not using ECC memory potentially sabotages the assumptions ZFS makes.

 

[TremorAcePV]

Thanks for you insight. I'm definitely classification 2.

What I meant by "only RAID 1" is that, the way a "scrub" was described, it would only apply to a RAID 5/6 array.

From the page,

"Every ZFS filesystem is also verified with checksums from top to bottom to ensure data integrity. If inconsistencies are found, parity blocks can be used to repair corrupt data. A regular scrub is turned on by default and can be rescheduled or configured from the web interface."
The term parity blocks being used makes me think of RAID 5/6, but unless you are calling an entire drive a "parity block", then a scrub, as defined, wouldn't apply to RAID 1.

But then again, I'm a noob at this so far, so I could be completely wrong. If I did put it in RAID 1 (essentially) and left scrubbing on, what would the parity block be? The other drive?

If I were to just turn off scrubbing and did a scan from time to time (every day or two), what would you suggest to use for that? Assuming I can't use anything in FreeNAS for it without ECC as the results would be the same as leaving scrub on.

 

[jgreco]

So what you're saying is that it makes sense for the system to repair your data if you've got a data protection model that is based on parity computations, but not if it has a data protection model that is based on simple redundancy. That makes no sense to me. The scrub system in ZFS does not work the way a RAID5/6 system does. Instead, it traverses every allocated block of the pool, looking for problems. So a scrub of a nearly empty pool is very quick. Error detection and recovery in ZFS is much smarter than your average RAID controller. Unlike your RAID controller, if ZFS reads a block off one disk of a mirror, or of a RAIDZ, and it does not pass the checksum, ZFS will not pass that back to the requesting application... ZFS will instead attempt to reconstruct the data using whatever data-protection model was used. But a scrub is needed for actual repair (rather than "on-the-fly reconstruction") of the problem. ZFS doesn't get all worked up about the specific data protection model that was used. You can even have a single disk, with copies>1 set, and should something render some of the disk blocks unreadable, you'll still have an intact filesystem as long as one copy of each block remained unaffected.

 

[russnas]

i know ECC importance but to me media data isnt as important, i can live if something happens , if business or professionally then its a must.
i have my data backed up on several drives, the cost of a ecc motherboard 1155 or 755, ram made it unaffordable for my current needs and supermicro boards are not in circulation here, so i built a pc with parts that can sell to the gaming area later on and buy a better unit.

being my first freenas, i saw this a better option than htpc
i have been safe with data corruption so far, (thank god).

 

[TremorAcePV]

So what you're saying is that it makes sense for the system to repair your data if you've got a data protection model that is based on parity computations, but not if it has a data protection model that is based on simple redundancy. That makes no sense to me. The scrub system in ZFS does not work the way a RAID5/6 system does. Instead, it traverses every allocated block of the pool, looking for problems. So a scrub of a nearly empty pool is very quick. Error detection and recovery in ZFS is much smarter than your average RAID controller. Unlike your RAID controller, if ZFS reads a block off one disk of a mirror, or of a RAIDZ, and it does not pass the checksum, ZFS will not pass that back to the requesting application... ZFS will instead attempt to reconstruct the data using whatever data-protection model was used. But a scrub is needed for actual repair (rather than "on-the-fly reconstruction") of the problem. ZFS doesn't get all worked up about the specific data protection model that was used. You can even have a single disk, with copies>1 set, and should something render some of the disk blocks unreadable, you'll still have an intact filesystem as long as one copy of each block remained unaffected.

Alright, thanks. That clarified how the scrub feature works.

I'll just stick with my current plan then. I'll upgrade when I feel that my data has either grown too large to bother re-downloading (like, 3TB+) or I get something I absolutely can't lose (family, financial, etc).

 

[survive]

Hi TremorAcePV,

The one really nice thing about AMD boards is that most all of them actually do ECC. If you haven't bought your RAM yet check & see if your board supports it.

-Will

 

[TremorAcePV]

Hi TremorAcePV,

The one really nice thing about AMD boards is that most all of them actually do ECC. If you haven't bought your RAM yet check & see if your board supports it.

-Will

Lo' and behold, there is an ECC option in the BIOS that is currently disabled.

Hmm, I'll be buying 2x4GB sticks of ECC RAM now. That will leave two slots for later if I ever decide to upgrade. It's currently running and setup (the shear amount of options kind of dumbfounded me at first, but I figured it out with tutorials) now, and it's only using 2 cores at about 2.1GHz.

I lack a wattage meter, but I'm going to guess that it has fairly low power consumption.

Sadly, the motherboard only supports up to 16GB of RAM, but I don't think I'll need more than that.

Thanks for your help. I never would've realized that if you hadn't said something.

One last question, I am going to buy an Intel NIC for my NAS. I assume a $30 one is fine? Are Realtek NIC's really that bad, and what do they do that makes them so bad?

I've heard they are hit or miss, some working fine and others failing spectacularly. How do I know which one mine is?

 

[russnas]

yea i just found that many amd boards support ecc, another reason to chose them over intel

 

[russnas]

heres a few good reads about on why ECC ram is a must

http://cr.yp.to/hardware/ecc.html

http://lambda-diode.com/opinion/ecc-memory

http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

 

[survive]

Hi TremorAcePV,

Take a look at this for way Realtek NIC are not exactly loved:

http://fxr.watson.org/fxr/source/pci/if_rl.c

Starting at line #48

-Will

 

[cyberjock]

That is.. HILARIOUS! I figured the story would be a mess in driver land.. but I didn't expect that it would suck so badly that there would be comments in the Realtek drivers mentioning how crappy it is! Awesomeness!

 

[jgreco]

Oh there have been all sorts of commentary about the quality of hardware in the drivers for years. "UTSL" isn't a comment aimed exclusively at code jockeys. Specifically relevant to ZFS and FreeNAS, there is some useful documentation for ZFS tuning present in the source code that isn't immediately obvious from Google searches for tuning.