Blog article advising AGAINST ECC RAM

[no_connection]

I have run v7 several years with non-ECC memory with no known data loss or errors. So it works just fine, until it won't. Which is why I am migrating all data to a new ECC based server.
ZFS should never panic or hick up on corrupt data or metadata, with metadata being spread out in several copies (from ZFS presentation by Sun)
Yet that happens to regular users as proven above.

Theoretically ZFS should handle a few bitflips in RAM just fine, correcting it's mistake on the next pass. A stuck bit or thrashing RAM, not so much.
If ZFS could detect excessive "random" corruption and halt when all disks appear to generate errors, then that might save the pool. But it would not know the difference between that and resilvering a replaced disk as it uses the same process.
Or better yet, implement memory scrubbing treating every transaction with checksums. Although I doubt that would provide much performance if even implementable.

 

[joelmusicman]

Or better yet, implement memory scrubbing treating every transaction with checksums. Although I doubt that would provide much performance if even implementable.

What you're describing is exactly how ECC RAM works. Because it's hardware based the performance impact is negligible.

Edit: That's the reason why instead of trying to reinvent the wheel, the FreeNAS devs spec ECC RAM as a requirement. It really should've been all along, as ZFS was inherited from Solaris which also "requires" ECC. The trouble with ECC is that there's little way for the software to know if the hardware is there or not...

 

[cyberjock]

Some of the decision makers at iX have discussed putting some kind of warning in FreeNAS that would actually give you a yellow light or something else equivalent unless you have ECC RAM. The problem.. it's virtually impossible to cover all of the common configurations and accurately determine if a user is actually using ECC features or not. There's been examples of people buying hardware that "support ECC RAM" but don't actually use ECC features. This is usually in the AMD side of the house or Intel side of the house where people buy ECC and put it in, it boots and appears to work and they proclaim victory. But the chipset doesn't support ECC RAM, so they actually are using it in a non-ECC fashion.

 

[Pitaq]

OK, I'm old - I had my first professional programming job in 1969. I've seen a lot of changes since then, but one thing hasn't changed - hardware, all hardware, is in one of two states - it has failed, or it's going to fail. For this reason I don't configure any non-laptop system without ECC. RAM failures are fairly rare but I have encountered non-fatal RAM corruption in several systems in the last 10 years. Because of ECC it has been apparent there's been a problem before someone had to spend serious amounts of time to figure out what data had been corrupted, when it became corrupted, and if it can be recovered.

Everyone is entitled to their own view on this issue, but in my view, running without ECC is foolish. The mess that can be created by silent memory corruption is far more disruptive and expensive (and potentially catastrophic) than the price differential of ECC memory.

Think of ECC like homeowners or renters insurance - if you don't have anything of value, then you don't need it. Many people never make a claim on their homeowner's insurance - I haven't in 30+ years but I consider it important to my peace of mind.

 

[Szyrs]

I realise that this is almost a year old now but I'm new to zfs/freenas and it keeps coming up in Google and I have a couple of points to make.

The basic gist of this article seems to be that based on a lot of Google data, one can conclude that in real life, there is roughly an 8% chance you'll be effected by memory errors, which amounts to an error or two a year and for most non-enterprise users that small risk amounts to acceptable loss. It actually goes on to indicate other, simpler ways around the same problem but doesn't go on to say what. Backups don't prevent bit rot, and running an entire second machine is in no way cheaper than buying ram alone. Comments then debate the value of some peoples data a Lot of people talk about why ecc is more of a requirement than a recommendation.

I'd just like to take a step back from those comments and look closer at the Google research that is being cited. I believe that some of these conclusions are misguided.
1. The figure of 8% is given per dimm module, so it only applies to machines with one single module installed. The more accurate figure they offer is 30% of machines. I say more accurate because the recommended installed memory for zfs is something like 4gb+1gb per 1tb hard drive space.
2. Dimm modules that produce one error almost always produce more. This means that there is no neglegable loss from memory errors. You may only have an 8% of a dud per dimm module, but once you get one, it'll only get worse.

With regards to value, an 8gb ecc kit is roughly the same price as a storage hard drive. It'll probable outlast it though. And you'd get some beer money if you sold it second hand.

As for analogies about playing in the street per hour, try this one. If you stop brushing your teeth now, will your teeth fall out tomorrow? Next week? A year? ...it depends on a lot of things. Only one thing is sure. You will lose your pretty smile, sooner or later. But you will save money on dental care...

 

[marbus90]

Some of the decision makers at iX have discussed putting some kind of warning in FreeNAS that would actually give you a yellow light or something else equivalent unless you have ECC RAM. The problem.. it's virtually impossible to cover all of the common configurations and accurately determine if a user is actually using ECC features or not. There's been examples of people buying hardware that "support ECC RAM" but don't actually use ECC features. This is usually in the AMD side of the house or Intel side of the house where people buy ECC and put it in, it boots and appears to work and they proclaim victory. But the chipset doesn't support ECC RAM, so they actually are using it in a non-ECC fashion.

My idea was a simple string matcher. One could at least weed out /some/ non-ECC users trough only greenlighting the specific Celerons, Pentiums, Core i3s and Xeons. If it doesn't match:
"Your system does not use ECC memory. Please consult the documentation, the forum and/or the IRC-channel for further information."
Maybe one could do that with chipsets too, as only the server series chipsets from intel support ECC. That would at least be a workaround until the memory management can verify the info.

 

[porcomaster]

Good morning, I appreciate enormously the amount of information found in the forum, first I'm sorry for my grammar as english its not my first language, well, my uncle works with video editing and need a system for his small business, I always wanted a system for myself and before making his system, i decided to make one for me, I decided on an inexpensive system that in Brazil was $ 500, which in the United States would be 400, the memory ECC would double the price , $ 150 for 4GB here plus motherboard and processor ECC , but after this fantastic discussion I will make the system of my uncle with ECC, my system is only for torrent and movies, so non-ecc will not be a problem, anyway I would like to know if I run non-esc and if i lose data , i will also lose hard disk , or after formatting I can reuse it, I already made the purchase so do not have much choice lol :p ... but the in system upgrade i will put ECC, so out of curiosity, I'm adapting one case of an old iron lsr7, and the system will be


Msi Intel Lga 1150 Mini Itx H81i R$ = 363,89 / $ 112,32
2 Hyperx Fury 4gb 1866mhz Ddr3 Cl10 Dimm R$ = 217,88 / $ 67,25
Pentium G3258 R$ = 440,82 / $ 136,06
Fonte Evga 400w R$ = 244,90 / $ 75,60
Hd Seagate 2tb Modelo St2000dm001 R$ = 372,90 / $ 115,10

Total R$ = 1640,39 / $ 506,33

$ 1 - R$ 3,23974782

ps : minimun wage in brazil is 1,6 dolars and hour

anyway let me thank neck hole for his beautiful blog , that help me a lot

 

[Robert Trevellyan]

if I run non-esc and if i lose data , i will also lose hard disk , or after formatting I can reuse it

Running a system with non-ECC RAM does not increase the risk of hard disk failure.

 

[porcomaster]

Running a system with non-ECC RAM does not increase the risk of hard disk failure.

thank you very much , :p ...

 

[Mirfster]

Msi Intel Lga 1150 Mini Itx H81i R$ = 363,89 / $ 112,32

Is this your motherboard?

• I can't really tell if it supports ECC, maybe someone else will be able to determine that
• It has a "Realtek RTL8111G" NIC; so YMMV

Might be better to start a new thread (perhaps in the "Hardware" section) so as to not divert/necro this one. ;)

 

[Ericloewe]

I can't really tell if it supports ECC, maybe someone else will be able to determine that

It doesn't. Only C2xx PCHs do.

It has a "Realtek RTL8111G" NIC; so YMMV

Between "Why is this so slow?" and "Why does this not work at all?".

 

[porcomaster]

i actually pick this one , http://motherboards.specout.com/l/804/ASUS-H81I-Plus , same "Realtek RTL8111G" but i may cancel , if i find another itx under 600 reais(200$) on brazil , i will open in hardware section , i don't want to necro this post , and i will search on this lan card to see if i can get at least 400 Mbs

 

[Mark Holtz]

Just thought I would put my two cents in....

First off, there is nothing that says that you cannot run FreeNAS on a gaming motherboard using an AMD motherboard using non-ECC motherboard and using cheap hard drives. As far as I can determine, it should run. As a matter of fact, my first FreeNAS box was running off an Intel Core 2 Duo E6850 with a single hard drive. It worked!

Note that I forgot a single keyword: "Well". That is an important difference.

The emphasis that is made for FreeNAS is that "You are building a server." Furthermore, you are building a server for long-term data storage. That's different from a gaming rig. And, if you are going to go against the advice of several gurus, you had better know what you are doing.

Remember that Inter Core 2 Duo? It was originally a Windows XP machine that was junked and had both the memory and hard drive removed. Yet, it was good enough to scrounge up some spare parts, install FreeNAS, and play around with it. It was "good enough" to set up some backups, install Transmission, and doing some file sharing. Good enough to learn without investing a lot of cash. The failure that was experienced was not from the computer hardware, but from a single fan that was going to be a BLEEP to release. It caused me to move up my official build.

I have received some advice for my build in this thread. And, in that thread, I either followed the given advice or had good reasons not to. Supermicro motherboard? Of course. Start out with 16GB of ECC memory? Nawwww..... I'll go with 64GB of ECC. Only using a single 2TB drive? It's a temporary measure until I can afford the drives I want. Using a more powerful processor? Maybe I'm wasting my money, but there is this thing called "transcoding" on a media server.

But, there is one piece of advice that I ignored from the FreeNAS experts, and that is in the boot drive. The typical recommendation is either to use a USB stick, SATA DOM, or a small SATA SSD drive as your boot drive. I ended up using the NVMe point and put in a compatible m.2 drive as my boot drive. Why? There are too many USB drives out there with varying speeds, and prone to failure. Using a SATA SSD or SATA DOM means that I'm giving up a SATA port. So I ended up using a Samsung SM951 128GB drive, knowing full well that it is probably eight times larger than needed. Unfortunately, 128GB was the smallest size that I could get, and it cost $100. And, you know what? It showed up as "nvd0" in FreeNAS, and is working fine as a boot drive.

Lesson of the day? Listen to the experts, and use their advice to make your final decision. Know which advice to follow, which advice to take and enhance upon, and be aware of the consequences and take responsibility for the advice that you elect to not follow.

 

[porcomaster]

Just thought I would put my two cents in....

First off, there is nothing that says that you cannot run FreeNAS on a gaming motherboard using an AMD motherboard using non-ECC motherboard and using cheap hard drives. As far as I can determine, it should run. As a matter of fact, my first FreeNAS box was running off an Intel Core 2 Duo E6850 with a single hard drive. It worked!

Note that I forgot a single keyword: "Well". That is an important difference.

The emphasis that is made for FreeNAS is that "You are building a server." Furthermore, you are building a server for long-term data storage. That's different from a gaming rig. And, if you are going to go against the advice of several gurus, you had better know what you are doing.

Remember that Inter Core 2 Duo? It was originally a Windows XP machine that was junked and had both the memory and hard drive removed. Yet, it was good enough to scrounge up some spare parts, install FreeNAS, and play around with it. It was "good enough" to set up some backups, install Transmission, and doing some file sharing. Good enough to learn without investing a lot of cash. The failure that was experienced was not from the computer hardware, but from a single fan that was going to be a BLEEP to release. It caused me to move up my official build.

I have received some advice for my build in this thread. And, in that thread, I either followed the given advice or had good reasons not to. Supermicro motherboard? Of course. Start out with 16GB of ECC memory? Nawwww..... I'll go with 64GB of ECC. Only using a single 2TB drive? It's a temporary measure until I can afford the drives I want. Using a more powerful processor? Maybe I'm wasting my money, but there is this thing called "transcoding" on a media server.

But, there is one piece of advice that I ignored from the FreeNAS experts, and that is in the boot drive. The typical recommendation is either to use a USB stick, SATA DOM, or a small SATA SSD drive as your boot drive. I ended up using the NVMe point and put in a compatible m.2 drive as my boot drive. Why? There are too many USB drives out there with varying speeds, and prone to failure. Using a SATA SSD or SATA DOM means that I'm giving up a SATA port. So I ended up using a Samsung SM951 128GB drive, knowing full well that it is probably eight times larger than needed. Unfortunately, 128GB was the smallest size that I could get, and it cost $100. And, you know what? It showed up as "nvd0" in FreeNAS, and is working fine as a boot drive.

Lesson of the day? Listen to the experts, and use their advice to make your final decision. Know which advice to follow, which advice to take and enhance upon, and be aware of the consequences and take responsibility for the advice that you elect to not follow.

thanks for your advice, i am doing that to learn , so i can do lots of better for my uncle :p ... , in last case i sell it .... but at least i will be able to use my HD , processor and power source :p ...