ECC Ram with Lz4 compression.

[Patek2]

Hello,
I'm using IronWolf 2TB x2 drives with mirror configuration to have constant backup data. To be safe from data corruption on one of those two drives, Do I have to use ECC memory? As my server I'm using HP Prodesk 600 G1 and I don't think this PC is capable of reading ECC memory.

 

[Ericloewe]

LZ4 compression is not relevant to your question and does not affect the answer.

The answer is that if you value your data, you should take all reasonable precautions to safeguard it, and that includes ECC RAM.

As my server I'm using HP Prodesk 600 G1 and I don't think this PC is capable of reading ECC memory.

Indeed, you would need a platform with ECC support, such as those in the Hardware Recommendations Guide (see my signature).

 

[winnielinnie]

ECC RAM assures the data you intend to be written (as a record) is correct before being written to the storage media.

After this point, due to checksums and redundancy, ZFS will assure the data remains correct.

With non-ECC RAM, if the data were to be corrupted before being written to storage, ZFS will simply keep this ("incorrectly") written record integral.

According to ZFS, everything checks out.


ECC RAM
Create text file with the content: "apple"
Before writing it to storage, the file's content is actually: "apply"
The corruption is detected before writing it as a ZFS record to storage.

Non-ECC RAM
Create text file with the content: "apple"
Before writing it to storage, the file's content is actually: "apply"
This is not caught, and you in fact write a ZFS record to storage.
ZFS creates a checksum and uses redundancy for the file that contains: "apply"
Running scrubs and reading the file will not report any corruption. Because the checksum matches the record.
Your file will always "correctly" have the content: "apply"

 

[Patek2]

ECC RAM assures the data you intend to be written (as a record) is correct before being written to the storage media.

After this point, due to checksums and redundancy, ZFS will assure the data remains correct.

With non-ECC RAM, if the data were to be corrupted before being written to storage, ZFS will simply keep this ("incorrectly") written record integral.

According to ZFS, everything checks out.


ECC RAM
Create text file with the content: "apple"
Before writing it to storage, the file's content is actually: "apply"
The corruption is detected before writing it as a ZFS record to storage.

Non-ECC RAM
Create text file with the content: "apple"
Before writing it to storage, the file's content is actually: "apply"
This is not caught, and you in fact write a ZFS record to storage.
ZFS creates a checksum and uses redundancy for the file that contains: "apply"
Running scrubs and reading the file will not report any corruption. Because the checksum matches the record.
Your file will always "correctly" have the content: "apply"

So having scrubs and backup created in mirror is actually worthless for me?

 

[winnielinnie]

So having scrubs and backup created in mirror is actually worthless for me?

No. Not worthless.

Just that ZFS can't work magic if data is not properly written due to non-ECC RAM.

It's up to you to decide if the extra costs (and finding hardware) is worth it. If you use ZFS with non-ECC RAM, you still get all the benefits of ZFS. It's just that ZFS cannot "correct" what faulted RAM might have missed.

 

[Arwen]

@Patek2 - While memory bit flips are rarer than disk problems, without ECC memory you don't know if you have a problem during operation. (Off line / boot time memory checks can be done if you suspect a problem...)

And to add another complication to @winnielinnie's Non-ECC RAM first post, their is a window of time with ZFS where data could be check summed while in memory, and then the data damaged by bad memory. Thus, bad data written to disk causing permanent data loss, but detectable.


It is about risk avoidance. How much you want to avoid, and can afford to implement.

In my case, except for my NAS, none of the other home computers, all Linux, (desktop, laptop & media server), implement ECC memory. All use ZFS, both for OS, (Mirrored), and data pools, (some Mirrored with the Media pool striped). Even the backups use ZFS.

Now would I have preferred my desktop, laptop and media server have ECC memory? Yes
Would I pay extra for it? Yes
Was it available when I bought the devices? No.

If / when my media server kicks the bucket, I will attempt to get a replacement with ECC memory. (The media server is the oldest of the 3.)