Are RDX drives a scam? (Practical long-term storage dilemmas)

[winnielinnie]

My next M-Disc burn is going to use OpenZFS as the file system.

I find that intriguing. I did not think it was possible (or feasible) to use ZFS on a write-once medium, such as a disc. How does one replicate a dataset to a disc (e.g, DVD-R, BD-R, M-Disc), if you must first create a zpool on said disc? Have you successfully used ZFS on a DVD-R in the past?

Checksums to detect file corruption

Wouldn't using PAR2 essentially accomplish the same in terms of data integrity and recovery, without the need to make ZFS work for a DVD-R or BD-R medium?

 

[winnielinnie]

I use M-Discs for archival storage, and many Blu-ray burners will burn M-Discs. Of course, I made the mistake of buying 25GB ones, before I bothered to check my "critical" media, which ends up being less than 4.7GB single layer DVD size.

Upon further research, there appear to be contradictory conclusions about "M-Disc labeled Blu-Ray discs." (See below how I still lean towards using M-Disc labeled BD-Rs versus standard writable BD-Rs. You can probably guess which is which in the photo below.)

From what I have gathered, when it comes to DVDs, M-Discs use a radically different physical material ("stone-like") that is resilient against deterioration and environmental conditions. This is in contrast to standard writable DVDs which use an organic layer and dyes to record data.

However, when it comes to Blu-Ray writable discs (BD-R, BDXL, etc), M-Discs "supposedly" may not use any material that is inherently superior than the inorganic layers already used in standard Blu-Ray discs. Yet, there are promising home tests that some people have undertaken that show these M-Disc labeled Blu-Ray discs do in fact hold up against harsh conditions compared to standard Blu-Ray discs.

Left: TDK BD-R, 25GB | Right: Verbatim M-Disc, 25GB
Both discs subjected to the same harsh, unprotected conditions for over 6 months outdoors.
Partially buried in soil. Hanging on a branch. Direct sunlight. Rain. Hail. Very high and very low temperatures. High humidity.


With all of that said, I'm most likely going to purchase an external Blu-Ray/M-Disc reader-writer; burn a combination of 25GB (and possibly 50GB or 100GB) M-Discs; label them; and finally store them away in a fire-proof safe, along with the external reader. The extra cost of the M-Discs seems worth it to me, if the worst-case scenario is they're just as good (and possibly better) for data longevity, especially considering it's a one-time purchase of no more than a handful of these discs.

This is not a meant as a backup substitute, nor exclusive archive, of the data in question. It's a one-and-done method tangential to the ongoing "copy everything over to the next iteration of digital storage" game of Evil Musical Chairs.

 

[joeschmuck]

Nothing against using M-Discs but I have dozens of cheap CD/DVDs (none are BlueRay) that are from the early 1990's that are all in perfect condition. I do not have them sitting in direct sunlight often and I don't bury them in dirt nor stick them though a wood chipper or rub them with sand paper. I treat them with respect. The above photo is purely marketing. While M-Discs from Verbatim may be able to handle harsher environments, I have no plans to put mine through such environments.

I'm not trying to change anyone's mind, I took think M-Discs are good but I'm just making sure you understand that a normal person would not subject media they care about to those environments.

 

[cobrakiller58]

I have had a couple CD-R and DVD-R that after over a decade on the shelf near my computer were near unreadable, took over an hour to copy all the data off the BD-R and a ton of read errors and retries. The disc had no physical damage, no exposure to sunlight or other harsh conditions and other discs stored with it did not have the same problem even ones written the same day. I purchased an external BDXL writer shortly after that to back certain data up but so far have just dumped the data back onto my FreeNAS and it's backups......

maybe i'll just get a laser cutter and print onto stainless steel? that should last a good long time in most peoples homes

 

[jgreco]

I find that intriguing. I did not think it was possible (or feasible) to use ZFS on a write-once medium, such as a disc. How does one replicate a dataset to a disc (e.g, DVD-R, BD-R, M-Disc), if you must first create a zpool on said disc? Have you successfully used ZFS on a DVD-R in the past?

I would imagine you can make a temporary block device (RAMdisk, etc) and then dd the results onto the optical disk.

Wouldn't using PAR2 essentially accomplish the same in terms of data integrity and recovery, without the need to make ZFS work for a DVD-R or BD-R medium?

Sure, but the "2" in the name is the problem there. Parchive underwent a flurry of development for a bit and then became sort of a historical oddity with a limited use case.

Note that I'm the primary developer of the Diablo Usenet server software so I have more than a Wikipedia-lookup level of familiarity with the topic.

 

[jgreco]

[crazy]
So the brilliant solution here would be like a ZFS pool of a RAIDZ3 of optical media discs... hahaha
[/crazy]

 

[winnielinnie]

I'm not trying to change anyone's mind, I took think M-Discs are good but I'm just making sure you understand that a normal person would not subject media they care about to those environments.

No plans on throwing mine through a wood chipper either, but I see what you're saying. However, it is indicative of the quality of materials, which is a good sign for me, as when it comes to long-term cold storage I don't mind paying extra upfront for something I'm only going to do "once" as a complement to my other ongoing backups. (The M-Discs would exclusively hold specific files and archives, with access given to others.) There have been other "accelerated aging" tests conducted as well, which also lend some merits to the claims that M-Discs are likely to last hundreds of years if kept in ideal indoor conditions. Though the tests done on their M-Disc labeled BD-Rs were carried out by the company itself. (Whereas the M-Disc DVD-Rs were tested by the U.S. Government. Take from it what you will.)



Which brings me to this...

I have dozens of cheap CD/DVDs (...) that are from the early 1990's that are all in perfect condition.

...and this,

I have had a couple CD-R and DVD-R that after over a decade on the shelf near my computer were near unreadable, took over an hour to copy all the data off the BD-R and a ton of read errors and retries. The disc had no physical damage, no exposure to sunlight or other harsh conditions and other discs stored with it did not have the same problem even ones written the same day.

Like @cobrakiller58, I too have experienced the same thing: CD-Rs and DVD-Rs that appear in great condition, yet are either outright unreadable (corrupted file table) or only partially recoverable. They were from the same batch stored indoors and barely used. Their lifespan is unpredictable and can go from "fully readable" to corrupted without warning, unless you're constantly running diagnostics on them (e.g, dvdisaster), which is not practical.

Some discs were burned in the past before I (or a family member) even considered "backups of backups" or "ongoing copies as storage technology evolves." The label on one of the unreadable CD-R reveals that sentimental photos were written on it, and I have no other copies of such photos.

 

[pschatz100]

There are organic materials used in the production of DVD and Bluray disks that deteriorate with time. Heat, light, and humidity all have an impact. This is what contributes to their unpredictable lifespan. In general, I would expect name brands to be more reliable than noname "cheapies", but it is still a matter of luck. I could imagine it is possible to choose materials that will last longer, which is what the archive quality DVD's supposedly do, although I've not had the opportunity to test anything for 100 years - yet.

Another issue with DVD's is that the printed labels on the top side of the media play an important role in their performance. For long term storage, do not use regular marker pens to write on them. For short term use it is OK, but over time the ink interacts with the material of the label and can cause it to fail - which will impact your ability to read the disk. They make special marker pens for this purpose.

For years, I had been in the practice of copying my archive DVD's to new media every five or six years. Never had a problem. Lately, I have been archiving to usb flash drives. We'll see how that goes.

 

[joeschmuck]

My archives are my FreeNAS server, a single USB Hard Drive, a single USB Flash Drive, and DVD media. All but the Flash contain everything. Most of my backups are automated except of course the DVD's, that I tend to do periodically as I collect enough data to closely fill a DVD, which doesn't happen often. once in a blue moon I will go through my saved media and purge old data (typically programs that are obsolete) and I will build a new recovery disc with all the pertinent software I need.

I'm sure everyone has thier own way of doing things and mine changes as time moves one. At one time I used 3.5" Floppy, then DC300 QIC tapes, then moved to ZIP drives, then to CD media, and now DVD media as the main archival media. In 5 years it could be some sort of Flash media.

However, it is indicative of the quality of materials, which is a good sign for me, as when it comes to long-term cold storage I don't mind paying extra upfront for something I'm only going to do "once" as a complement to my other ongoing backups.

And I can't argue with that at all, I fully agree with your idea and I might go down that path myself if I hadn't had such great luck with my DVD's. I have never had a DVD that I couldn't read when used for backups. I have had some abused media that lays around, gets scratched, sits in the sun, those did happen so I'm not claiming that DVDs never gave me problems but it was my neglect which caused the problems.

My goal for my previous posting was just to say that normal good quality DVDs are reliable as well if you take care of them.

 

[Arwen]

@Arwen I thought perhaps for a moment you may want to discuss 1911's

No, I prefer FN 57. There is a factory down the street that makes them, so I can say I buy locally. Besides, 1911s have more of a kick than FN 57s. Plus, the flash of a FN 57 should scare any idiot away if you miss with the first round.

Wait, what are we discussing on a NAS forum???

 

[Arwen]

@winnielinnie To make a ZFS optical disc, as @jgreco has said, I'd;

• Make a temporary zVol of the correct size, with compression disabled.
• Make a Zpool in that zVol, (possibly with compression enabled). Probably use "ashift=12" for 4KByte sector, (or larger).
• Create any child datasets desired, (Media; ones with "copies=2"; Encrypted for tax or other private paper work).
• Fill it with your data.
• Scrub it once and verify it's good.
• Export this temporary pool.
• Burn it to disc, (or several)

To read:

• Install disc
• Import pool with "zpool import -o readonly=on PATH_TO_DEVICE"
• Copy any data desired

The questions remain, will this work?
Can you "scrub" a pool that has been imported "readonly=on"?

If you can't scrub it, you can try a simple read of all files to see if any throw errors.
Or possibly copy, bit for bit, the optical disc to a new zVol. Then scrub the zVol.

Depending on how busy I end up being in the next few days, I plan to try to burn a ZFS pool to a 25GByte M-Disc. It's a cheap enough test.

 

[Etorix]

Wait, what are we discussing on a NAS forum???

FN 57 = FreeNAS version 57, because Middle-Earth is a lot more advanced than Earth. What else?

 

[kiriak]

How many laser disks one will need for one copy of his data?
Will he want another one copy?
For how long will he feel safe without checking their integrity? 5 years? 10? more?
Will he need to check also that the reader is working OK and how often? Is a backup reader required?
How often and how much new data for archive will be created?
How much all these cost?

It looks that for the same or less money, any combination of 2+1 or 2+2 (HDDs + SSDs) is an alternative worth considering of.
More copies of the data, more and different media, can be kept in more places,
can be made and checked way faster.

Of course it requires periodic checking and less frequent periodic transfer to new media.
But these can save from obsolete hardware or failing of the media for any reason (if more than one copy exist).

 

[Patrick M. Hausen]

writing to tape from a FreeBSD-based system (i.e, FreeNAS and TrueNAS) is apparently reserved for masochists.

Reading that I wonder how we could possibly have been running tape changers in our datacenter for decades without any trouble. The Amanda "taper" process was always able to saturate the SCSI tape drives ...

 

[Heffy]

Very interesting read. My 2 cents worth... as a mechanic, I know you can't depend on any drive that relies on rubber based belts and drive components. Store that vacuum tube reel to reel tape recorder or brand new floppy/CD/DVD drive away for five years and you'll be lucky to read one tape/floppy/cd/dvd with it.

Sure, currently, you can pay big bucks for tape to digital conversion but one day soon, I can see this disappearing.

Don't laugh, but I did the audio tape to digital conversion after buying a second (the first was inherited!) 60 year old reel to reel vacuum tube recorder. It worked for the first four or five tapes before the belt was too far gone. Maybe, just maybe, I could get a replacement belt for the other 25 - 30 tapes.
The exercise was worth the effort, but not for the remaining tapes.

Show : NAS system

MOBO: ASUS M5A78L-M/USB3 AM3+ AMD 760G + SB710 USB 3.0
CPU: AMD FX-8320E Eight-Core Processor (3214.50 MHz K8-class)
RAM: 4-4Gb DDR3 1333 ECC Memory
PSU: ???
NIC: Intel 82576 Gigabit
BOOT: 240Gb PNY SSD boot drive
DRIVES: 1-3Tb & 4-4Tb WD-Red CMR NAS Hard Drives Configured as RAIDZ2
CASE: Compaq mid size 6 bay case
TrueNAS Core 12.0 U1
FreeBSD 12.2-RELEASE-p2

 

[joeschmuck]

Maybe, just maybe, I could get a replacement belt for the other 25 - 30 tapes.

Not a FreeNAS topic but since you brought it up... You should be able to locate one fairly easy, you will need to take a few measurements such as the width and the diameter of the belt (make sure the diameter is small enough to fit snug). Do not just go looking for a belt specifically for your device or you could be looking for a long time or paying a premium price. I would also ensure the belt has some fibers in it for rigidity/strength, but that is just me.

 

[Jakester Reno]

About 10 years ago, a company I help support IT tasks contracted with a healthcare related records management system company. The server provided had an RDX drive for backups. The company only wanted to provide 3 cartridges for backup rotation. I insisted on one for each day of the week M-Th and five 'Friday' cartridges. Within a year one of the drives had a failure. I disassembled the failed cartridge to find a 160GB WD Scorpio Blue drive. Years later when the server was no longer in use, I wanted to erase the cartridges. Since the server was no longer operational, I connected a good RDX drive, removed from the cartridge, to a standard SATA controller. Neither Windows nor Linux could work with the drive, which left me the only option to physically destroy the drives to get rid of the data.
Comments on the drives: 1) They do have a metallic tape shield to protect the logic board 2) There is a rubber mount to help protect against physical shock. 3) They are relatively expensive for the capacity.

I doubt that these drives will retain data significantly longer than the typical external USB drives. Because a special controller (the RDX carriage assembly) is required to read the drives, there is the additional concern that in the future, perhaps the drives will not be usable because support is dropped for the format. I have greater confidence that a computer decades in the future can be located that has a workable USB-3 compatible port to read the data, more so than the RDX format.