Backup Solution? HDD vs M-DISC vs Other?

[MalVeauX]

Greetings all,

I'm starting to consider going a bit deeper into real backup solutions and I'm curious what the community is doing out there for important data. But not large pools of living working data sets, more like archival approach to small but important data sets. The common 3, 2, 1 method of course comes to mind. However, that requires refreshing fairly often and that will become super tedious when you consider over the course of a few decades. But maybe it's still superior? This is why I'm looking to see what others are doing.

Data sets to consider:

Basically anything archival such as family photos and some basic documents, records, etc. Mostly doesn't take up a lot of space. Anywhere from 250Gb to 1TB per year at best (when video is involved, its near that 1TB mark, when it's less video, it's near that 250GB mark). The size is mostly due to RAW formats of photos. Also looking to maybe backup any important videos and all my music library as that would be dreadful to attempt to rebuild one day. I don't care so much about losing video as most of it is not important and can be gotten again without much fuss. The music however would be difficult. The photos would be impossible of course to retrieve without true backups.

Current backup solutions I use:

Currently I'm using one or two living copies of the data (the photos for example). I have them on a 2nd hard drive as a second physical copy. I have most of them also off site in a cloud, but only the JPG format, not my RAW's. So currently I feel fairly ok. But I'd like to go a bit farther than this and start archiving the RAW stuff too.

Considering new solutions or reinforced current solutions:

HDD will only be ok for a few years, then must be refreshed.
Cloud requires internet connection.
Optical formats (M-Disc) can last our lifetime, but are low capacity and slow to write.
Tape is.... still around?
SSD is not good enough for archival class backup yet.
Flash is not good enough for archival class backup yet.

Sort of leaning towards M-Disc for the small data sets for archival purposes. Seems pretty bullet proof. Maybe I'm missing something?

Again, this is from the idea of archival backup, not simply redundant data sets. For redundancy of living data, I will have all my data available on my live system as a physical copy, on my NAS with redundancy as a second physical copy (technically three physical copies as a mirror) and I'm considering a 3rd copy on a physical hard drive to be refreshed every 3~4 years in a fireproof box. But I'm thinking, how about a long term archival approach to the smaller data to avoid the constant refresh? Maybe optical?

So what are you guys doing for archival backup solutions for the long term?

Very best,

 

[Basil Hendroff]

That's a lot of fiddling around. Maybe okay for in the interim. You haven't mentioned the word replicate. If you're not replicating, you're creating unnecessary work for yourself. There are effective ways to minimise the tedium.

This is what I do for data. Replicate all datasets from a primary FreeNAS server to secondary FreeNAS servers located on the same network. The most important data is replicated to an offsite server (at the folk's place) on occasion and the link to that server then dropped to try to satisfy the 3-2-1 rule that @Heracles advocates and which makes good sense (I'm not quite there yet though as I drop the link logically and not physically, but based on my risk appetite, I'm okay with that). Backup is a no-brainer with this arrangement. It just happens transparently in real-time. The hardest part of this arrangement is disciplining myself to fire up the link to the remote server on occasion to allow important data to sync. I usually do this when I'm aware of significant changes to key data.

Based on my own experiences, I'd suggest a longer-term two-stage approach for you:
Stage 1: Fund a second FreeNAS server and replicate your data to it. You still have your current hardcopies for backup. Once you're comfortable with the arrangement and can appreciate the benefits of replication, move to stage 2.
Stage 2: Fund a third FreeNAS server and replicate your data to it. Make the most powerful server your primary server. Move your least powerful server offsite as an archival server and refresh the data on it on occasion. Box your hardcopies and store the boxes in a place the sun don't shine. It's unlikely you'll ever need to access them again.

Note: As I'm a happy snapper, I also have near real-time backup of data from my smartphones to the primary FreeNAS server. I use a combination of Resilio Sync, Plex Camera upload and NextCloud instant upload to have photos and videos backed up in triplicate.

 

[MalVeauX]

If you're not replicating, you're creating unnecessary work for yourself.

Based on my own experiences, I'd suggest a longer-term two-stage approach for you:
Stage 1: Fund a second FreeNAS server and replicate your data to it. You still have your current hardcopies for backup. Once you're comfortable with the arrangement and can appreciate the benefits of replication, move to stage 2.
Stage 2: Fund a third FreeNAS server and replicate your data to it. Make the most powerful server your primary server. Move your least powerful server offsite as an archival server and refresh the data on it on occasion. Box your hardcopies and store the boxes in a place the sun don't shine. It's unlikely you'll ever need to access them again.

Note: As I'm a happy snapper, I also have near real-time backup of data from my smartphones to the primary FreeNAS server. I use a combination of Resilio Sync, Plex Camera upload and NextCloud instant upload to have photos and videos backed up in triplicate.

Thanks, interesting.

So this may keep a data set alive in the short term, but in the long term, this is relying on discs that still need to be refreshed every few years; which means all of these disks across three separate servers need to be refreshed. Doesn't seem like an archival method to me, more of an "uptime" approach because the hardware being used is merely redundant perhaps, but doesn't last. Without disc refreshing, all three servers are just subject to time after 3+ years, assuming no faults.

So I'm curious what would be a good backup solution to compliment the idea of replication (which is just physical copies separate from the primary live data in production).

Refreshing a lot of discs every few years, even if you never add data to it, doesn't seem like a good long term way to acrhive data as a backup. But I'm not sure, which is why I'm asking.

Very best,

 

[Basil Hendroff]

I must admit, archiving isn't on my radar at all. Maybe it should be. I like having my data available all the time, and this is at odds with archival solutions, which take your data offline completely.

Having said that, there is some data that I don't have online and those are desktop image backups. I don't see the value of having these online so I keep them on portable external HDDs. That's the closest thing I have to archival data.

I'm probably too lazy to go to the trouble of archiving, but if I was to consider it, personally, I'd do what I can to avoid slow, low-capacity media. What do you see as the issue of hiving off data to a high capacity, portable HDD, duplicating that for redundancy to a second drive, and then storing those drives offline?

 

[MalVeauX]

I'd do what I can to avoid slow, low-capacity media. What do you see as the issue of hiving off data to a high capacity, portable HDD, duplicating that for redundancy to a second drive, and then storing those drives offline?

This is a good point; I don't think any of us would want to deal with really low speeds when it comes to really high capacity. I think this is why I wonder why cloud options are viable for so many, I'm guessing it's because it's convenient but rebuilding a large data set from that even with a good internet connection is going to take a while. So I started wondering, why do cloud backup when you can do something like M-disc 100Gb optical backup that will certainly outlast a cloud option? It's another slow solution, it may cost more up front, but it won't disappear when the cloud host disappears and you don't have to worry about snooping or randomware on their end, etc.

The issue I have with an external portable hard disk is that it's mechanical and will fail eventually and so for archival purposes, this is not good. But to put this into context, I'm not talking about data that is not really going to ever be wanted anyways. Think more about how many of us have lots of family photos or videos these days. I have virtually no photos from my youth because we just didn't have digital cameras or anything. So most of my childhood photos are on a polaroid or kodak film and had to scan it to preserve it. So I'm thinking of all the important truly irreplaceable data sets, like that, for a family and then also documents that are sensitive and not something you'd want out there for snooping (think birth certificates, social security documents, all your legal stuff, etc). Putting these on an external is current, but if you leave that for 10 years in a fireproof/waterproof safe, will it spin up again? And will all the data be 100% validated? That's a big risk. You would need at least two physical copies to at least gamble that, if not three copies. I think for a living data set that is being actively used and worked on, hard drives with redundancy are fine. But for the long term run, archival approach, I don't think a hard disk approach will suffice. It's easy to think we will just refresh drives, buy more drives, etc, and maintain an active data set alive with no faults. But interruptions occur, life happens, time happens, and time is what I'm really interested in. It's what archival method will survive time when forgotten? So far, it seems like optical media, like M-disc, is the only affordable thing that could do this. But I'm still not completely convinced, hence the questions.

I think it's a very fair point to make concerning having data status play a big role in this. I'm not talking about massive archival approach of all your data, like literally all of it (massive amounts of TB of data over the course of a few years, etc). I think we do not need archival level hardware and media to handle living data sets that are important but not crucial to survive time. I think living and working data sets that are important and commonly accessed should be on redundant media, multiple physical copies, accessible with fast operation (like NAS pools, etc). Having an external hard disk that is a physical copy of that working platform that is a cold copy makes a lot of sense, but under the idea that it's a temporary backup, ie, maybe good for 2~3 years unplugged and unused, but the risk ramps up rapidly after 3 years sitting in cold storage even. So then that third option for physical backup comes up, do we go into off-site with more physical media like hard disks of our own, or via cloud (with all its risks) or do we explore things like optical for that 3rd option for that utmost critical stuff to have survive the test of time?

3, 2 & 1 with archival in the final stage:

I'm thinking of doing something like this, but I'm curious what others would do, or think of it as I'm not sure I'm aware of all potential options out there:

Primary data sets (all data):

Living on a redundant platform of media, such as mirrored hard drives accessible over network or directly. Example would be FreeNAS running on a server with fairly good hardware for error and fault tolerance (ECC RAM, ZFS file system, redundant mirrors). There is always two physical copies of the data for failure of a single drive. Re-build is very fast (mirror only). A pool would not however survive a controller fault or memory (ie, bad data written to both drives at the same time). I'm not sure what would survive that other than using ZFS to catch this in the first place. This would be the active system we're all using in someway right now.

Tertiary data sets (only important data that needs uptime):

Copied to a physical hard drive that is external from any system (both the primary clients, the server, etc) but accessible once attached to a machine, either via eSATA, internal SATA, USB3X, etc. This drive lives as a cold copy in cold storage, so never fired up commonly, other than to put data on it. This would be something done on a schedule, for someone who needs seriously redundant material, this may be often (daily?), but for someone else just building in another layer, this could be a monthly (?) or other occurrence. The idea would be that if for some reason the redundant media server went down or the data flipped bits somehow, you had this 2nd (or 3rd) physical copy available to correct with and would have rapid uptime. An alternative would be a 2nd redundant platform (2nd NAS server with pools with redundancy) instead of a single hard drive cold copy (this would be for uncompromising uptime for super important data). Otherwise, I think a physical hard drive with a 2nd (or 3rd) copy of the physical data sets that are important for uptime would cover this without too much cost (shuckable 8TB or higher capacity perhaps).

Archival data sets (only data that needs to survive time that you don't want to refresh every 3 years):

Instead of cloud storage or off-site hard discs, and since any solid state approach is currently not viable at all (flash/USB sticks) for long term, I think engraved M-disc (25~100GB discs) are an option and becoming affordable. This would not be to backup 8+ TB of data. This would be only to backup critical data for a focus on surviving time. So think of data that will matter still in 10 years or more, not data that will be completely irrelevant in 10 or more years. In my case, this would be irreplaceable photos, videos, documents and some software related to them. This medium will outlive hard discs and again no solid state approach is viable yet (so USB3X flash sticks simply will not survive 10+ year with certainty). No internet required to get this data. Will last your life time and then some. A drive capable of doing the read/write is about $65 new these days, the media is about $12 per 100Gb disc ($0.12/Gb) or $3.2 per 25Gb disc ($0.128/Gb) or $5.6 per 50Gb disc ($0.112/Gb). Since the average cost is so close, I would lean towards the 100Gb discs to save physical space over time. Writing would be done at slow speed with validation. Accessing one of these discs would not require some special knowledge of an operating system or file system (like ZFS) for a relative to recover in 20+ years if you were unable to. Your choice whether to encrypt or not. Will survive an economy collapse and if stored properly will survive most natural disasters short of a volcano.

But I'm curious, if there's something else available? I think it would be obvious if solid state was viable that it would be the way to go. The pure convenience of a flash stick would be great for this purpose and super easy to store and maintain and access. But this will not survive 10~20 years, would require refreshes, same as a hard drive, so might as well just use hard drives for this if going that direction. I can't seem to find a viable way to long term store data that is accessible to mortals other than optical media like m-disc tech.

Enterprise class SSD is another option, but cost per capacity is still high and ultimately needs refreshing too, but could be considered for more than 3 years, where hard discs need to be refreshed/replaced likely, but likely not something to consider over 10 years, so ultimately still requires replacement and refreshing to a new medium. I think that's viable. But the data isn't there yet in terms of long term storage. Short term, we know SSD is superior to HDD in every way, other than cost per capacity. But it's the long term numbers that matter for this final option, to last the test of time. There's plenty of research at the highest level of endorsement behind m-disc technology though, ie, engraving and life span.

Thoughts?

Very best,

 

[joeschmuck]

Got to tell you, I'm a very simple minded person when it comes to archiving data. I retain all my data on my NAS and I periodically write a few DVD's which store 4.7 or 8.5 GB of data. I write a set for photos and I write one for my financial data (typically a CD-R). Sometimes I will create a set for purchased software and drives, etc.... things I cannot easily download or do not want to go search for, but to be honest, those software packages I'm placing more and more on removable USB hard drives, along with a copy of my important data (photos and financial records) just for fast recovery should I need it, and yes I have needed it such as when I did a clean install of Win10 in January.

Restoring over the cloud is slow, restoring from a NAS is fine but then you need to maintain another NAS. So DVD media is my choice of long term and safe backups.

Now if concerned about your home being destroyed where you store your backups, I guess a backup in the cloud makes sense but I would only put the stuff you really need to keep there and automate the process so it's always up to date, remember that storage is money. AND ensure you know the account information and password to access it. It doesn't help you if the account information gets destroyed with your home.

 

[Constantin]

Archival disks like the M-series are great. Having data going back to 1988, I'll share the following thoughts:

1. You better have a reader for said disks handy - and the wires / interfaces needed to connect to them. For example, for fun I recently hooked up a 128MB Magneto-optical disk to my current MB Pro. Needed a long chain of components to make it happen. (it just worked, BTW, to my utter amazement, no drivers needed)
   • Thunderbolt 3 - Thunderbolt 2 adapter (Apple)
   • Thunderbolt 2 - Firewire 800 adapter (Apple)
   • Firewire 800 - FireWire 400 cable (OWC)
   • Firewire 400 to db 25 SCSI adapter (OrangeLink)
   • db25 SCSI to SCSI 50 pin flat adapter and from there into the DynaMO 128
2. You better have the programs needed to read the data (my old word files are not that functional).
   • Even PDFs, JPGs, and like "universal" standards may fall out of fashion, long-term.
   • Or, will you assemble a growing list of operating systems hosted on a VM that allow you to go back to older versions of Word, etc. when such access is needed?
   • Some of my older data is inaccessible thanks to me using encryption and losing the passwords. (see below)
3. Think carefully about access - for example, if this data is sensitive (records), then encryption may be necessary but then how to ensure you can decrypt in the future? Or will the stuff be stored in a lockbox somewhere, etc.? God forbid you die, how will your executor or your descendants be able to access said data, should it be necessary?

 

[MalVeauX]

Got to tell you, I'm a very simple minded person when it comes to archiving data. I retain all my data on my NAS and I periodically write a few DVD's which store 4.7 or 8.5 GB of data. I write a set for photos and I write one for my financial data (typically a CD-R). Sometimes I will create a set for purchased software and drives, etc.... things I cannot easily download or do not want to go search for, but to be honest, those software packages I'm placing more and more on removable USB hard drives, along with a copy of my important data (photos and financial records) just for fast recovery should I need it, and yes I have needed it such as when I did a clean install of Win10 in January.

Restoring over the cloud is slow, restoring from a NAS is fine but then you need to maintain another NAS. So DVD media is my choice of long term and safe backups.

Now if concerned about your home being destroyed where you store your backups, I guess a backup in the cloud makes sense but I would only put the stuff you really need to keep there and automate the process so it's always up to date, remember that storage is money. AND ensure you know the account information and password to access it. It doesn't help you if the account information gets destroyed with your home.

Thanks, good points. So you're using a NAS and optical as archival with some USB flash sticks in-between basically, for a 3 physical copy approach with living data, tertiary (on flash) data, and archival class medium (DVD-R, 25~100 year depending on conditions) for the 3rd set. I like this. I would think I would really like (1) NAS with redundancy, (2) second NAS with redundancy and (3) optical medium (M-disc, 100Gb) for long term archival and life time hording. But that second server is costly considering it's just a backup to a redundant system.

How do you store your DVD media for long term?

Very best,

 

[Constantin]

One more thing: You also have to verify that the reader is functional from time to time. The DGR DynaMO 128MB MO Drive is built like a tank and likely will outlast me. A later, similar drive sourced from Pinnacle Micro (4.6GB MO), treats me to the click / clunk of death when I try to use it for extended transfers.

Padded sleeves for the media in a cool, dry environment seem like a pretty good bet. M-disks also appear to have better resistance to fading unlike DVD media. Within DVD media, there are format differences that affect longevity (DVD-R vs. +R) as well as the layer to be engraved (dye or gold). Taiyo Yuden disks were highly regarded but you had to watch out or you could get bit by a counterfeit.

 

[MalVeauX]

Archival disks like the M-series are great. Having data going back to 1988, I'll share the following thoughts:

1. You better have a reader for said disks handy - and the wires / interfaces needed to connect to them. For example, for fun I recently hooked up a 128MB Magneto-optical disk to my current MB Pro. Needed a long chain of components to make it happen. (it just worked, BTW, to my utter amazement, no drivers needed)
   • Thunderbolt 3 - Thunderbolt 2 adapter (Apple)
   • Thunderbolt 2 - Firewire 800 adapter (Apple)
   • Firewire 800 - FireWire 400 cable (OWC)
   • Firewire 400 to db 25 SCSI adapter (OrangeLink)
   • db25 SCSI to SCSI 50 pin flat adapter and from there into the DynaMO 128
2. You better have the programs needed to read the data (my old word files are not that functional).
   • Even PDFs, JPGs, and like "universal" standards may fall out of fashion, long-term.
   • Or, will you assemble a growing list of operating systems hosted on a VM that allow you to go back to older versions of Word, etc. when such access is needed?
   • Some of my older data is inaccessible thanks to me using encryption and losing the passwords. (see below)
3. Think carefully about access - for example, if this data is sensitive (records), then encryption may be necessary but then how to ensure you can decrypt in the future? Or will the stuff be stored in a lockbox somewhere, etc.? God forbid you die, how will your executor or your descendants be able to access said data, should it be necessary?

Super interesting, thanks! That's very impressive to dig up 1988 data. I don't think I have anythings surviving honestly from the 90's even at this point as I'm pretty sure I chucked any floppy or later CD/DVD that I had at the time after migrating their contents to hard drives and then migrating that data over time to newer drives sequentially over time. I do actually still have some CD's from the 90's and lots of DVD's from the late 90's, but I haven't accessed them in 20 years or more at this point; but it's nice to know that over 20 years later, I can read them in a common optical drive still to this day). But I would certainly like to know I have specific data (like family photos, videos, documents, legal stuff, financial stuff, etc) on a medium that can be recovered easily in 20~30 years, or more even. That's what is peaking my interest in this concept recently, since we can all build fancy NAS systems with redundancy with enterprise class SSD, but ultimately, if you were injured or anything happened, could just anyone recover this data in 5 years? 10 years? That's where I'm starting to think. Part of getting older I suppose.

Great point about having a reader handy (and to test it for working status). While we have no way to know what will be available in the future, we can only see what the trends are so far in the past. But very good point about having the hardware and software handy to deal with the medium chosen for archival. This is why I started thinking that having a complex operating system and hardware setup (such as ZFS and networking with pools) would be a very difficult thing to deal with by someone who is not knowledgeable about such stuff, finding it 10 years from now, and trying to recover anything from it, not even knowing what file system it is, and how to even read the discs if anything can even be plugged into a SATA port in 10 years or more from now. This is why I'm really thinking of hard drive, SSD, etc, all being not suitable for long term survival of irreplaceable data. I think optical media may be the only suitable option for long term.

So far, optical has not disappeared. It's rarely the media of choice for portable data now, internet solved that for most applications. But optical is still a heavily invested in part of archival hardware for governments, major corps, university, etc. The concept of physical media still being critical in a time when ransomware and hacking are real threats when connected to the internet. Cold copies of data on physical media will be needed even more so now than ever. So I'm not thinking optical is going anywhere any time soon, other than improving.

Also a great point about containers and software; using various formats can make it hard to read something 10 years later. I do have VM's for various operating systems with various software ready to go for nearly anything. But I'm thinking more about what would happen in 20 years, if I stroke out, could a family member deal with my data and recover important things? So I don't want to rely on someone knowing virtualization, etc. I think it would be more appropriate to expect future generation to be able to understand a JPG or PDF even 20 years from now. Some containers have truly lasted the test of time. And you cannot beat plaintext when it comes to true forever compatible for important documentation or information without formatting. Encryption would be a personal choice. I think if stored off-site, or online in a cloud, I would favor encryption. But for personal storage of optical media? I would probably opt to not bother with encryption as physical theft would be required to generate a ransom (not that my information is worth anything, its just personal). Plus I don't want a family member in 20 years trying to deal with encrypted data on any media.

So truly thinking about 20+ years, a surviving family member inheriting my stuff, which includes hardware/media, and wanting them to be able to recover irreplaceable data in the form of family photos, videos, documents, etc, I'm thinking:

• Optical media without encryption (physically will last 100+ years if stored properly; accessible physically)
• Data in common containers that are already 10+ years validated (TIF, JPG, PDF, DOC, TXT, AVI, etc)
• Hardware (readers; power connectors; data cables); such as an external m-disc BDR reader via USB (refresh the hardware as available if new connections become the norm as long as they are still capable of reading the optical media)
• Store media in fireproof, waterproof safe with low humidity and stable temperature

I think it's much more likely for a family member with zero expected know-how to deal with a m-disc bdr optical disc with JPGs and PDFs in 20 years and to be able to figure out the hardware or find someone who knows this stuff, compared to asking the same of someone with a bunch of ZFS based pools and Linux distros and VMs to recover said data, assuming you can find hardware with SATA ports, etc.

Again, so far, I have some CD's and DVD's from the late 90's surviving and readable on common hardware today without any special knowledge of how to access the data of file systems. That's incredible if you consider 20+ year old media and data standards. So I think it's arguable that optical will not drift too much in the next 20+ years, and if nothing, the media is better than it was (m-disc for example).

Very best,

 

[joeschmuck]

How do you store your DVD media for long term?

I have a black DVD brief case that hold about 50 CD/DVD pieces of media. It can also hold a few flash drives and a USB Hard Drive.

with some USB flash sticks in-between

No, I use a USB Hard Drive. While I do have a copy of my financial data on a USB Flash Drive, it is not my archival backup.

As others have pointed out, you need to ensure you are using technology that will be operational in the future. I'm not sure how popular M-disc is, I've heard of it but never seen it used in my company nor used at military sites I've worked at, and it's not something I see people talking about. Now DVD's are popular. But whatever format you choose, you need to ensure you have a device to restore it with. That also means having device drivers (software) able to work that hardware for whatever computer you connect it to.

And to this point, a story... Way back in my younger days I invested in an archival storage medium called ZIP DISK. 100MB of cold hard storage, huge capacity all in a Floppy Disk footprint but about 1/4" thick. The drive could be internal IDE or external USB. Well guess what, that hardware has been forgotten because the capacity doesn't touch a USB Flash Drive these days. Zip Disk was good for it's time but time past it by.

 

[Constantin]

With enough time, I may start burning archival, unencrypted copies of family pics to M-disk for off-site storage. I have the disks, the writers, but currently lack the time. It is a good idea though.

One thing my recently-deceased dad brought up repeatedly is how little your kids might be interested in wading through miles of archives. They may just want a top 50 pics of their childhoods, full stop. He also felt that tagging pictures with the names in the metadata or sorted by name in folders was also super important. Don't expect kids, descendants or researchers to remember the name of your friends 30 years from now.

That in turn lands me on the software issue. He used Lightroom and the tags were embedded in the Lightroom archive. I have a perpetual license of lightroom here and plan on installing a Mojave VM to supplement the the Snow Leopard one I already have. The Snow Leopard VM will allow me to go back to the PowerPC days while the Mojave VM will allow me to use 32-bit applications.

 

[MalVeauX]

I have a black DVD brief case that hold about 50 CD/DVD pieces of media. It can also hold a few flash drives and a USB Hard Drive.

No, I use a USB Hard Drive. While I do have a copy of my financial data on a USB Flash Drive, it is not my archival backup.

As others have pointed out, you need to ensure you are using technology that will be operational in the future. I'm not sure how popular M-disc is, I've heard of it but never seen it used in my company nor used at military sites I've worked at, and it's not something I see people talking about. Now DVD's are popular. But whatever format you choose, you need to ensure you have a device to restore it with. That also means having device drivers (software) able to work that hardware for whatever computer you connect it to.

And to this point, a story... Way back in my younger days I invested in an archival storage medium called ZIP DISK. 100MB of cold hard storage, huge capacity all in a Floppy Disk footprint but about 1/4" thick. The drive could be internal IDE or external USB. Well guess what, that hardware has been forgotten because the capacity doesn't touch a USB Flash Drive these days. Zip Disk was good for it's time but time past it by.

Thanks; so you have your archival data in a brief case? Do you store it somewhere safe that can survive a fire or flood or any other potential but real disaster (baring volcano or asteroid level stuff)? I realize a lot of folk keep things at home, or when they do keep something off-site, it may increase redundancy but it increases anxiety to theft or tampering perhaps. I would think a sufficient safe box would survive the typically possible events, like fire or water damage from real disasters to avoid needing true off-site redundancy (though this is still a possibility with a long term storage medium like optical as it will not take up much space and isn't connected to the web, etc).

M-disc is just the medium itself that is written to, same fashion as typical optical medium but a little different; when inserted into any typical optical reader, it will read as any other media. The difference is simply that it is engraved and the material itself. However, for the sake of argument, even if we do not explore m-disc long term, even archival class DVD-R and BD-R optical discs will survive 25~100 years when stored properly. So either way, we're just looking at optical mediums stored well.

I too used to use Zip Drives in the 90's. I used to plug them in via serial port (the same people used for printers at the time). The 100Mb discs were pretty good and the speeds were quite decent. I migrated a lot of my data back then on zip discs and they lasted a long time, never lost data or had failure. But then again, after that point, hard disc capacity went up rapidly later on and optical storage became affordable (CD-R) so the zip discs became obsolete at that point when I could use a CD-R to get 5+ times the capacity (and these CD's I still have, and they work!). The data on the hard drives simply migrated over the years to newer hard drives with some of it on optical CD-R and DVD-R between major moves over the last 20 years. I still have it all and its accessible with common hardware. Regardless of those CD-R's being made with an old IDE optical drive, today's optical drives read it and the data on it is still common container and file systems.

That said, while I have some flash sticks still alive, I wouldn't think they would be a good gamble for another 20 years, they way optical media is a good gamble for 20+ years.

Very best,

 

[MalVeauX]

With enough time, I may start burning archival, unencrypted copies of family pics to M-disk for off-site storage. I have the disks, the writers, but currently lack the time. It is a good idea though.

One thing my recently-deceased dad brought up repeatedly is how little your kids might be interested in wading through miles of archives. They may just want a top 50 pics of their childhoods, full stop. He also felt that tagging pictures with the names in the metadata or sorted by name in folders was also super important. Don't expect kids, descendants or researchers to remember the name of your friends 30 years from now.

That in turn lands me on the software issue. He used Lightroom and the tags were embedded in the Lightroom archive. I have a perpetual license of lightroom here and plan on installing a Mojave VM to supplement the the Snow Leopard one I already have. The Snow Leopard VM will allow me to go back to the PowerPC days while the Mojave VM will allow me to use 32-bit applications.

That is an excellent point. However, I would argue that context matters. Previous generations did not look for photographs of family with today's veracity and access to such media. But, ask someone born into today's tech, even a child, for images and they can dig up thousands to show you anything and anyone. While in the past we may not have been so interested, we are today I think as a people far more interested since the technology is available. People are literally drawn to the idea of getting information on their family and even just random people in general and will harvest and ingest any images they can of any of them. Social media, for example, is a glaring example of our current habits regarding picture consumption and wanting to look back into time on someone. I can't do that on my childhood photos or my past family's other than digital scans I made. But someone born into today's decades, well, they were born with internet and cameras in their hands at all times with the ability to lightspeed share them.

But the point is still valid; we could put effort into preservation and our descendants could literally throw it all in the trash. Mean while, google and others will archive our images for all eternity because they simply can.

I think its a great point to have a "top 50" or something that shows simple good representations of family members with plain text documents for records of who they are, etc. Even if my kids don't care. My kid's kid's might be into it. Who knows?

I won't be using cataloging software for that; I keep it simple, file name and folder structures in common file systems. No software needed that is specific for catalog or metadata. Habits from my post-DOS days when we were allowed long file names finally.

Very best,

 

[joeschmuck]

hanks; so you have your archival data in a brief case? Do you store it somewhere safe that can survive a fire or flood or any other potential but real disaster (baring volcano or asteroid level stuff)?

Nope, I use to have a cloud archive and might set up another one. Having just moved I could start taking that archive over to a family members house, that makes the most sense. I do have a fire safe but I"m not sure I'd trust plastic media in it, in spite of it's claims.

my recently-deceased dad

Very sorry to hear about that. I've had 3 fathers pass since March this year for people that work for me. One of the fathers was well known by the entire team. None were Covid-19 victims. He sounded like a smart man. I try to sort mine by date. And I like the idea of a Top 50 list of family photos. The way some people take photos, they could delete 99% from their smart phone and save the 1% that has value.

 

[MalVeauX]

Nope, I use to have a cloud archive and might set up another one. Having just moved I could start taking that archive over to a family members house, that makes the most sense. I do have a fire safe but I"m not sure I'd trust plastic media in it, in spite of it's claims.

And I like the idea of a Top 50 list of family photos. The way some people take photos, they could delete 99% from their smart phone and save the 1% that has value.

Realistically, I agree, I wouldn't trust most mediums at all in a fireproof safe against true fire, they're commonly only rated for 30 minutes at a certain temperature (they're just insulated boxes really) and then anything inside is likely to melt or combust potentially depending on the temperature that eventually will rise in there. They're really just meant to survive a flash fire, not a full on melt-down on a house burned to the ground over the course of a few hours. Little will survive that that isn't extremely expensive and extremely large & heavy. But again having no protection what so ever against flash fire that isn't sustained and water damage or flooding is not an attractive alternative to having at least some form of protection against the unknown.

It's true, most photos from today's cellphones could be reduced to 1% or less maybe of their total images that are purposeful and worth keeping in the long run. That said, if the photos are in compressed containers like JPG, the reality is we can store a crazy large amount of them on optical or any other media without much effort and also, images are small enough that cloud options are indeed a realistic option for them especially across multiple platforms (examples being things like Flickr, SmugMug, Google, Amazon, etc). For pennies we can store a ton of images across multiple networks if we really wanted it to last and their hardware and uptime solutions are way better than we can implement in our homes. But I wouldn't want to put all my data on there in large capacity, nor my personal documentation stuff that doesn't need to be floating out there, even if encrypted.

++++++++++++++++++
++++++++++++++++++

Just on another tangent in the form of comparing optical media, I found a really interesting real world test by someone who archives lots of images and media, such as microscopy, where images are vital over time and archival images are incredibly important. So they often explore long term solutions so that someone can get to this information if needed one day, since microscopic information is truly critical for now and the future.

This fellow did a pretty real world test on a standard BD-R vs a M-Disc BD-R. The results are pretty interesting.

Review of MDisc vs BDR exposed to the environment for months (real world).

Very best,

 

[joeschmuck]

http://www.microscopy-uk.org.uk/mag...-uk.org.uk/mag/artsep16/mol-mdisc-review.html

The link does not work, you had too much stuff in there so I fixed it in your posting. I will give it a read.

Edit: Good read, impressive.

 

[MalVeauX]

The link does not work, you had too much stuff in there so I fixed it in your posting. I will give it a read.

Doh, sorry, I'll try to correct it; edit, never mind, you did that, thanks!

While it's by no means a mass trial with repeated results, it's a pretty solid test to warrant gambling perhaps on the m-disc medium over a standard bluray bd-r disc. In a pristine environment, they likely will do the same job. But exposed at all to anything ugly, maybe the m-disc has an advantage?

Both M-Disc BDR and standard BDR would be handled by the same optical drive, so that's a wash in terms of comparing cost.
Above, M-Disc BDR was about $0.12/Gb.
And just looking at a spindle of typical BDR, is about $0.03/Gb to $0.04/Gb or so.
So about 3 to 4 times the cost per Gb to use M-Disc BDR of any capacity.
So the next question becomes, is it worth spending 3~4 times the amount on capacity to get the durability of the M-disc? Especially in the context of the discs living in a climate controlled space, low humidity, moderate temperatures, in a fireproof/waterproof safe (common type, so really just good against flash fire and water damage from a storm, nothing catastrophic like volcano or lost into a sink hole under Florida).

Very best,

 

[Constantin]

I’d go m disk simply because the small delta in net cost. The much bigger cost is time. With m disk, you’ll only have to do it once.

 

[MalVeauX]

I’d go m disk simply because the small delta in net cost. The much bigger cost is time. With m disk, you’ll only have to do it once.

Agreed. I think if the data size was so significant that it would matter, then maybe it would matter for cost. But for smaller data, being specific about what needs to be preserved for the long run, it could be better spent on the M-disc for the survival of the medium itself allowing the survival of the data, all external things set aside (like accessing it, modern hardware to access it, software, file system, containers, etc in the future).

Out of curiosity, do you happen to have a M-disc with a JPG on it somewhere and a typical BluRay player (not in a PC, but something meant to just play videos possibly or a console that uses them)? Basically something that was never intended to read an M-disc. I'm curious if some legacy hardware bluray players will read a basic M-disc BD-R and see a JPG and be able to display it. That would be super interesting.

I have a M-Disc BD-R and some 100Gb M-Disc BDXL's on the way, otherwise, I'd test it myself now.

Very best,