Autoimport broke timestamps on 1TB of photos - Backup now Un-Syncable

[freshfeesh]

[This is a repost of this, which I just realized is probably in the wrong forum]

About three weeks ago I re-imaged my compact flash Freenas boot drive to eliminate the possibility that some user configuration error was causing some network problems that I had already sunk a lot of time into. Happily, the error persisted, eliminating the user as the source of the network error. I'm writing now about a different error:

After reimaging, I autoimported the RaidZ1 array of (3) 3TB disks per plan. Everything was fine until I tried to run a sync, and every single file on Freenas was coming up with a "Date Modified" of 3 weeks ago, equal to the time I copied the files over to the array, rather than what should be, on average, about a year and a half ago. The sync software wanted to overwrite all files on other machines with the "newer" freenas files. I did the reimaging pretty soon after I first put the files onto the array, and I can't remember if I synced at all before I reimaged, i.e. I can't be sure that this problem has been caused by the autoimport, or at the time that I originally placed the files on the drive. However, when I place files onto the array now, date modified remains correct and unchanged. All other metadata on the files looks unharmed. Jpg's are by far the most common file type, but all file types are represented, and all exhibit the problem. These are CIFS shares shared with Windows machines only.

The more I think about this, the more I realize that this is a serious issue. I know that the latest versions of all my files still reside on other machines, metadata in-tact, so I can recreate the backup from scratch if necessary, this time. Any further down the line and I wouldn't have that luxury. My freenas server will be the center of a multi-machine setup. The location of the latest version of a file will arbitrarily be on either the freenas machine or one of the other machines, and the whole thing will depend on syncing to work. Were I to need to autoimport the array again, there would be no way to manually examine all the files from the various sources against the ones on the server. I would have to summarily discard one or the other.

Did I do something wrong? Is this a known failing? Is there a fix?

 

[cyberjock]

What sync software are you talking about? LOL

 

[freshfeesh]

Not sure why the LOL, but it's Beyond Compare at this point for the manual work. The error shows up there, in Win Explorer, and in files that I copy from the server to the desktop.

 

[StephenFry]

I've had some file time/date issues in the past -not at all related to FreeNAS/ZFS- and after a lot of searching, found a Windows utility that can take the EXIF date+time and apply that to the file's attribute data.

Can't for the love of all that is Google find it at this moment, but thought you might like to know it does, in fact, exist. Can't help you out with a proper idea to fix this, though, other than restore a backup or snapshot.

 

[freshfeesh]

noobsauce80, sorry if you were laughing with, me not at me. I guess I did try to have a laugh at the beginning of that post, but was not too happy by the end of it.

StephenFry, thanks for the tip. It is good to know that that exists, but given the other file types that are on the volume, I'd have to just recreate the whole thing. Do you remember if your issue happened on discrete files, or an entire file system?

 

[cyberjock]

I was laughing at you, but it wasn't in a mean way. I was laughing because you were obviously frustrated but forgot a very small but important piece of data :)

I haven't heard of Beyond Compare so I can't help much. But I'm definitely going to go read about it because I'd like to be familiar with some kind of sync program. Never know when you might need one.

 

[StephenFry]

Do you remember if your issue happened on discrete files, or an entire file system?

Neither, it happened on a system, not a file system per se.

More than once, I've had to correct time/date stamps because a system was set to a wrong day/month/year/timezone.

 

[freshfeesh]

I haven't heard of Beyond Compare so I can't help much..

Well, Beyond Compare discovered the problem, it didn't cause the problem. It lookes like the files were altered by the autoimport, although there is a possibility that they were altered by the ftp program (Filezilla) that I used to populate the array in the first place. If the autoimport did it, then this fault is of concern to anyone who may ever autoimport a volume...

 

[cyberjock]

How did you come to this conclusion?

On the surface I'm really not buying that auto-importing caused your issue. I KNOW that FTPing files changes the times. I had this happen to me in 2008 when I copied 2TB of data over FTP because it was faster than SMB.

 

[warri]

Yeah, FTPing doesn't seem carry over the original file's metadata (date, etc). Check if your backup program maybe can identify files based on their names, size and/or hash. If not, maybe think about switching to another one. I recently used Areca Backup, seems to do the job quite good and is free as well.

 

[freshfeesh]

I KNOW that FTPing files changes the times. I had this happen to me in 2008 when I copied 2TB of data over FTP because it was faster than SMB.

I can't speak with 100% certainty yet. As I've said, I can't rule out the ftp transfer, but until what you just said I was sure it had to be autoimport because ftp is old, mature tech and autoimport is a newish function to a newish front end that's still getting it's legs. I've seen a number of issues discussed on the forum get traced to GUI front end issues.

Thank you very much for reporting your similar experience with FTP. I added the files to the array with one 1.4TB transfer and one 6GB one. I want FTP for the same reason you used it in 2008, but I would actually much prefer that to be the problem and be able to have confidence in the array. I'm surprised I've never heard of the issue before. I'll try to replicate my problem and report back.

 

[freshfeesh]

Yeah, FTPing doesn't seem carry over the original file's metadata (date, etc)

Whoa, ever? How come they don't teach you this stuff in school. Not having the (file system) time metadata, particularly on pictures, is a non-starter for me.

 

[freshfeesh]

Yep, FTP'ing did it. Much obliged, gentlemen. Definitely not the resolution I was expecting, although much easier.

I can't believe I didn't know about this. When will this n00bness be over.

 

[Stephens]

December 21, 2012. It won't be long now.

Here's a thread discussing the issue. I'd actually forgotten about it myself as I dealt with it a long time ago and used FlashFXP combined with a server that supports setting date/time on file transfers. I recently did it with FreeNAS and Mozilla and... not so fun. I don't know what FreeNAS would do in combination with FlashFXP, but you might want to test it if you care.

 

[paleoN]

Beyond Compare has a FTP portion and supports setting date/time on compliant FTP servers.