iscsi volumes turned raw

[Starpulkka]

I dont know a lot about iscsi, but when i was intrested in it i tested it on freenas 8 and windows. I mounted freenas iscsi ntfs partition (all formatted and space allocated) in windows and then started copying files to it and then i pressed reset in windows machine. When windows restarted and i again mounted iscsi ntfs partition it was in RAW mode. Boom a first try i got a fail =)
And that was end of my iscsi experience.

 

[Lorenzo Vittorio]

@jgreco

But I tried to connect via a new connection on a different windows - same result; in this case I can't see the serial number being an issue

 

[HoneyBadger]

Over the years, we've had some users show up with similar issues, and I have actually seen this firsthand as well. It appeared to me at the time to be likely related to the serial number (etc) changing and VMware didn't seem to like that at all.

Saw this on an EMC box (iSCSI device S/N changing) and VMware had to resignature the volume. Not a fun time.

That behaviour doesn't seem to be limited to FreeNAS. Seen it with other stuff like EMC LifeLine as well.

Add QNAP, Thecus, Drobo, and Dell EQL boxes to that list.

Snapshots may not be useful in this case, because it is the client that is being obstinate, and also the client that is managing the filesystem layer.

You can sometimes fix it in-client via CHKDSK and CONVERT (the latter to force it to reexamine the volume and decide "hey, this is already NTFS") but having a "known good" state from a snapshot is helpful as well.

 

[jgreco]

having a "known good" state from a snapshot is helpful as well.

I'm curious how... the snapshot would seem to suffer the same fate as the live disk image. I've probably missed something obvious, though?

 

[mav@]

Snapshot taken while everything was fine allows to limit recovery just to finding proper target/initiator configuration with unlimited number of attempts. Without snapshot there is no way to be sure that data are still there and not corrupted even more by some recovery attempt.

 

[HoneyBadger]

A snapshot should at least be consistent at the ZFS level, which gives the NTFS journal a better shot at not being hosed and being able to cleanly recreate the filesystem. But yeah, they won't be quiesced snapshots. It's just improving the odds.

I should try deliberately breaking some of my Hyper-V CSV stuff to see if I can replicate this.

Snapshot taken while everything was fine allows to limit recovery just to finding proper target/initiator configuration with unlimited number of attempts. Without snapshot there is no way to be sure that data are still there and not corrupted even more by some recovery attempt.

This as well. If you have a pre-crash snapshot you're at least reasonably confident that state is OK and has nothing to do with whatever caused it to show as RAW in the first place.

 

[jgreco]

So it doesn't offer anything to the disk visibility problem that we're discussing.

 

[HoneyBadger]

In this situation the LUNs/disks are visible just fine, it's NTFS that's having issues.

Having a snapshot prior to crash means you have a point in time where the client filesystem has a much better chance of being in a non-screwed state.

 

[cyberjock]

To add a little to what jgreco said about "things we have seen" I've seen lots of people claim it was a bad USB drive and that replacing it got them "back up". Then a few days later they find out it wasn't the USB stick that was bad. The USB stick was a symptom of some bigger problem, bad RAM, bad PSU, etc.

So, assuming that everything you are saying is true and you haven't done things that many of us would consider "ludicrous" or "just plain stupid", I'm wondering if something else is wrong with the machine besides the USB stick and you just don't know it yet.

You're the second person to have an iSCSI device go raw, but the first was clearly determined to be hardware related later. Not saying that you should assume this is hardware-related, but I tend to think that the problem was NOT the USB stick. Simply reinstalling to a new USB stick and doing a config restore should have been all you needed to do to be back up, 100%. I've done it, many times, even once this week, and I can vouch that it does work that way. As for why it didn't work for you, the only thing I can say is to dig deeper into what is going on. I'm betting something else is wrong and you just haven't figured out what it is yet.

 

[jgreco]

Yes, definitely, if you have a reloaded config, that takes care of the serial number (etc) issues and it should just magically work. Definitely pull the system and go through hardware burnin again (he says, guessing that there's no "again" in reality).