Need assistance rebuilding FreeNAS/dataset configuration

[jason.rohm]

Good evening.

I've once again backed myself into a technical corner and need help from those more expert than I to find my way out.

Today I attempted to do a BIOS/firmware upgrade on my LSI 9305-16i from the FreeNAS OS itself (9.10.1-U4). I started to do the BIOS upgrade and a kernel panic occurred. On reboot, the kernel panic reoccurs with similar output when importing the main volume.

I booted to DOS and did the firmware and BIOS upgrade again with no error. However, the system still craps out each time it boots. I attempted a FreeNAS "upgrade" over the top to ensure that there wasn't something wrong with the OS, but same result.

I plugged in a new boot media (USB thumb) and did a fresh install. I am able to mount the Volume but don't know what I need to do to get my dataset and iSCSI extent back. I'm afraid to just hammer away on it for fear of losing data. It wasn't clear to me if I could just create a new dataset and extent or if that would overwrite what is there.

This unit is backing my lab VMware environment and holds a ton of stuff that I really don't want to lose or have to recreate from scratch. I THINK the data is all there, there is just something farked up with the FreeNAS configuration, so I need to import/recreate in a nondestructive way.

Thanks all.

 

[snaptec]

In a good world you have a Backup from your config, do you?

If not Import the pool through the gui and reconfigure iscsi nfs and so on


Gesendet von iPhone mit Tapatalk

 

[jason.rohm]

In a good world I wouldn't have even tried this to begin with.

I can recreate the configuration. I just want to be sure that recreating the iscsi isn't going to overwrite my existing data.

In a good world you have a Backup from your config, do you?

If not Import the pool through the gui and reconfigure iscsi nfs and so on


Gesendet von iPhone mit Tapatalk

 

[jason.rohm]

I've done a fresh install and attempted a GUI import of the volume. The output is modestly different (2 minutes of garbage scrolling by too fast to read) but the outcome is the same. The system panics and reboots.

When I import the volume from the CLI, it joins and mounts up just fine. Is there some sort of metadata that is FreeNAS specific that might be messed up? Is there a way to recover/reset?

 

[jason.rohm]

Additional information: I can only mount up the pool in readonly mode. Mounting it without the readonly flag results in the crashes. However, I CAN mount it up on the original boot media if I unplug the disks, forcing it not to mount at boot time. This allowed me to download my configuration, but I don't know how much good that will do if I can't get the data off of the pool.

 

[jason.rohm]

Now that I have the original boot media in (and hence the original configuration), I actually see my VMware hosts connecting to the iSCSI LUN, but VMware shows the datastore offline yet.

I think this just became a VMware question, but how do I get my hosts to mount up a VMFS5 volume in readonly? If I can read the system, I can copy everything over to my other NAS, wipe this one, and start over.

 

[jason.rohm]

Wasn't able to get or find a way to get VMware to mount read only. However, I am able to mount the LUN from my Windows box and use Diskinternals VMFS recovery tool to read the data. Unfortunately, that tool is $699 to unlock, so I'm going to run out other options first.

I have a zdb running against the pool right now. With any luck, that will perform some magic.

 

[Ericloewe]

Were you using the pool during the upgrade procedure? That's a genuine Very Bad Idea (tm).

Your best bet is to copy the data over to a new pool. As for avoiding this in the future, if you have any doubts about what might be using the pool, use sas2flash (or in this case sas3flash) only from the EFI shell.

 

[jason.rohm]

Is there any way to know how long zdb is going to run or what it is doing? I started it with multiple -v in an attempt to get some sort of feedback, but nada so far. Been running about 12 hours.

My zpool is about 40TB with about 4TB actually in use.

 

[jason.rohm]

[QUOTE="Ericloewe, post: ... ? That's a genuine Very Bad Idea (tm). [/QUOTE]

Yeah. I knew that going in but did it anyway. This is what I get for clicking "I'm feeling lucky".

 

[joeschmuck]

Since you stated that you had access to your data earlier, did you ever copy it off? Waiting for zdb to complete might be a waste of time. A rebuild might save you a ton of time.

 

[jason.rohm]

Just an update. The zdb has been running for better than a month now. About a week in, I started getting some output. The data increments fairly regularly but it doesn't really mean anything to me. There steady but light disk access.

At some point I need to kill this off and recover the hardware for other purposes. Everything that I had on other media (backups) has been restored, but it would be really nice to get all my snapshots, clones, and other "not backup worthy" stuff back.

Is there any reasonable way to estimate when/if this will actually finish? Is there any value in stopping now to see if things are working or will that waste the month's "effort"?