Entire LIVE environment is down! DNS problems? help!

Status
Not open for further replies.

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
I cut my arm off...

I have 3 esxi hosts all pointing to an ISCSI extent on my FreeNAS box. They use DNS for the ISCSI target. They have my primary domain controller 10.10.1.108 as the DNS server. My problem is that 10.10.1.108 is a VM on the datastore. I brought my esxi hosts and my freenas box down today. When I brought everything back up, my esxi hosts do not see the datastore. I checked in freenas and I can still see my extent. (this summer i accidentally deleted it and lost all of my data once already).

Can anyone help me please make sure my data is still in tact, and help me reconnect to my datastores!

I will be here clicking refresh to answer questions.

And if anyone is willing to do a webex or something similar, let me know. I am getting desperate and I think because I messed this up once before I nervous about clicking anything until I know I haven't lost it all. Maybe I can throw a gift card or something your way for the help!
 

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
The only change I have made was on FreeNAS. When I brought it back up, it said it's IP address of 10.10.1.85 had "no carrier" but the second link 10.10.1.87, had a link. So after scrambling for 20 minutes or so, I deleted nic(zero) (which was 10.10.1.85) and made nic(1) that IP address because that is the IP all of my vmware hosts look for for the datastore.

So:
FreeNAS: 10.10.1.85
ESXi1:10.10.1.75
ESXi2:10.10.1.77
ESXi3:10.10.1.79

I will post ISCSI info soon.
 

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
ISCSI Target Base Name: iqn.2011-03.example.org.istgt
Discovery Auth: none

Portals:
1: IP Addr: 10.10.1.85
Port: 3260
--I also noticed 3 additional lines under the portal config. In one, the IP address: Box says "-----" with port 3260. The next two say IP Address: 0.0.0.0 port 3260.
Should I delete these blank entries for the portal?

Extent:
Name: extent1 Type: File Path to the extent: /mnt/Vol1/extent1 Size:5TB

Initiators:
Group ID: 1 Initiators: ALL Authorized Network: ALL

Target:
Target name: target1 Target Alias: ISCSI target 1 Target Flags: read-write Portal Group ID:1 Auth Method: Auto

Target1/Extent1


Should I redo my portal and target? I just don't want to mess with this too much until I get the all clear on my extent. Thanks for your help folks!
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
I hate to be a buzzkill, but this is probably a complex problem that requires intimate knowledge of your setup. I don't know how much help you are going to get on the forums.


Quite literally though, if your extent exists and you haven't deleted it and recreated an empty extent, then all your data is there. It's just a matter of setting up hard info (static IPs, etc.) to make your networking work enough to bootup the system. Of course, this is almost certainly going to break a bunch of crap once you get back up because your DHCP and/or DNS servers are going to be upset that you did stuff "behind it's back".

This is why basic services like DNS and DHCP should never be VMs. :P
 

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
I have the unit up and can access the web interface. I just need to make sure it is ok for me to delete my portal config, target config, and then make new ones and link to my extent. Do you see any harm in doing this? Good to know about DNS after the fact...eh live and learn I guess.

**Edit** Yes I can still see my extent and it's available space and usage, so all my data looks safe.
 

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
OK I got DNS working and setup on an old DNS server (physical) and I can now ping the FreeNAS and my ESXi hosts. It's just the fact that my hosts do not see the ISCSI via FreeNAS. Here is the message I am getting in vSphere:

esxi_ISCSIerror.jpg
 

Jaknell1011

Dabbler
Joined
Aug 28, 2013
Messages
49
Okay I rebuilt the target and had to remove a few screwy lines in the portal configuration, but I am now back up and running. What should have been a 30 minute job has now officially taken from 10:30AM to 6:30 PM. Oh well it's done and we are working again. Sorry for the chaotic posts.

For the record, and for future googlers out there, if your ESXi hosts cannot see your ISCSI target after a reboot, check your portal and target configurations. In my case I had to remove the target out of the ESXi storage adapter configuration and add it back in. Also, it is safe to edit the portal and target, JUST DON'T TOUCH THE EXTENT!

Hopefully someone will run into this in the future and get a glimmer of hope.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
To add to the lesson learned, if your esx hosts have a local storage based datastore, create a DNS vm on a couple of them. I keep an AD server on each of two local datastores and one on my iscsi datastore.
 
Status
Not open for further replies.
Top