Domain name resolution issue

[Jailer]

Ok you all are a smart bunch so I have a question for you. What direction do I point my ISP in to figure out a domain resolution issue I'm having? My ISP is a local operation and I'm on a wireless connection since I live in a rural area and that's my only option. When it's working it works pretty good for what it is. When it's not it sucks.

Specifically about a month ago I started having issues with my internet connection. When it starts acting up the issue seems to be terrible latency and things slow down to a crawl. The bigger issue though is I can no longer connect to the 2 domains I own from my home network. They do resolve from outside my network without issue. The ISP can't seem to figure out what is going on. It's worked fine for the last 5 years without issue but for a month now I haven't been able to resolve them. A traceroute gets a response from my router and the local ISP tower that I'm connected to a mile down the road. The next hop is my-public-IP-address.ispmgt.com (<- like that) and then the connection is dropped. I've checked my router logs and don't see anything that looks out of place but to be honest this is a bit over my head. I've also done a packet capture while running a traceroute and opened it up in wireshark but again I don't really know what I'm looking for. I have to think my ISP made some sort of network configuration change and screwed something up (wouldn't be the first time) since it worked fine for 5 years up until a month ago. Nothing has changed in my network either equipment wise or configuration wise, it's exactly as it was before it started acting up a month ago.

Any suggestions on what may be happening? Is there anything more I can do to maybe shed some light on the issue with my ISP or am I at their mercy to hopefully figure it out?

 

[Spearfoot]

Ok you all are a smart bunch so I have a question for you. What direction do I point my ISP in to figure out a domain resolution issue I'm having? My ISP is a local operation and I'm on a wireless connection since I live in a rural area and that's my only option. When it's working it works pretty good for what it is. When it's not it sucks.

Specifically about a month ago I started having issues with my internet connection. When it starts acting up the issue seems to be terrible latency and things slow down to a crawl. The bigger issue though is I can no longer connect to the 2 domains I own from my home network. They do resolve from outside my network without issue. The ISP can't seem to figure out what is going on. It's worked fine for the last 5 years without issue but for a month now I haven't been able to resolve them. A traceroute gets a response from my router and the local ISP tower that I'm connected to a mile down the road. The next hop is my-public-IP-address.ispmgt.com (<- like that) and then the connection is dropped. I've checked my router logs and don't see anything that looks out of place but to be honest this is a bit over my head. I've also done a packet capture while running a traceroute and opened it up in wireshark but again I don't really know what I'm looking for. I have to think my ISP made some sort of network configuration change and screwed something up (wouldn't be the first time) since it worked fine for 5 years up until a month ago. Nothing has changed in my network either equipment wise or configuration wise, it's exactly as it was before it started acting up a month ago.

Any suggestions on what may be happening? Is there anything more I can do to maybe shed some light on the issue with my ISP or am I at their mercy to hopefully figure it out?

I don' use my ISP's DNS servers, I use the free OpenDNS servers instead:

Use OpenDNS

There are other alternatives, too.

 

[Jailer]

Im using cloudflares dns servers. I tried using googles to check if that was the problem and it didnt help. Of course I cant do anything right now because my internet is down again…….

 

[Ericloewe]

So, is this strictly a DNS issue? Doesn't sound like it is. If it is, getting rid of the current DNS provider is straightforward.

If it isn't and bypassing DNS by resolving hostnames through alternate means and connecting using IP addresses shows that you're not getting to where you should be getting, then your ISP has something big that's wrong. The what depends on the layout of their network, how they're connected to the wider internet, etc.

 

[Constantin]

You could also try rigging up a pi hole with DNSCrypt per the instructions on Dean Seaman’s website and see if having a known-good local DNS server solves the issue. For all you know, something might be getting injected somewhere.

 

[Samuel Tai]

This smells like a radio interference issue. Maybe there's a microwave oven somewhere close to the upstream antenna that's leaking enough power to cause packet mangling. Or a new wireless tower. Or some kid playing with software-defined radios.

 

[Constantin]

Better antennas can help a lot but the backhaul antenna in question is owned by the ISP, so that's trouble. I'd have the ISP see if there is pattern to the issues their backhaul is experiencing. Perhaps something new got built or grew into the fresnel zone? Was another radio added nearby with a overlapping channel? Or the radio is malfunctioning? Someone streaming multiple Linux ISOs at once?

FWIW, I *really* like my RF elements horn. The thing is built like a tank and offers much better resistance against interference than the 32dbi 620mm ISO-equipped UBNT dish it replaced. Same radio so the swap-out was easy.

 

[Samuel Tai]

Thanks, @Constantin! RF Elements is a good resource to have in the back pocket. Which horn did you use?

 

[Constantin]

Thanks, @Constantin! RF Elements is a good resource to have in the back pocket. Which horn did you use?

I use an Ultrahorn. Even in an environment as noisy as Cambridge, it allows decent PtP long distance connections through some tree canopy, etc. There are literally hundreds of Wifi hotspots in any given direction and I was able to make a 2km connection with a -66dbm signal despite a tree being in the way and the antenna on the other end not being aligned perfectly with my location.

I plan on trying again, perhaps by donating a Ultrahorn to the ISP (netBlazr) so they feel more inclined to add me as a customer.

 

[Jailer]

So, is this strictly a DNS issue?

I'm not sure and I'm trying to figure that out.

If it isn't and bypassing DNS by resolving hostnames through alternate means and connecting using IP addresses shows that you're not getting to where you should be getting, then your ISP has something big that's wrong.

I have a couple services that connect via my public IP and a specific port and they don't work either. Same thing, I try to connect through my home network (phone via wifi) and it's a no go. Shut off wifi on the phone and use the LTE connection and it works just fine. I'm stumped. That's a good point and one that I haven't brought up with my ISP yet and I should. There's no name resolution involved at all there.

My router, pfsense, has a hostname lookup function in the diagnostics menu. When I do a hostname lookup on my domain it reports back correctly with all the correct information.

I'm thinking it's time to look for a new ISP.

This smells like a radio interference issue. Maybe there's a microwave oven somewhere close to the upstream antenna that's leaking enough power to cause packet mangling. Or a new wireless tower. Or some kid playing with software-defined radios.

I would think radio interference would be somewhat intermittent. Also with all the leaves off the trees this time of year one would think that things would improve not get worse.

Better antennas can help a lot but the backhaul antenna in question is owned by the ISP, so that's trouble. I'd have the ISP see if there is pattern to the issues their backhaul is experiencing. Perhaps something new got built or grew into the fresnel zone? Was another radio added nearby with a overlapping channel? Or the radio is malfunctioning? Someone streaming multiple Linux ISOs at once?

FWIW, I *really* like my RF elements horn. The thing is built like a tank and offers much better resistance against interference than the 32dbi 620mm ISO-equipped UBNT dish it replaced. Same radio so the swap-out was easy.

I'd pick up a new antenna in a minute if I was sure that was the problem. But for the problem to literrally show up overnight I'm betting it's a network configuration issue. Makes me want to go in to the IT business to weed out the idiots. Problem is a small town local run business like this can't afford to keep real talent and that's usually where the problem lies.

ETA: Running a traceroute on my public IP shows the exact same behavior as running a tracroute on my domain. I'm thinking my ISP has really screwed something up here.

 

[Ericloewe]

If you can't reach some hosts at all, but others you can, I'd rule out the physical layer for now and have your ISP start looking at what and how they're routing.

 

[Jailer]

Well I'm throwing in the towel on this one. I found out there is another wireless provider that offers service in my area and as a bonus it's much faster than I currently have for the same price. I wish I would have known about them sooner.

 

[Constantin]

Excellent! Depending on your needs, a good antenna on your end like the Ultrahorn may make a big difference also. If rural, the 620mm 32dbi antenna from UBNT is a less expensive and likely more performant choice. The ultrahorn is more of a urban PtP solution.

 

[Jailer]

Definitely rural. Largest town in my county is about 10k people and Im 10 miles from there. Closest town is 7 miles away. Ill keep that in mind once they get the new service set up.

 

[Jailer]

Well so much for that. Company came out today and did a site survey and no dice. I've got a neighbor 1/4 to the east of me, two a 1/4 to the south of me and several to the west of me that are with this company but they can't get signal at my place. My luck sucks and I'm stuck with the POS ISP provider.

I called my current provider again today to complain and they told me there's nothing wrong on their end and it's a problem on mine. I'm so frustrated right now I could scream.

 

[Ericloewe]

Is telling them to beat it and instead embracing Low Earth Orbit an option for you?

 

[Jailer]

Is telling them to beat it and instead embracing Low Earth Orbit an option for you?

Unfortunately no. I'd love to jump on the starlink bandwagon but they don't offer static public IP's and I need one for hosting the domain's. Plus they don't offer it in my area yet.

I'm going to contact the new company tomorrow and see if exploring options such as elevating the antenna are an option. I'm not opposed to dropping some cash on a small tower to get a signal so I can switch.

 

[Constantin]

I'm going to contact the new company tomorrow and see if exploring options such as elevating the antenna are an option. I'm not opposed to dropping some cash on a small tower to get a signal so I can switch.

Presumably they can tell you where their tower is and how tall yours will have to be to have a clear fresnel zone? They might also be interested in renting space on your tower to hop to other customers in your area.

 

[Jailer]

Well I'm stuck trying to convince my ISP that they have a configuration issue. There's no way to get new service unfortunately.

I'm going to paste this information here so you all can see what I'm dealing with. The first hop is obviously my router. The second hop is the local tower that I connect to (my upstream gateway). That third hop looks like it's being looped back to me and sees a response but the connection gets dropped after that. If I try to ping that address (67-209-241-26.ispmgt.com) it doesn't resolve. If I try to do a traceroute it doesn't resolve. I'm lost as to why it's changing a tracroute on my public IP to the address listed. Is that some sort of NAT going on with their equipment?

Code:

C:\Users\Desktop>tracert showersnet.com

Tracing route to showersnet.com [67.209.241.26]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  pfSense.localdomain [192.168.0.1]
  2     1 ms     2 ms     4 ms  172.23.116.1
  3    25 ms    34 ms    30 ms  67-209-241-26.ispmgt.com [67.209.241.26]
  4     *        *        *     Request timed out.
  5     *        *        *     Request timed out.

C:\Users\Desktop>ping 67-209-241-26.ispmgt.com
Ping request could not find host 67-209-241-26.ispmgt.com. Please check the name and try again.

C:\Users\Desktop>tracert 67-209-241-26.ispmgt.com
Unable to resolve target system name 67-209-241-26.ispmgt.com.

I also did a packet capture on my WAN and tried to go to my domain but I don't know how to use wireshark to see what's going on. I'm going to have to learn how to use wireshark so I can show my ISP where the problem is. They are literally putting no effort into this at all. There response, "We ran some diagnostic on our equipment and everything looks good on our end".

 

[Ericloewe]

2 1 ms 2 ms 4 ms 172.23.116.1

That's a private IP address, so there's definitely NAT involved at your ISP.

Anyway, the 67.whatever is your ISP or close to it, so try the following:

1. Ping 67.209.241.26 (i.e. with the IP address instead of the DNS name). If you can't even reach them, assuming they are responding to pings, it's clearly a problem on their end.
2. Ping someone well-known on the internet, like google's DNS servers at 8.8.8.8. If your pings get to the ISP's network but not to google, that's clearly a problem on your ISP's end (or rather, one within their sphere of responsibility).

I'm not sure wireshark is going to be very helpful to you, in this case. If your packets get to your ISP's network, everything will look right except that you get nothing back and TCP connections timeout before they're established.