Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

How FiOS Works, and How To Ditch Verizon's Equipment

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
I looked all over for this information; I found it, essentially, in no one place. While this is not FreeNAS-related per se (hence, it is in Off-Topic), I still decided to put this post here because not only will it be academically interesting to many of you, but I suspect many people just like me (minus the FreeNAS) will stumble upon it because they are searching for this information, and maybe, they'll join us in FreeNAS. :) A disclaimer, however, before I begin: I only figured this stuff out to precisely the extent that I needed in order to make correct decisions about what equipment to buy; I am not an expert on Verizon's product line, nor am I an SME on the underlying FiOS technology.

In the United States, and in most suburban areas (urban areas are sometimes left out) with people with decent incomes, Verizon has had a product out there for just over ten years, called "FiOS". The gimmick with this product is that it's essentially FTTP (fiber-to-the-premises): fiber comes straight from the backbones to the "last mile" as they say, and then to (usually the side of) your house, where it terminates in an ONT, and provides customers with "triple play" services, to include cable TV, internet, and the modern instantiation a la fiber of POTS. These interface readily to the pre-existing phone wire and/or coax in your home. The centerpiece technology (aside from the FTTP) is something called MoCA, which, in a nutshell, allows ethernet to be carried on coaxial cable, "out-of-band" as it were.

Back in the day, Verizon provided 5, 10, or 20'ish Mbps (at first asymmetrically, but later same up and down) internet service; that seemed pretty impressive in 2006. When they used to do this, they simply didn't even fire up the ethernet on the ONT, they just delivered the whole cable TV + internet over the single coaxial feed, using "MoCA for the WAN". They had this really monstrous and technologically immature router from ActionTec known as the MI424, and this router MoCA'd that internet connection for you into 4 bound ethernet ports right on the MI424 plus 802.11b/g. At the time, there wasn't a lot of MoCA equipment people like us could just buy on the market, and there wasn't much call for it. Net result: You more or less needed that MI424 inline if you wanted to get things like video-on-demand and program guides, which were all provided over IP to the set-top box(es), which themselves were MoCA devices, unbeknownst to anyone really. The main problem here was the MI424 (or at least, the firmware) was never a totally satisfactory thing, and higher-level hobbyists that were using FiOS (like me) lamented both that we were essentially trapped on the MI424 (the only escape came at too high a cost of convenience), and that the thing was just a disaster (e.g., many models had such woefully small NAT tables that they could be overpowered by simply searching for available Day of Defeat servers on Steam), and to really top it off, had terrible wireless range, and performance. Later revisions of the MI424 brought to bear things like GigE on the LAN and better 802.11, repaired some of the earlier boo-boos, but it was still painfully clear to any advanced user that it was really sub-par stuff.

Over time Verizon got better. They rebranded the current instantiation of FiOS as "Quantum" (I guess?) and provided now this G-1100 router, which handles the 50, 100, 150+ Mbps that we now have on FiOS, which seems to work somewhat differently in terms of topology. The G1100 has MoCA, good routing tables, OK performance, and a much improved (but still disappointing) wireless access point. While still disappointing, because the router is designed for Aunt Sally, not DrKK, I would have probably lived with it. But now the story gets interesting.

For these higher speeds, Verizon has NOT been using the MoCA for the WAN. Verizon has been using a direct ethernet connection. That direct ethernet connection gets fed in the G1100, which then busts it out like any switch to its 4 LAN ports. There is also a coaxial port on the G1100 which has NOTHING to do with the cable TV (yes, you heard that correctly---there is a *SEPARATE* coaxial cable from the ONT that never touches the "router" that carries your cable TV), but merely provides a LAN MoCA bridge to your network. If you look over your FiOS Quantum FiOS install, you should see two things coming in through the ONT: Ethernet, and the coaxial cable for the cable TV. There is this coax from the G1100 which MEETS with the aforementioned coax, presumably in a splitter, near in the network ingress point. This latter has NOTHING to do with the Cable TV, and EVERYTHING to do with everything else.

Consider your set top boxes. You should notice (with Quantum) one of them, presumably the most convenient/out-of-the-way one, is a big, heavy, box, that has all your hard drives (DVR) in it, and so on. I will call this the "master" STB. The remaining boxes are these tiny things, with a tiny PCB in them, about the size of a paperback book, they are the "slaves" in my nomenclature. Here's the deal:
  1. The master STB presumably gets an IP address from DHCP over its MoCA, and announces its presence (either at the IP or ethernet layer, didn't investigate this thoroughly).
  2. The slave STBs do the same, and they listen for their master.
  3. If you watch TV at the master STB, the master STB appears to demultiplex the requested cable tv channel from the traditional CATV signal on the coax, and presents it to you.
  4. If you watch TV at a slave STB (and this is interesting), the slave STB appears to tell the master STB what channel you want. and the master STB now demultiplexes that channel for you, and sends it up to the slave STB via IP/ethernet, typically getting to this layer on MoCA (because all STB's are MoCA boxes as well).
You can see this in action---disconnect your master STB, and you will see that you cannot watch TV now any of the slave boxes, even though all slave boxes have access to the CATV signal on the coax, see. So essentially, this means that any cable TV you are watching on any of the slave STB's, is really IP television provided to you on demand from the master box. But this also means something else: you'll notice your slave STB's have an ethernet port on back. Sure enough, if you disconnect the coax ENTIRELY, and just hook up your ethernet, you are going to be able to watch TV. Further proof that you're watching IP television on the slave boxes. This was the moment of revelation for me.

So, I have problems with running Verizon's G1100 router:
  1. They open ports, that I cannot un-open, nor control. I do not want Verizon on my LAN.
  2. The wireless, while better, is not sufficient.
  3. They charge me to run it, and I don't want to run it.
  4. If it were up to me, there'd be a far more cuspy network in play, with routers I totally control.
These problems were "annoying". I understand why they have port(s) open into your LAN---if they didn't, they're going to triple their hassle and cost with all of the morons using FiOS calling customer server for lost wi-fi passwords and whatnot. I understand they have to charge for things and make a profit. etc. I get it. I would have let it slide. But then the most annoying thing of all time happened:
  1. The MoCA LAN on the G1100 started flaking out a bit. Like, the hardware piece, not something Verizon was doing. It would totally drop out of service for hours at a time, and since I was using MoCA at my office, and for my wireless access point (Ubiquiti AC-LR), having the router drop its LAN connection meant all of *MY* stuff was no longer on the LAN. That, my friends, wouldn't fly.
  2. The thought of calling Verizon and explaining this to them filled me with dread.
  3. I knew neither the first, second, nor third level of person I'd talk to at Verizon would understand what I was saying, and it would take the fourth level.
  4. I knew that each level represented a 30 minute wait time.
  5. I knew that each level represented 20 units of systolic blood pressure increase.
So I decided it was best not to even call. It was time, now, to cut Verizon's router totally out of the loop, because I knew I couldn't rely on it. The next I'm in China, or Australia, or, hell, even at work for the day, and router drops, and I have a wife and kids going ape****? No thanks. So that's when I began to try to unravel the mess.

Ultimately, after trying several lower-priced versions, I determined that there was no substitute for the Actiontec MoCA bridges. These are by far the best made, and most reliable MoCA bridges for consumers out there. I went to the Verizon router, I disconnected the LAN MoCA coax from the router and splitter (leaving, obviously, the cable TV cable--that is nowhere near the router--in place), and instead ran it to the separate ActionTec MoCA bridge, my own piece of coax, and ethernet connection back to the router. Currently, these guys are $70 apiece in the United States. I used the Actiontec bonded MoCA 2.0 bridge to provide my own LAN to coax bridge at the router side, leaving the G1100 in place for now. That worked, and immediately, everything was working better on my LAN. Excellent. Then, after some discussion, I decided I need a proper Nerd's Router....the kind you can't buy at most stores, and the kind you don't even know how to begin using unless you're practically a sysadmin. For these, the two main low-cost players are MikroTik and Ubiquiti Edge Router. As I am a big Ubiquiti fan, I was partial to the Edge Router. But there's a pesky and horrifying UDP-reordering bug that so far has resisted a fix, for over a year, with the CPU's on those products. As I run some pretty serious UDP both LAN-to-LAN and WAN-to-LAN, that was a non-starter for me (sorry, Ubiquiti, I wish it were different). So I went with the MikroTik hEX router.., which is an amazing little tiny 4-watt thing slightly larger than a pack of cigarettes, yet orders of magnitude more beefy than Verizon's router.

Now. If you've never used something like this before, it's hard core as hell. These things are way way way way more complex than 98% of people out there can even dream of using. There are no pretty menus with easy to understand options. You basically have to program these things from the command-line interface, or from a GUI which is only half a step away from a command line interface. You "forward a port", for example, by accessing the Internet Protocol menu, firewall rule submenu, then inserting a routing rule, putting it in the dst-nat chain, tieing that to ether1 on the input and for IP packets that match protocol 17 (or 6, or whatever your use-case is), and then triggering the action dst-nat to such-and-such an IP address and port address on such-and-such ports on such-and-such switch chip in the router. The word "port forward" occurs nowhere, in any of the approximate 150 subsections. But let me tell you, *IF* you can handle it, the feeling of joy you will experience is unbounded, once you have this thing running. So I'm not going to tell you how to handle a MikroTik router, but if you can handle it, let me tell you, that's the best $60 you'll spend in your life. Here is one tip you'll need: Most people are accustomed to their ISP-provided router providing "Hairpin NAT" for them automatically---this router provides nothing at all. You'll want to put a LAN-to-LAN Hairpin NAT as a "masquerade" in there. If you don't, then any WAN-looking requests you make from inside the LAN to destinations in the LAN won't route correctly. (for example, if your WAN IP is 1.2.3.4, and you have a port forward from port 19999 to port 20000, and you try to access 1.2.3.4:19999 from within your LAN, it won't work right). Anyway, you'll need that tip most likely ;)

So here we are. The G1100 now serves no function in my network, and I have removed it from the network. Every feature of FiOS works, including all video on demand, all television, all program guides, etc. Join me in liberation, friend.

router_small.jpg
hammer_small.jpg


On the left, the Actiontec MoCA bridge, and the Mikrotik hEX router, grand total, approximately 5 watts, and about 10 times the performance and reliability of the G1100, pictured at right, with suggested treatment.

Edits: For missing words/typo, also spelling of "MikroTik" vs "MicroTik"
 
Last edited:

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,824
knew that each level represented 20 units of systolic blood pressure increase.
At 20mmHg per support level, that's a guaranteed stroke by the third. Sounds about right for an ISP.

I should post how Microsoft's platform (it's no longer theirs, they sold it off) works, later, since it's far less asinine and it's in wide use. It's rather similar, but the router is significantly more important.
 

nightshade00013

Neophyte Sage
Joined
Apr 9, 2015
Messages
1,258
Just a note here on Mikrotik.

You can install it on x86 components as well. You pay for the license but the freeish version will still work for testing and some limited things.

It's actually designed for use by wireless isp's and it is a pain to work with. But it can be done and if you already have some hardware sitting there you can try it out.

The bad news however is most devices are a pay to upgrade. Just because you have a version now it doesn't mean you get to upgrade it later. They usually give a single version upgrade after that pony up some more money. This sucks since it is actually based on the linux kernel.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Everything works so much better. NO drop outs on my TV's, no dropouts on my MoCA connected devices, even small things like the "wi-fi calling/texting" on my cell phone are now working MUCH better.

Highly recommend.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
The bad news however is most devices are a pay to upgrade. Just because you have a version now it doesn't mean you get to upgrade it later. They usually give a single version upgrade after that pony up some more money. This sucks since it is actually based on the linux kernel.
If you buy one of their "RouterBoard" devices, such as the router I picture above, then you can upgrade both the OS and the firmware to your heart's content. There is the issue of license levels, of which I think there are 6, in increasing price, with increasing features unlocked. The routers they sell come with a "4", which is 100x more stuff than any SOHO would need. If, however, you buy a separate license for your own hardware, they are very clear about this: You cannot pay the "difference" in cost if you need a higher license level---you must buy the new license level outright, so exercise caution.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Also:

This explains why when I bought the MoCA box, which has a "Coax in" and "CATV out" coax connectors, why the "CATV out" connector didn't provide cable to the TV (and so, ultimately, I had to provide my own splitter)----because that tv was never using CATV at all, it was using IP television provided by the "media center" box.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Just for the record, I built a nice little shelf, and cleaned everything up.

cleanedup.jpg
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
For the record, we're a few months in now, and it is totally pimp. I wish I would have done this two years ago. 100% error-free, snappy, perfect, and no Verizon crap.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
I've just had an interesting Zen moment.

I was twiddling around in my verizon.com account, and while doing that, I realize, "Hmph. Now that i've ditched the router, I don't have the right WAN side port forward to remotely access the STB's for the DVR and stuff."

OK, not that that's a big deal. I think we're all better off in the universe if we don't have port forwards to LAN equipment for convenience items like DVR's and what not. But that didn't stop me from trying to just make sure I *could* do it if I wanted to, now that I've ditched the equipment from Verizon.

So, this post right here appears to give the correct information, or a road to figuring it out. Interestingly, it did not work. So I figured, I think logically, "OK, maybe the ports are different now, something something something." So I open up the full ingress logging on my MicroTik router, which of course results in zillions of internet l33t h4x0r packets being logged by me, etc., but I'm committed to see what port Verizon is actually trying to use to get at my STBs. Quite surprisingly, after bouncing out the many Chinese, Bosnian, Moldovan, and Russky hack background radiation, there's NOTHING. No record of Verizon trying to hit my router when I am trying to access my DVR remotely from their account management features. Wut?

So I go on my VPS's, and fling a few packets back to me on various ports, sure enough, they show up just fine. So definitely, I'm not even getting Verizon's packets.

What the hell is going on? Verizon's not even *TRYING* to hit my DVR? So I go and think about it outside in the -20C weather today for a few minutes. And it dawns on me, in a moment of zen.

I had ASSUMED (and I guess I thought this was a decent assumption) that Verizon associated my account to my DHCP leased IP based on the physical layer or the DSLAM layer. "The customer is at such-and-such physical address, and that line has such-and-such IP lease." Well I guess that's not right. I suppose what the deal is, is Verizon associates my account with the MAC address of my Verizon-provided router (a different layer entirely), and since I have a different router, thus this is not a MAC address they know about. So their network blithely replies to DHCP lease requests on Verizon FiOS lines, but those leases are only known to belong to a particular customer when the MAC address is something in their records. So I have a successful IP address/DHCP lease, but Verizon really doesn't know to whom they gave it to. LOL. That's interesting, no? This means that when I do, from my Verizon account, "contact DrKK's router", they literally have no idea how to do that.

So I suppose what I have to do now, is RELEASE my DHCP lease, prevent a new one from being assigned, change the MAC address of my MikroTik to match the old Verizon router's fixed MAC address, restart the Mikrotik, ask for a new lease, and give the system a chance for that to propagate. **THEN** set up some kind of port forwards to the set-top-boxes.

Fascinating, no? Anyway, I can't do that now, because it will involve disrupting the network in the house for maybe an hour, and that will cause wife/kids meltdown. But I'll do it later and report back.

If anyone has gone through this and knows that all of this is correct, let me know please, save me some exploratory trouble.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
13,189
Are you sure the DVR is actually local? Latest trick is to centrally DVR everything at the headend/central office/"cloud"/wotevr.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Are you sure the DVR is actually local? Latest trick is to centrally DVR everything at the headend/central office/"cloud"/wotevr.
Yes I'm sure. Not only do I see all the hard drives in the master STB, but I have observed the DVR serving packets LAN-to-LAN.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,824
This raises a few questions:
  • Can they bill you, assuming there's a data cap?
  • Do they have any clue which one of their customers you are?
  • How is tech support going to react?
I actually have a story about the last one (not with Verizon, obviously). Service was out for several days because of a broken fiber somewhere. After it was restored, the guy said something about remotely resetting the router (I can't remember why, but I didn't find it too weird at the time, which is weird...) and I told him he could try all he wanted, but he'd get nowhere since it wasn't their router. You could kinda tell he was completely lost without his script to help...
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
This raises a few questions:
  • Can they bill you, assuming there's a data cap?
  • Do they have any clue which one of their customers you are?
  • How is tech support going to react?
I actually have a story about the last one (not with Verizon, obviously). Service was out for several days because of a broken fiber somewhere. After it was restored, the guy said something about remotely resetting the router (I can't remember why, but I didn't find it too weird at the time, which is weird...) and I told him he could try all he wanted, but he'd get nowhere since it wasn't their router. You could kinda tell he was completely lost without his script to help...
Right, so, I've been thinkning about that.

There appears to be two control systems. There is the control system that handles bandwidth, packet inspection, routing tables, physical layer/fiber, etc. This is where their non-customer-facing tech people work. As far as they're concerned, everything's fine. I can change my bandwidth, or whatever, and they just change it and the fact that I'm not on their router couldn't be more irrelevant. The mistake is assuming that this is where everything in terms of customer equipment was handled; apparently, it's not.

The second control system, that handles all the user-level devices and whatever, and anything that involves controlling routers or STB's, or whatever, is a separate deal. It is this second system that appears to be keyed towards the gateway itself, and its MAC. At least, this is my hypothesis. This is probably a bunch of very lightly-trained dudes that run scripts, as you say, Eric.

There's really no other logical explanation for this, is there?
 

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
So I suppose what I have to do now, is RELEASE my DHCP lease, prevent a new one from being assigned, change the MAC address of my MikroTik to match the old Verizon router's fixed MAC address, restart the Mikrotik, ask for a new lease, and give the system a chance for that to propagate. **THEN** set up some kind of port forwards to the set-top-boxes.

Fascinating, no? Anyway, I can't do that now, because it will involve disrupting the network in the house for maybe an hour, and that will cause wife/kids meltdown. But I'll do it later and report back.
While our situation is mostly similar (I have my old Verizon router in place only for the MOCA function), the last time I released my IP it took over 3 hours to get my WAN back online.

I replaced the Verizon router with my pfSense server quite some time ago but had no desire to muck things up with the STB reception possibly loosing guide/on demand features. All I did was disable the Actiontec's wireless, removed the antennas and plugged in an ethernet cable from it's WAN port to my network switch. My pfSense gave the verizon router an IP and the STB has worked without issue ever since.

FYI here in Texas, Verizon sold out to Frontier Communications, but I was "off their grid" long before that transpired ;)
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Now we're several months in. FINAL REPORT:

* This is {profanity} awesome.
* I wish I would have done this years ago.
* I am now a Mikro-tik fanboy, and am going to their conferences.
 

nightshade00013

Neophyte Sage
Joined
Apr 9, 2015
Messages
1,258
Glad to hear it's working well. Used to have some mikrotik stuff I dealt with while working for a WISP. It worked well but was a pain to deal with when it wasn't working right.

Anyway did you see that they are having issues with some of the mikrotik routers? https://www.wired.com/story/router-...-operation-compromised-more-than-100-targets/ not sure if it will apply to you but what I grazed over in another article is they were shipping that way.
 

ere109

Member
Joined
Aug 22, 2017
Messages
159
I do not have FIOS, but just spent 45 minutes reading and enjoying this write-up. All the tricks are out there for the tech we use - we just have to find them and implement them correctly.
 

Phishbum

Newbie
Joined
Dec 11, 2018
Messages
1
I’m curious to hear if your DVR and guides still work on master and slave STBs. Looking to ditch Verizon router and replace with Linksys Velop with 1 Actiontec bonded MoCA adapter
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
I’m curious to hear if your DVR and guides still work on master and slave STBs. Looking to ditch Verizon router and replace with Linksys Velop with 1 Actiontec bonded MoCA adapter
The DVR and guides are completely unaffected. The DVR is a local/LAN thing, and the guide appears to be something initiated client-side (though I didn't verify this). Thus, they work just fine.

The only thing that does NOT work is the stuff that "reaches in", for example, controlling the DVR schedule from OUTSIDE the house (on say, a mobile app), and when you're on the Verizon website itself there are various minor things that it is supposed to be able to do (e.g., telling you your wi-fi password, telling you what's up with your DVR status, whatever, etc) that it can no longer do. But these are all things that anyone reading posts on a forum like this wouldn't care about anyway. For completeness, I did some research and found out what ports the old Verizon router was listening on to accomplish these tasks, and forwarding them appropriate to the master and/or slave box did not help. When I then listened to all packets ingressing WAN-side, I did *NOT* see any from Verizon attempting to get in. Thus, it would appear Verizon had no idea where to find my STB's from the WAN. I believe the explanation there (?) is that the old Actiontec box had a MAC address, which I did not spoof on the Mikrotik replacement; thus, when the Mikrotik did its DHCP request and got its IP, Verizon did not connect it to the account, since that is done by MAC? This is a total guess. This would of course mean that level-of-service (do I give this turkey an IP address? And at what speed to I serve him?) is done at a much lower-numbered layer than the administrative part, which would not be my design, but whatever.

Short version: Guide and DVR works fine inside the house, and is unaffected.
 

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
590
@DrKK Haha ... I was about to reply with "Check Mikrotik/Routerboard devices" when i was reading your first post :D I am on Routerboard for ~10years never going back to the plastic overpriced SOHO boxes with futuristic design and blinking logos.

Just a note here on Mikrotik.
The bad news however is most devices are a pay to upgrade. Just because you have a version now it doesn't mean you get to upgrade it later. They usually give a single version upgrade after that pony up some more money. This sucks since it is actually based on the linux kernel.
Uhm i don't follow. I have a bit old RB450G and it still gets FW updates as well as mipsbe packages.

Edit: Arw fu*k. Just noticed the timestamps of the post(s). Lol...
 
Last edited:
Top