Networking 101: Understanding iBGP
Building on last week's look at BGP, this week's installment of Networking 101 considers iBGP for connecting multiple border routers.
Internal BGP is a mechanism to provide more information to your internal routers. Most of last week’s installment on Understanding BGP focused on a stub configuration, where a single router served all the BGP sessions for an autonomous system (AS). This time we’ll delve into the practical use of BGP: iBGP and what it takes to accomplish multihoming.
If you were to add a second BGP router and connect it to another peer, your network wouldn’t gain much until the IGP knew what to do. There are a few options here, and one is a grave mistake. You cannot simply redistribute all of the Internet routes into your IGP and hope for the best. It’s really fun to do, actually, because the OSPF process normally takes down the router. Also, you need to get the routes learned from one border router to another, but that information will be lost unless both border routers are speaking BGP.
The solution is to set up an internal BGP peering between all of your border routers. The conventional wisdom is that your network will consist of a core (or transit, or backbone, or whatever you’d like to call it) network where this iBGP runs, and a default route will be injected into the widely used IGP (OSPF or other). As long as the IGP gets packets into the backbone, the routers there will be able to choose the best exit strategy.
Backbone networks can be quite complex, and the AS_PATH provided in BGP isn’t robust enough to guarantee the absence of loops. This means that iBGP will not advertise any routes it learns from its peers, which seems to make it useless, right? Not quite: the limitation means that every iBGP router must peer with every other iBGP router. This is also referred to as a full mesh. This is also a big pain.
When the network becomes sufficiently large, adding an additional BGP/iBGP speaker means that you have to make another connection to every router from that new one. Because of these issues, we have a few handy tools available in our arsenal: route reflectors and confederations.
Thankfully these terms are actually intuitive. A confederation is simply a separate internal AS. The idea is to break up the internal AS into smaller parts and tie them together with the external BGP. Each internal group will still have a full mesh, but that can be as fine-grained as you’d like. Using confederations is a configuration hassle, so most people opt for the alternative: reflectors. A route reflector is an iBGP router that will readvertise routes to other iBGP routers. This works by creating clusters of iBGP routers, and connecting them with a reflector.
A reflector will not send every route either; instead it only sends the best paths to its peers. When using a route reflector, there is a problem of convergence. Recall that “convergence” means that every router in the network has the same information, or more precisely, they all have a consistent view of the topology.
Synchronization is an important concept in BGP, especially when you’re a transit AS that ships traffic between peers. Let’s say we have two routers connected to separate AS peers, and we also have a multi-hop iBGP setup. So we have two routers (routers A and B) speaking EBGP with other AS’s, and iBGP between themselves, but indirectly. If a route for our peer on the other side of router A gets advertised into the iBGP mesh, and subsequently router B advertises it to its peer, a blackhole could happen. Why? Because it’s likely that the peer on the other side of B will want to immediately start sending traffic to the peer on the other side of A, but our iBGP may not have converged yet. Some of the interior iBGP routers may not know the way to the AS on the other side of A.
When you enable synchronization on a router, this means that the IGP (iBGP in this case) and EGP must have the same information before those routes are used. The basic design principal of a transit AS is that you don’t advertise something until you can forward traffic to that route.
A few different approaches are available to deal with iBGP and synchronization. We may turn on the synchronization option on our routers and wait for the IGP to have a route for the destination before it’s advertised to peers. Another option is to simply use a full mesh, so that iBGP convergence isn’t an issue. Clearly that isn’t going to happen when a network’s core needs to scale: it will implement something like reflectors that cause iBGP’s full mesh to be broken.
The real alternative, if you don’t enable synchronization, is to use route recursion. A recursive route lookup uses the BGP next-hop attribute to actually make a different route lookup. The IGP can use the destination network instead of the AS-path to determine where it gets sent. Even if the iBGP hasn’t converged, the routers will still know how to get to that network, since it will exist in the router it was advertised from, who will know the next-hop.
We know that routing isn’t dependent only on the protocols, because we’re human and we’ll have reasons for changing the behavior. BGP is limited in that the AS_PATH length is the only external mechanism used for route selection. That said, however, there are a few attributes of BGP that will allow you to influence the path a packet takes. Without getting into the BGP decision algorithms for route selection, there are still a few BGP attributes that you should know.
The MED, or MultiExit Discriminator, is used to indicate a preferred path. The MED is essentially a weight, and the lower value wins. This is a simple mechanism to say which entrance point you prefer, if you have two options for a path. The MED is used to tell a peer which one it should take, and it is only passed to your direct peers.
The LOCAL_PREF attribute is used to tell your iBGP peers the best way to get out to a differnt AS. Again, this is another mechanism used to prefer one equal (from BGP’s perspective) path over the other.
All in all, iBGP is really just BGP used in a different way. We hope the iBGP discussion has helped to round out your understand of BGP and inter-AS routing.
In a Nutshell
- iBGP is BGP used internally as a mechanism to exchange BGP information between multiple BGP border routers.
- Routers speaking iBGP must be connected in a full mesh to prevent loops.
- If reflectors or confederations are used, the iBGP mesh may have convergence issues that can cause blackholes.