Dual-STEP Protection Mechanisms – No-Export
Starting with the first day of Comm School, you were taught the importance to redundancy. I’d be willing to bet that within the first two weeks, you probably heard the phrase “two is one and one is none” several times. What you were likely not told was that every time you add redundancy, you add complexity. If you want your network to be able to employ your added redundancy without user action, it is even more difficult. I’ve actually heard before that “when shot X goes down, we need to move the cable from here to over there”. This is what ends up happening when Marines are told to implement more complexity than they are trained to implement. I wanted to write this post to help bridge the training gap and explain a configuration I would expect us to see much more frequently with dual-STEP connections increasing.
To illustrate this, I put together a very small topology of three routers: two STEP routers (routers 1+2) and one deployed router (router 3, USMC-RTR). Each STEP router is originating a default route into BGP towards our deployed router, and each router has a loopback corresponding to their router number which is also advertised in BGP (1.1.1.1, 2.2.2.2, 3.3.3.3).
Without any special configurations, you can see from the pictures, the deployed router will learn each of the STEP loopbacks, and pick one of the STEPs to use as a default route. At first look, it seems fine. Take a look at the BGP table from STEP2. This is a problem. Specifically, STEP2 is receiving a default route from our deployed router (which received this default route from STEP1). You could split hairs and also mention that we should not be routing ANY traffic between the two STEPs (which means we should also not tell STEP2 about STEP1’s loopback and vice-versa). What we are looking to do is to make sure we continue to advertise our deployed networks to each of the STEPs, but not serve as a pass-thru or tandem for the STEPs to reach each other. The main reason we do not want to allow this is because any traffic that goes through our link is bandwidth we are losing. In the case of a default route, ALL of the STEP traffic would be going through your link, which would most likely render your link unusable. As always, I wouldn’t be writing about this if I hadn’t seen it before. Now that we know what the problem is, how do we fix it? This is BGP based, so chances are there are several ways you can accomplish this. I’m all about using the simplest and easiest way, which I believe is to use a BGP community called “no-export”. You can get the official definition and some more info on this here. Here’s a look at what the configs would look like:
From the configs above, you can see that we use a route-map to apply this community on our deployed router. Specifically, the route-map is used under the individual neighbor command set and it includes a direction. Using the route-map this way marks ALL routes received by your STEP connections with the no-export community, and prevents them from being advertised out to the other STEP. If you are employing Dual DSIDs (IAMs), this configuration will still work as long as you have both of your DSIDs using the same BGP AS. If you were, for instance, providing failover to another unit using BGP peering, this protection would actually break this failover because the no-export would not only prevent the STEP routes from being advertised to the other STEP, it would also prevent them from being advertised to another BGP peer in another AS.
If you don’t want to use the no-export community, you could filter in the outbound direction and allow only routes you have originated. This will require more maintenance as you add and delete networks plus adds in the possibility of fat-fingering different routes, but is certainly an acceptable alternative, and depending on your scenario might be what you have to end up doing.
So what do the updated bgp tables look like after the config changes? Let’s take a look:
That is more like it! Looking at STEP2, you notice we are no longer advertising a default-route, we are advertising our deployed network to both STEPs (3.3.3.3/32), and we are still getting the STEP’s networks and a default-route from both of them.