QoS on the Black Core
Black core routing (BCR) brings a lot of interesting capabilities and employments, but will require me to modify my standard approach to QoS. I have always used a combination of NBAR, ACLs, and DSCP markings for my classification, but when dealing with encrypted traffic, your options for classification are generally constrained to DSCP (and to a limited extent ACLs). To think through how to attack this new QoS problem, I would recommend breaking up your task list into two parts. Part one includes the actions taken prior to the encryption device. This will be the more standard approach, but with some additional thought put into what will happen to the traffic after the next hop. Part two would be the actions taken on the black core router that can only “see” encrypted traffic (after the encryption device). If you haven’t already checked out my previous posts on QoS, there is some background here and here. I’m writing this post with the assumption you will be performing this QoS config on the edge of your network before it’s sent out a SATCOM-based WAN connection.
During part one, we will classify, mark (if not already marked as we want it), and shape the traffic before it hits the encryption device. We’re going to classify so that we can give certain traffic priority over other traffic, we’ll mark the traffic so that after it hits the encryption device we can still know what it is, and we’ll shape the traffic because we don’t want to send too much traffic (when sending over-whelming amounts of TCP based traffic over SATCOM, oversubscription can cause significant issues) and so we can be the most efficient possible with our limited WAN bandwidth. Before attempting to classify traffic, which I would recommend you do on the inbound interface of your router, you need to think about how complicated you would like to make things (more on this later). When using a black core router you need to make an aggregation of all the different schemes. For instance, if you want traffic A on enclave A to be treated differently from traffic A on enclave B, you need to at the very least create multiple classes of traffic on your black core router, and consider marking them differently before the separate enclave routers send them to an encryptor. I would recommend you shape on the interface connected to the encryption device at the max data rate of your satellite shot for each of the routers before they hit the encryption devices. This will oversubscribe your resources and will likely lead to some packet loss, but for efficiency reasons it is important the individual routers have access to the entire pipe (maximum bandwidth). Before moving on to the next step, make sure to verify the classification, marking, and shaping are all working as you expected (combo of Wireshark and show commands should do the trick).
Next up is part two. This is where you need to aggregate all of your (now encrypted) traffic. I’ve always employed simple classification policies where (not including sub-classes) I keep it to a priority class, three-four traffic classes and the default class. While there are multiple ways to approach this, I would recommend sticking to this scheme, even on your black core router. One thing I have seen frequently misunderstood is the role of the three to four traffic classes. Specifically, thinking the first traffic class is the most important, and so on. They are all equally important, speaking from a priority standpoint. You can assign more or less bandwidth as you see fit to further separate. Where you make the big decision, especially as it relates to complexity, is whether to implement WRED or not. WRED, Weighted Random Early Detection, will operate within the traffic classes you have created and provide further differentiation between traffic classes. The concept is that you differentiate within the classes by mixing the ratio at which you drop traffic during congestion. It is important to know this only works on TCP-based traffic, dropping UDP traffic won’t achieve the desired effect. Also keep in mind that for design purposes, you will need to mark each of these sub-classes differently (WRED configuration works off of DSCP markings). A quick example: Inside your “Work Applications” class, you have spreadsheets, presentations, and word processing. When you experience congestion, you can have WRED drop 1/20 spreadsheet packets, 1/16 word processing packets, and 1/8 presentation packets. In doing so, you would be treating spreadsheets best, followed by word processing, then significantly worse would be presentations. While it may not seem like much, dropping 1/8 or 12.5% of packets of a certain type will make a significant impact. To illustrate, I received complaints about a WPPL (Wireless Point to Point Link) shot once and upon testing determined there was about 8% packet loss. Changing out the cable and getting less than a percent of packet loss caused the users to feel like it was “over twice as fast” as before. The point is, even a small percentage of packet loss can cause a noticeable impact (secondary point being there is such a thing as “KINDA WORKS”).
The last thing that needs to be done to finish up part two is to shape the traffic before it hits your transmission device. Just like before during step one, you will shape at the max data rate for your transmission device (ex. 4Mbps if that is the SATCOM bandwidth assigned to you).