Throughout this series I will leverage screenshots and other source material that Cisco provides for the solution, including product documentation and Cisco Live sessions as well as material from the dCloud demo we are using for this series. Whenever I use non-original resources, I will provide a link to those resources at the end.
The Control Plane and You
Viptela was not the first WAN solution to disaggregate the control plane from the data plane, allowing multiple routers to share a common views of the network. IWAN did it with Performance Routing, and before that, BGP was doing it with scalability techniques like route reflectors. The vSmart Controller bears some similarity to these, but goes far beyond what each of them were capable of doing. Viptela had no intention of leveraging only BGP, nor was it a solution of solutions like IWAN. Because of this, the control plane was designed from the ground up to be scalable and disaggregated, as well as give each WAN edge the policy and visibility that translates business intent into network functionality. The heart of the Cisco SD-WAN control plane is the purpose-built protocol that’s used between the vSmart Controllers and WAN Edges: the Overlay Management Protocol.
vSmart Controller: The Next-Gen Route Reflector
It is a gross oversimplification to refer to the vSmart as a simple route reflector, but it is also an important part of relating the routing that most engineers know with this solution. So let’s take a moment to review BGP route reflectors in order to understand the similarities, and then differences, between the two.
In the case of BGP, the reason for route reflectors is a byproduct of how the protocol works. BGP is a path vector protocol, and this means that instead of there being some computation shared within the autonomous system (Like OSPF and EIGRP uses), BGP uses other attributes, most notably the AS-Path. This is because BGP is a zero-trust model meant to accept routes from only (somewhat) trusted peers.
What makes BGP a path vector protocol instead of a distance vector protocol is that the loop avoidance mechanism is the AS-Path itself, which could consist of hundreds or thousands of intermediate routers. The details are hidden because of the original intention that BGP was a protocol to use between separate autonomous systems. In the case of iBGP, however, we now have a problem. If the AS-Path is the loop avoidance mechanism, and all of the routers exist within the same AS, our only means of avoiding loops just disappeared. To solve this problem, iBGP peers will not pass on routes learned from other iBGP peers. While this resolves loops, it also means that a full mesh of iBGP routers is required, running separate control plane adjacencies for every other router. This is clearly not scalable, and is why route reflectors exist.
Each iBGP router can establish a BGP peering with one or more route reflectors, which will reflect the other iBGP routers’ routes to them. With route reflection, there is, by default, some visibility loss in routing as the route reflectors make best-path decisions and reflect only those routes, and it is here where the ‘vSmart as Route Reflector’ comparison stops.
The vSmart Controller does act as a route reflector for the WAN Edges insofar as it is a centralized point of control plane peering and route exchange, which improves the scalability of the solution. However, the Overlay Management Protocol was built to carry far more than route updates, and it is OMP itself that makes the vSmart much more than a route reflector.
How the vSmart Controller Works
Before getting into OMP in more detail, let’s take a look at how the vSmart actually works as part of the SD-WAN fabric.
The vSmart Controller is a virtual appliance that exists either within a public or private(on-prem) cloud. It provides the control plane for the network fabric, facilitates the data plane encryption between WAN edges, and propagates the centralized policies that establish and direct the fabric. I refer continuously to the vSmart in the singular, but the common deployment involves multiple vSmarts, which maintain their own OMP adjacencies with WAN Edges and each other. These devices can be regionalized in order to provide the fastest connectivity as well. In a global deployment, it’s likely that a pair (or more, if needed) would exist in every region. WAN Edges can be configured with vSmart affinity so that they will use the most appropriate ones first, but others in case of a failover.
The Fabric Control Plane
The vSmart receives its policy direction from vManage and disseminates it to all WAN Edges with which it has an OMP adjacency. It also receives OMP routes from the WAN Edges on a per-VPN basis, and, based on policy, distributes the routing information to the other WAN Edges, much like a BGP route reflector. The main difference between BGP route reflectors and a vSmart Controller is that the vSmart will not necessarily choose a single path if attributes are equal, whereas a BGP route reflector will (by default) choose a single path from the best-path algorithm. In fact, the vSmart can advertise between 1 and 16(!!!) equal-cost paths, though by default, it uses four.
By default, WAN Edges connect to the vSmart Controller via DTLS connection, establishing an SSL tunnel to establish OMP updates. Remember from previous posts that the vSmart and WAN Edges know how to reach each other because of the vBond Orchestrator, and that each device uses certificates to authenticate the other.
TLOCS are a topic which we will cover later. For now, consider a TLOC as a BGP Next-Hop for OMP based on the physical transport connected to a WAN Edge.
If DTLS makes you shake your head in disgust (looking at you, security guys) because of how hard it is to facilitate firewall traversal, TLS can also be used. This is not an all-or-nothing proposition, either. If there are use cases where the vSmart should support TLS and DTLS, it can speak either, depending on the WAN edge.
The Data Plane Encryption Facilitator
Anyone who has configured an IPSec site-to-site tunnel is familiar with the chicken-and-egg problem of securing data plane connectivity. Traditionally, the problem of how to establish secure connectivity between endpoints communicating in the clear has been solved by the use of IKE phases. In IKE Phase 1, endpoints communicate in the clear about how to establish a secure tunnel, and in IKE Phase 2, the secure data plane tunnel is actually established.
The vSmart Controller (and specifically OMP) supplant the use of IKE phases for data plane secure connectivity. Because the WAN Edge and vSmart already have a secure tunnel, there is no need for IKE Phase 1. By including this in OMP we also increase our scale numbers. OMP carries the TLOC information advertised by the WAN Edge to the vSmart, which, based on policy, will advertise that same info between WAN Edges. This is also how the per-VPN arbitrary topology is enforced within the SD-WAN fabric.
Because the vSmart takes care of the IPSec exchange details, including rekey and lifetimes for the security associations, it is functionally similar to a GETVPN Key Server. It’s a fairly loose comparison, but learning the SD-WAN solution is about relating it to other things which are already familiar.
The Centralized Policy Enforcer
Unlike a traditional L3 VPN using BGP and MPLS, the segmentation between VRFs (VPNs in Viptela terms) does not require separate control plane adjacencies. The VPN ID is part of the OMP update, and this is what allows per-VPN topologies and policies to be enforced centrally.
The centralized policy is what is used to determine what data plane tunnels should be built, what routes should be advertised or filtered, and any application-aware policies that would use telemetry to force traffic over particular transports.
Overlay Management Protocol: At a High Level
Now that we’ve talked about the vSmart Controller and its place within the SD-WAN fabric, let’s cover OMP at a high level. We’ve talked a bit about all the details carried by OMP, and generally what OMP does. The closest analogy for OMP would be BGP, with different address-families to carry the different information that OMP uses. This is by no means a perfect analogy, as OMP pulls triple duty as control plane, policy enforcer and data plane security facilitator, but it is close.
OMP and BGP share a lot of DNA, but OMP is purpose-built for the Cisco SD-WAN solution and thus carries the relevant info about TLOCs, OMP routes and service routes. Let’s briefly discuss each, with the understanding that OMP really deserves (and will get) its own deep dive.
TLOC: Short for Transport Location. It’s a combination of attributes, namely the public/private IP (if different) address, the color of the transport (ie, internet, mpls, gold, red, etc), and the encapsulation type (IPsec or GRE). There’s actually a lot of detail for TLOCs which we will cover in a deep dive of OMP later, but at a high level, you can think of TLOCs as the next-hop of any OMP-advertised routes, similar to BGP. The TLOC IP will actually show the system IP address of the WAN Edge (think BGP router-id) in the routing table rather than the actual transport IPs.
OMP Routes: Routes advertised into OMP carry traditional routing information, but also more that allows the per-VPN segmentation and arbitrary topology enforced by the vSmart. These types of routes can be advertised into OMP right now – there are others on the roadmap:
- Connected
- Static
- OSPF
- BGP
In addition to attributes such as origin, site ID, tag, path preference and the route’s originator, others such as VPN ID and TLOC are included. This allows the vSmart to control the VRF (VPN) segmentation with a single adjacency, and also to enforce the arbitrary topology via TLOC (next-hop) advertisement.
Service Routes: These routes are advertised into OMP by sites that offer a service such as a firewall or IPS. By advertising a service into a VPN, the vSmart can enforce a security policy such as, “All traffic between two subnets/sites must be inspected by a firewall”, without having to do something creative with traffic engineering. The service can be advertised into a VPN, and traffic which meets the criteria is redirected to the service and then on to its destination easily.
Service chaining can simplify certain use cases without having to build extra infrastructure.
We’ll get a lot more into OMP and TLOCs when we start talking policy creation, and before that I’ll dedicate an entire post to OMP. This post is more focused on the vSmart itself and its role in the SD-WAN fabric. Was it enough detail? Were there topics I missed, or didn’t provide enough detail about? Let me know!
Next time we’ll deep dive on the WAN Edges themselves, which will wrap up the component showcase part of the series.
Image Sources:
Introduction to SDWAN (Viptela) Architecture – Mossadaq Turabi
