Throughout this series I will leverage screenshots and other source material that Cisco provides for the solution, including product documentation and Cisco Live sessions as well as material from the dCloud demo we are using for this series. Whenever I use non-original resources, I will provide a link to those resources at the end.
The Orchestration Plane and You
To be completely honest, I actually don’t like the name ‘vBond Orchestrator’. I think it would be more accurate to have called it the ‘vBond Facilitator’, as it facilitates SD-WAN fabric onboarding. The vBond holds the information needed to authenticate the vEdges/cEdges that wish to join the fabric, as well as the the list of vManage and vSmart controllers to pass along to those routers.
Where’s the Easy Button?!
I want to begin by saying that this entire post is optional, depending on how much of the infrastructure you plan to manage yourself. The simplest solution to managing trust and onboarding (including the vBond Orchestrator) is to allow Cisco to do it for you.
Cisco can manage the entire trust infrastructure for you, including certificates, vBonds and onboarding.
Everything beyond this point is really just to help you understand how it all works beneath the covers, and should provide some direction to those who wish to do all the infrastructure themselves.
vBond Authentication
There is an entire SD-WAN fabric cross-authentication process which happens before the first vEdge/cEdge tries to join the fabric. All elements of the SD-WAN fabric authenticate each other using a whitelist model utilizing certificates. The vSmart Controllers and vManage will authenticate and be authenticated by the vBond as well. This is important as we start to discuss the process by which the vEdges/cEdges authenticate and then join the SD-WAN fabric.
For the vBond to authenticate any vEdges/cEdges attempting to join the fabric, a few things must exist. This list changes a bit based on whether the plan is to use zero-touch provisioning, pre-configure the routers, or even to replace the default certificate authority with an enterprise CA.
- vManage must have an entry for each vEdges/cEdge, including the serial number (or token if using a Cloud vEdge), and it must not be in a state of Invalid. This was discussed in the previous post about vManage.
- vManage must have an Organization name defined which matches the vBond and vEdges/cEdges certificates, or it must be configured to manage the Cert Signing Request/Certificate install for the routers during the onboarding process if the vEdges/cEdges have only the default root CA installed (common in zero-touch provisioning).
- vBond must have the list of vEdges/cEdges from vManage and a valid certificate signed by a trusted CA. Note that using an enterprise CA makes true zero-touch provisioning impossible, as the routers must have the enterprise root CA installed. However, vManage can still take care of the CSR and certificate install when using an enterprise CA.
- vBond must have a reachable public IP address (or sit behind a static 1:1 NAT which is reachable) and there must be a DNS entry for the vBond’s address. If not using zero-touch provisioning (ZTP), the vEdges/cEdge devices must have a minimal configuration added which includes the system-ip, the DNS name of the vBond and the DNS server which can resolve the name, and the organization name. We will discuss how ZTP differs in more detail later.
The vBond Orchestrator. Note that there is a valid certificate, system-ip, and organization. Notice also that the protocol used by the vBond for communication is DTLS.
vManage is what manages and maintains the trust level and certificates for each element of the solution, and it relies upon a trusted root CA. The vManage certificate trust process is covered in greater detail in the previous post. The vBond will receive the list of authorized devices from vManage, and it is this list that the vEdges/cEdges will be checked against. It is also this list that contains the vSmart and vManage which will be disseminated to onboarding vEdges/cEdges.
How the vBond Resolves a Standoff
Let’s get into the details of how the vBond validates each other SD-WAN element. The vBond will never initiate the DTLS connections to the other SD-WAN elements, it will only wait to be contacted. However, there is mutual authentication in order to set up the DTLS tunnels needed.
vBond to vSmart:
The vBond validates the following:
- Check that it trusts the root CA installed
- White-list entry from vManage
- Certificate serial number
- Organization name (from certificate) compared to local certificate OU.
The vSmart validates the following:
- Check that it trusts the root CA installed
- Organization name (from certificate) compared to local certificate OU.
vBond to vManage:
The vBond validates the following:
- Check that it trusts the root CA installed
- White-list entry from vManage
- Organization name (from certificate) compared to local certificate OU.
The vManage validates the following:
- Check that it trusts the root CA installed
- Organization name (from certificate) compared to local certificate OU.
vBond to vEdge/cEdge:
The vBond validates the following:
- Check that it trusts the root CA installed
- White-list entry from vManage
- Organization name (from certificate) compared to local certificate OU.
The vEdge/cEdge validates the following:
- Check that it trusts the root CA installed for vBond (later, vSmart and vManage)
- Organization name (from certificate) compared to local certificate OU.
As you can see, there’s a LOT of cross-authentication involving white-lists, certificate trust and organization name checking, and the vBond is right in the thick of it. vEdges/cEdges will not maintain this relationship once authenticated, they will drop the DTLS tunnels to vBond once they receive the list of vSmarts and vManages (if there are more than one). The vSmarts and vManages however, will maintain a persistent DTLS connection to the vBond in order to provide any needed updates.
Let’s Talk Onboarding
With some very minor differences in the case of Zero-Touch Provisioning, and when utilizing Cloud vEdge infrastructure, the onboarding process involving the vBond is the same across all use cases. We’ll cover the ‘normal’ process of how vEdges/cEdges join the fabric (as pertains to the vBond) and then I will cover the use cases with extra steps.
When the vEdge/cEdge is powered up, it will attempt to contact the vBond’s DNS name. If it has been preconfigured on the vEdge/cEdge (along with a DNS server which can resolve the name) and a system IP, all is well. This first step differs if using ZTP.
The vEdge/cEdge will contact the vBond to set up a DTLS tunnel for authentication and exchange of information. If the vBond has a valid white-list entry for the serial number/chassis ID of the vEdges/cEdges, and if the root CA is trusted, the vBond will inform the vEdge/cEdge of the IP addresses of the vManage/vSmart.
One piece of information that the vBond will also pass along to the vEdges/cEdges, if relevant, is whether it is behind a NAT. It does this by informing the router of it’s public IP address (from the vBond’s perspective). If the router doesn’t have any interfaces configured with that IP, it knows there is a NAT. This will be vital to establishing data plane connectivity with ipsec later on as NAT-T will be needed.
After the vBond has authenticated the router and passed along the information for the vSmart/vManage, it will also send the router’s public IP to the vSmart/vManage to inform them that a connection request is incoming. The SD-WAN elements will still have to cross-authenticate, but this facilitates that process.
At this point, depending on whether the vManage is performing automatic certificate management for the vEdges/cEdges it may take care of cert enrollment/install. If the device certificate is already installed, the vManage/vSmart will take care of onboarding to the fabric from there, establishing persistent DTLS tunnels for the purpose of management, ipsec tunnel establishment facilitation, and policy/template application. I should note here that the administrator can control over what transports these DTLS control tunnels can be built to the SD-WAN infrastructure.
If the vEdge/cEdge is marked as Valid inside of vManage (see previous post), the router will establish ipsec tunnels as policy dictates for the purpose of data plane forwarding, as well as sending/receiving OMP routes. The router is now participating in the fabric.
Look at the vBond Orcherstrator connection history. It keeps track of the onboarding attempt for BR2-VEDGE1 (our ZTP use case). You see the private/public IP and port used (in this case, no NAT) and what transport was used.
Other Use Cases
Let’s talk about how the process changes with some other use cases. We’ll cover virtual hardware first, and then zero-touch provisioning. Except where specified, the onboarding process works the same as above.
Cloud vEdge/cEdge Onboarding: Virtually Simple
The main difference between the virtual vEdges/cEdges and the physical ones are the lack of a hardware TPM/SUDI chip. Instead, the vManage actually creates a one-time-password/token when the cloud vEdge/cEdge is added, which is then used during the cloud device instantiation to serve the same function as a serial/chassis id. When the router contacts the vBond, it will use this token as part of the authentication. After the cloud vEdge/cEdge is onboarded and connected to vManage, this token will no longer be used.
This Cloud vEdge will use ZTP to onboard. The token is produced by vManage when the device is added there, it must then be loaded into the Cloud vEdge instantiation scripts. Notice there is no certificate installed or organization specified when using ZTP.
Zero-Touch Provisioning: The Preferred Use Case
With ZTP, how the vEdges/cEdges discovers the vBond is slightly different. From the factory, there is a default configuration loaded on the router which includes the ZTP server set to ztp.viptela.com. This Cisco-administrated vBond uses the serial/chassis in order to inform the router of its organization name, and redirect it to the vBond which belongs to that organization whether it exists in a public cloud infrastructure or inside an enterprise.
Once the vEdge/cEdge is redirected, the vBond will authenticate the router, then that router will connect to the vManage in order to receive a configuration which includes a system IP.
After this is configured, the vEdge/cEdge will drop the connections to all parties and reconnect to the vBond with the correct system IP, at which time it will be upgraded if necessary. If upgraded, the vEdge/cEdge reboots. After the reboot, it onboards as normal using the vBond, then vManage. The full configuration will be loaded at this time and join the fabric.
This is the Cloud vEdge after it has been onboarded. The organization, site-id, vbond DNS name, and certificate are installed, and the token has been invalidated. There are two control connection transports listed below. Pay attention to the VS/VM number, this indicates over which transports the vEdge/cEdge has established DTLS tunnels to which vSmarts/vManages.
Did I cover all the use cases involving the vBond Orchestrator? Do you have more questions? Let me know. Next time, we’ll deep dive with the vSmart Controller. As always, thanks to The Dustin for minor edits/fact checking, and also a big thank you to Dan Dib for proofreading.
