Throughout this series I will leverage screenshots and other source material that Cisco provides for the solution, including product documentation and Cisco Live sessions as well as material from the dCloud demo we are using for this series. Whenever I use non-original resources, I will provide a link to those resources at the end.
The Management Plane and You
vManage is the central point of management for all provisioning, configuring, monitoring, and troubleshooting for the entire SD-WAN fabric. Breaking down the functions of the vManage into its component pieces helps to bring that down from the high level to the low level, which is what we’ll focus on here.
First of all, vManage itself supports a multitude of options in terms of access. It supports a single or multi-tenant configuration, allowing service providers to consolidate virtual hardware while supporting multiple customers. It also supports role-based access control, so the NOC (or a customer, in the case of an SP) can monitor the solution without the having full control over the environment. vManage also supports programmability via a REST API for those who prefer to automate or customize the administration of the solution.
Onboarding (Day 0 Planning and Deployment)
Certificates and Trust
vManage manages the trust relationships between the different SD-WAN elements, as well as the certificate management itself. Remember that the Cisco SD-WAN (Viptela) solution is white-list-based, meaning that only trusted elements can participate. vManage ensures that the vBond Orchestrator, vSmart Controllers, and WAN Edge routers are conforming and participating in that trust relationship by keeping track of the devices, certificates, and their trust levels.
The vBond Orchestrator will handle the actual first-contact authentication for the WAN Edge infrastructure, and distributes the list of vSmarts and vManages to them as well, but in terms of managing that list and the trust relationship, this function belongs to vManage.
vManage can be used to control the level of trust afforded to a device and how much it will be able to participate in the fabric based on that level of trust. There are three levels:
Invalid: This is the default state when a device is added to the vManage (unless forced to become Valid as part of adding it). A device with a trust level of Invalid can not communicate with any of the SD-WAN elements or participate in the fabric.
Staging: In this state, devices can contact vManage, vBonds and vSmarts for purposes of downloading configurations and other onboarding.
Valid: In this state, devices can communicate with other devices marked as Valid, join the overlay fabric and begin forwarding traffic. This is the standard operational state of devices that have been onboarded and which are now participating in the solution.
This list of WAN Edge routers shows that all have been onboarded and are participating in fabric routing.
How the other devices are added to the vManage depends on what they are. The vBond and vSmart devices will need a minimal configuration in order to allow the vManage to connect to them, generate a CSR (certificate signing request), submit that CSR for signing, then finally install the resulting certificate on the device, thus adding it to the overlay network.
An example of how to use the vManage to onboard a vSmart controller automatically.
In the case of hardware WAN Edge Routers, Cisco is working on redoing the licensing at this time, but basically, it involves uploading a special file to the vManage which includes the serial numbers, hardware type and other details to the vManage. The hardware routers include a TPM/SUDI chip which is used for the certificate/trust relationship.
In the case of Cloud vEdge routers or CSR1000v, the process is similar, but the certificate/trust relationship is based on a token as there is no hardware chip to leverage.
Notice that, if desired, the hardware can be force-validated in order to allow it to immediately join the overlay network upon configuration.
Let me share a few Cisco Live slides that illustrate the trust/certificate relationship better. I’ll link to the session/presentation below.
Implementation (Day 1 Configuration)
The next part of vManage we will look at are the configuration/implementation sections. vManage utilizes NetConf/Yang models to orchestrate policy to the rest of the SD-WAN infrastructure, as shown below.
vManage uses programmatic languages to orchestrate policy instead of something more traditional like SSH or SNMP. The benefit is that as components change, as long as they understand the models there is no need to specify exact CLI commands. This allows Cisco to quickly adapt and bring new offerings to the solution.
Configuration Templates
vManage relies heavily on templates in order to scale and simplify the configuration of the SD-WAN infrastructure. The way it does this is by separating full device templates from the features that comprise them, and further by separating configurations which must be specific on a per-device basis from those which are the same across any device with that feature. I will devote an entire post to exploring the template creation and application process, but for now let’s focus on how vManage takes care of them.
Global or Local?
Each template has the ability to specify whether a particular setting is globally significant (any device with this template will set the same value) or if it is locally significant (Such as IP addressing and host names). Based on the setting, the vManage may need input in order to push configurations. In the case of settings which must be locally significant per device, such as IP addressing, there are two options to supply the needed information. When scheduling a template push with vManage, the needed information prompts will automatically appear. Alternatively, vManage can produce a CSV file based on a device template which includes the fields which are locally significant so that an administrator can populate the information.
Here is an example of a device template CSV produced by vManage. Notice each field has values tied to the specific device id.
Feature Templates
Feature Templates are likely to be your first step in planning the configurations for the SD-WAN infrastructure. These templates focus on things like OSPF, SSH, AAA, and other features which are part of a full device configuration. Feature templates are the building blocks which make up a device template.
Here we can see the feature templates and to what device types they can be applied. Since this demonstration is in dCloud, all WAN Edges used are vEdge Cloud.
Device Templates
Device Templates are either a collection of feature templates, or an actual CLI configuration. The two template types can’t be mixed, but if desired, the device templates support a more traditional method of simply pasting in CLI configurations. There is a Variable button which allows you to designate variables within the CLI template, which will need definition as noted above. If a CLI template is specified, the full configuration must be provided, not just a subset.
Here you can see we have a mix of CLI and Feature-based templates. Note also the device model and devices attached.
Policy Configuration
This is by far the largest, most complicated, and hardest to learn part of the entire SD-WAN solution. It would be pure folly to try and explain the entirety of how policy-driven routing and data plane configuration works in a single post, and so I plan to cover many use cases in the coming weeks and months. For now, let’s focus on how we do the configuration using the vManage web interface and how vManage pushes this policy to the elements of the infrastructure. There are two types of policy: centralized policy, which applies to the entire SD-WAN fabric, and localized policy, which applies only to devices to which they are attached, and only affect the local policy routing of those devices.
Centralized Policy
As stated, a centralized policy determines a consistent policy for all SD-WAN elements. Commonly, the policy works much like a device template, which is to say it is comprised of multiple sub-policies which, together, create the complete policy. This is explained in the slide below. There are two types of centralized policy: A centralized control policy and a centralized data policy. The data policy is what is done with traffic such as traffic engineering, or application-aware routing. The control policy determines routing and topology information on a per-VPN basis.
The sub-policies to combine to create a consistent master policy which applies to the entire SD-WAN.
Localized Policy
Localized policy also has two subsets, a data policy and control policy. The data policy affects how to affect data locally on the WAN Edge, such as QoS. The control policy affects local policy routing decisions such as setting BGP extended communities.
This CLI will only be applied to devices that utilize this policy in the device template.
When a policy is deployed, the centralized policy will be deployed to the vSmarts, the local policy to the WAN Edges directly (see Policy Orchestration graphic above) and then the vSmarts will use OMP to distribute the control plane to the WAN Edges based on the policy.
The resultant set of policies will determine the overall SD-WAN forwarding, using control policy, data policy, and app-aware policy (if any). See below how they are utilized:
In this slide, service side refers to the specific VPN (fabric) and transport side refers to the underlay transport.
Operations (Day 2 Monitor/Maintain)
The last piece of the vManage we will spend time on is how we monitor/maintain the SD-WAN solution without needing to resort to CLI commands. The SD-WAN solution is intended to be administrated with no CLI required, so many of the standard CLI-focused tasks are represented in vManage. However, if CLI control is desired, the vManage is built to facilitate that as well.
Tools
The Tools section covers most operational-level tasks you may wish to perform on the SD-WAN, including an SSH terminal, the ability to rediscover devices, and a set of operational commands that can be deployed directly from the vManage, including gathering technical data for TAC analysis.
From this menu, several operations commands can be deployed directly, including one to gather technical data for analysis by TAC.
The built-in SSH terminal allows you to check the reachability of any device in vManage as well as open SSH connectivity to it.
Some issues may be resolved by rediscovering and refreshing the device data in vManage for SD-WAN elements, if needed.
Maintenance
From this section, elements of the SD-WAN software strategy can be executed, from setting the default version for devices to upgrading and activating the software, to cleaning up old files. Device reboots can also be executed, independent of software upgrades.
Any element (or all elements) can have software loaded before activating so that proper maintenance windows can be scheduled after software has been deployed to the SD-WAN device.
If needed, devices can also be rebooted from the Maintenance menu.
Administration
This is the section where the vManage itself is managed. Besides setting up users with role-based access, the settings for the vManage and Cluster administration (if a cluster is deployed) takes place here.
The Settings section needs further explanation due to the sheer number of settings available, which we will discuss below.
All of the settings for the vManage itself and how trust is set up can be found here, as well as Cloud integration and vAnalytics.
Organization Name: Deceptively simple, but the Organization Name must match across all SD-WAN elements and certificates to ensure trust is properly established.
vBond: This is where the vBond is defined in vManage, not to be confuserd with the actual vBond device.. We’ll talk about it more when we cover the vBond Orchestrator, but suffice it to say that this is where the DNS name for the vBond and port number to use is defined.
Certificate Authorization: This governs whether the vManage will perform the CSR and cert install for the SD-WAN devices or if the certificate enrollment process will be done completely manually by the administrator. Notice that by default Symantec is the cert authority.
vEdge Cloud Certificate Authorization: For the vEdge Cloud, which has no pre-installed certificates and no hardware chip which carries a serial, a token system is used. For this reason, the certificate enrollment process is different. The vManage can act as the certificate authority, or can be done manually via an enterprise cert authority if desired.
Web Server Certificate: This one is almost self-explanatory. The vManage is web-based, and most browsers will not trust a self-installed certificate. If needed, the vManage can request a trusted certificate for the web interface to avoid browser errors.
Enforce Software Version (ZTP): If this is enabled, when a WAN Edge uses zero-touch provisioning to join the overlay network, it will automatically be upgraded or downgraded to the version specified.
Banner: This allows a banner to be configured which will display when someone logs into the vManage itself. Banners for other SD-WAN devices are configured under device templates.
Statistics Setting: This setting governs what statistics, if any, are collected. If desired, it can be filtered down to a per-device setting.
CloudExpress: We’ll have to tackle Cloud integration in another post, but this setting just determines whether or not the CloudExpress feature is enabled. For an enterprise with no cloud-based solutions, this would not be needed.
vAnalytics: Just like CloudExpress, this will have to be tackled separately. In short, vAnalytics is an optional feature which leverages cloud-based analytics. If not being used, it can be disabled.
Client Session Timeout: Sets a timer before logging a user out of the vManage web interface automatically.
Tenancy Mode: This setting defaults to single tenant, ie, one vManage instance. The alternative is multi-tenant which will fit a service provider’s goal of allowing multiple customers to have their own SD-WAN environment without having to build a vManage appliance for each customer. Of interest is the fact that once a vManage has been converted to multi-tenant, it can not be converted back to single tenant.
Statistics Configuration: This setting just configures how often the statistics which were configured above will be collected. The default setting is every five minutes.
Maintenance Window: Surprisingly, this does not seem to actually bring anything down for maintenance. Its use is just to provide an announcement for users of the vManage system that a maintenance window is forthcoming and how long it will last. There is no actual interruption of service caused by configuring a maintenance window in this way.
Statistics Database Configuration: The statistics database sizing can be configured here.
Users
The User Administration menu is where users and role-based access control is configured. There are three default group roles: Basic, Operator, NetAdmin. The default permissions are shown below:
Other groups may be added as desired, and, of course, TACACS and RADIUS can be leveraged for authentication/authorization/accounting as well as local users/groups. Configuring AAA for login is beyond the scope of this deep dive, but it is almost identical to the standard AAA for other Cisco devices. If you wish to read up on it, I’ve linked to the docs at the bottom.
Hopefully I’ve given a good insight into the role that vManage plays in the SD-WAN solution and some specific details about how it is configured. Next time, I’ll cover the vBond Orchestrator. Once we get through all the SD-WAN elements, we’ll start deep diving into the solution itself, such as templates and policies.
Did I forget anything? Is there something you wish we’d covered that wasn’t? Is there too much, too little? Let me know below!
Attributions
Viptela RBAC Documentation Source: Link 1 Link 2
Slides Source: Introduction to Next-Gen SD-WAN (Viptela) Architecture , CLUS 2018 Orlando by Mosaddaq Turabi.
The Dustin for being an excellent editor.
