Network engineers like redundancy. It’s not that we just want double of everything – we want the networks we design and manage to be super fast, super smart, and super resilient. In the LAN and in the data center we’ve been logically joining network switches using technologies such as Cisco StackWise, the Virtual Switching System and Virtual Port Channels with fabric extenders in order to consolidate control and data plane activities and provide greater fault tolerance, easier management and multichassis etherchannel for path redundancy. These are great benefits, but they can be reaped only by proper design. Otherwise, an engineer may introduce more risk into the network rather than make it more resilient.

We can stack fixed form factor switches such as the Catalyst 2960X, 3750X and 3850X to unify the control planes of all the stack members. An end-host connects to two separate devices in the stack which appear as one logical switch. This provides an etherchannel link spanning multiple pieces of hardware. Therefore, StackWise uses a shared control plane design to provide a switch engine redundancy and link diversity for the end host. This is commonly used in IDFs for access layer port density but also in building MDFs to dual-home downstream switches or servers.

Similarly, VSS merges the control plane of the 4500 chassis switches as well as the 4500X fixed form factor switches. The 4500 chassis switch already has dual supervisors and multiple power supplies, so the addition of VSS to logically join two of these large chassis switches provides some serious fault tolerance. To an extent, VSS on the 4500X platform works like StackWise with the active switch managing both devices. These platforms lend themselves to be used in the distribution layer or in a collapsed core.

vPCs don’t by themselves merge the control planes of Nexus switches, but when used with Nexus fabric extenders, a network engineer can consolidate the control plane  of all top of rack switches in a row or entire data center in one single device (or pair of devices).

The heart of what these technologies do is centralize the control plane. The benefit of increased hardware and link fault tolerance is very clear, but the drawbacks are often overlooked.

When consolidating the control plane of multiple switches onto one switch, the switch engine will always be on board some piece of hardware that’s also forwarding traffic. In the case of the 4500 chassis switches, the supervisor engines take up a couple slots in the chassis, but in the case of all the other switches, the switch engine is itself also a fixed switch. This means that a failure will likely result in the redundant links to many end-hosts going down. This is the whole point of link diversity, but it presupposes a correct overall design. 

Consolidating the control plane can increase the failure domain.

This sort of fate sharing can be dangerous in the LAN and data center core. The design requires significantly more planning to actually produce the resiliency a network engineer desires. For example, are all end-hosts or downstream switches dual-homed? If not, a switch stack or VSS won’t help. Are the pair of Nexus 5548 parent switches each powered by the same circuit? Since FEXs send all their traffic to the parent switch for forwarding and policy decisions, the loss of the parent switches due to a power failure would stop the data plane for every single FEX associated with them.

The real key is to allow traffic in the data plane to continue in the event of a control plane failure. Today, vendors are investigating the disaggregation of the networking operating system from the underlying hardware. This would consolidate the control plane on one device (or HA pair) that doesn’t pass traffic itself. In the event of a failure of the primary and failover controllers, the data plane would remain mostly unaffected while network engineers repair the problem. There are still questions as to how MAC learning and link layer protocols such as LLDP would work, but this approach completely changes the dynamic of fate sharing in the LAN and data center. Proper planning and design is still necessary, but the technology would provide out of the box fault tolerance engineers want in their networks.

Using the latest version of StackWise (Stackwise-480), VSS, and Nexus fabric extenders makes sense in building IDFs and data center racks  in order to provide cheaper and easy-to-manage port density, but when designing large building, campus and data center cores, consider if there really is a significant increase to network resiliency. Maybe the risk isn’t really worth the benefit, or maybe the design itself undermines the technology. Until we move to a disaggregated switch design with a pair of intelligent controllers managing dumb hardware all over campus, engineers should be careful not to introduce more risk into their networks rather than make them more resilient.

]]> https://networkphil.com/2016/03/18/fate-sharing-in-the-network-core/feed/ 2 699 pgervasi Untitled https://networkphil.com/2016/01/18/short-and-sweet-cisco-vss/ https://networkphil.com/2016/01/18/short-and-sweet-cisco-vss/#comments Mon, 18 Jan 2016 21:15:56 +0000 http://networkphil.com/?p=390 Continue Reading →]]> This post is a short, sweet and to the point copy/paste resource for configuring Cisco’s Virtual Switching System.

Cisco’s Virtual Switching System (VSS) is a pretty common technology that joins two physical Cisco Catalyst switches into one logical switch thereby joining the two data planes. It’s different from stacking switches because you can use regular ethernet cables rather than stackwise cables and modules. The two technologies are otherwise kind of similar in the benefits they offer.

VSS limits you to only two switches in a cluster (switching system), and is used for the 4500, 6500 and 6800 platforms. This is then a great technology to build chassis redundancy at a large distribution or core level. In my mind one of the biggest benefits is multi-chassis etherchannel which provides both link and hardware fault tolerance for your switch uplinks or servers.

Below is an easy copy/paste config you can grab quick rather than dig through a white paper. VSL stands for “virtual switch link” which is the link for VSS traffic between the two switches. This example is from a pair of 4500X switches I put in not long ago and uses 4 port port-channel for the virtual switch link. There will be a bunch of QoS config that gets auto-generated as part of this brief process, and I’d personally tell anyone who asks that you did that part yourself manually and from memory.

Switch 1

conf t
switch virtual domain 10
switch 1
exit

int port-channel 5
description VSL for VSS
switchport
switch virtual link 1
no shut
exit

int range ten1/1 – 4
description VSL port for VSS
switchport mode trunk
channel-group 5 mode on
end

switch convert mode virtual

Switch 2

conf t
switch virtual domain 10
switch 2
exit

int port-channel 10
description VSL for VSS
switchport
switch virtual link 2
no shut
exit

int range ten1/1 – 4
description VSL port for VSS
switchport mode trunk
channel-group 10 mode on
end

switch convert mode virtual

Verification

sh switch virtual
sh switch virtual role
sh switch virtual link
sh switch virtual link port-channel