Posted on by & filed under Content - Highlights and Reviews, Information Technology, Tech.

By Ron Fuller

Ron Fuller is a staff engineer in the Network and Security Business Unit (NSBU) focused on NSX for VMware. He has 21 years of experience in the industry, and has held certifications from Cisco, VMware, Novell, HP, Microsoft, ISC2, and SNIA. His new video series “IP Multicast Fundamentals” is now available in Safari.

Multicast – the very mention of the word sends chills down the spines of many network engineers. While this often misunderstood technology isn’t as widely deployed as was once hoped, it is most certainly out there “in the wild.” Multicast is running critical applications in financial networks, providing heartbeats to load balancing systems, and also providing “on hold” music for many customers. Multicast is also seeing a resurgence in modern data centers where customers house their most mission critical applications and where a simple mistake can cost untold amounts of financial losses. What use case is causing customers to consider adding to, or building new multicast capable networks? The answer is an emerging protocol called Virtual eXtensible Local Area Networking (VXLAN).

VXLAN is an industry standard protocol described in RFC 7348 that is designed to address some very specific challenges associated with large L2 networking domains. Customers have fought with the restrictions and considerations with having broadcast domains that span vast swaths of their data centers. They have tried technologies like the Spanning Tree Protocol (STP) guard mechanisms (BPDUGuard, RootGuard, LoopGuard, etc) with varying levels of success. They’ve evolved their data center networks to minimize STP with protocols like virtual Port Channels (vPC), and many have eliminated STP with technologies like FabricPath. All of these tools have pros and cons but the ultimate goal for many is a fast, scalable, L3 routed fabric. So why not just deploy that and pass out high-fives all around? Server virtualization and the requirement for workloads to move around the data center. This single requirement is what drives many customers to keep their large L2 domains – until recently that is.

Now, with VXLAN customers can have the best of all worlds – fast, scalable L3 fabrics and L2 connectivity using the VXLAN protocol to transport their L2 frames across the L3 fabric. It does this by using VXLAN to encapsulate the L2 frame and wrap it with a User Datagram Protocol (UDP) header and marking the original source network as a Virtual Network Identifier (VNI) for transport across the L3 fabric to be de-encapsulated at the receiver and passed on as a native frame to the recipient. Neither the sender nor the receiver need to know anything about VXLAN, it is transparent to them. Pretty nifty, eh?

You might be saying to yourself “Cool story, Ron. Where does multicast come into play?” Great question, now that the background has been provided we can discuss the intersection of these technologies. RFC 7348 refers to the need for the overlay technology to provide the ability to deliver a broadcast frame to all hosts on a common VNI. How can this be accomplished? The underlying network is a L3 routed fabric and a review of IP fundamentals tells us that L3 devices stop broadcasts. The developers of VXLAN made a decision to use existing IP technologies that could help here, namely IP multicast. Multicast already has the ability to send a single frame from one source to many receivers and almost all modern networking equipment supports enabling multicast. Problem solved!

While it is easy enough to say implement multicast in your data center, the reality is quite different. Some planning must be done to understand the requirements and ensure the deployment goes as smoothly as possible. Do we just need simple sparse mode multicast configured? How much multicast state which includes the multicast routing table comprised of entries like *,G (pronounced star comma G), source to group mappings like S,G (pronounced S comma G) and Internet Group Management Protocol (IGMP) tables can your network devices handle? Does this impact convergence and if so, is that acceptable? There are a myriad of design decisions that must be asked and answered to ensure a successful implementation.

The VXLAN standard even offers a suggestion for those who read it all to consider a technology like Protocol Independent Multicast (PIM) BiDirectional (BiDir). PIM BiDir is a variant of the routing technology for multicast and its primary difference is that traffic flows along a shared tree which has it root at the rendezvous point. This is different from traditional sparse mode PIM networks where traffic can be switched to a shortest path tree. So why is BiDir recommended for VXLAN? There are two primary reasons:

1 – Simplicity – most data center networks have straight forward topologies. Perhaps a spine leaf design or a traditional core, aggregation, access topology which lend themselves nicely to a clear routing path.

2 – Scale – because BiDir is rooted at the rendezvous point the amount of state needed to support multicast is minimized which is a typical sparse mode implementation could be challenging.


If you are new to multicast this may sound daunting but relax it is pretty easy to deploy BiDir. For both IOS and IOSXE the only difference beyond the typical “ip pim sparse-mode” and “ip pim rp-address w.x.y.z” command is this “ip pim bidir-enable.” For NX-OS configuring BiDir is as easy as adding the bidir keyword to the end of the “ip pim rp-address w.x.y.z” command so that it now reads “ip pim rp-address w.x.y.z bidir.” One thing to keep in mind with NX-OS and BiDir support though is that some platforms do not support running PIM BiDir across a virtual Port Channel (vPC). Many Nexus designs utilize vPC so you’ll want to check the release notes on to see if your version of NX-OS supports this capability or not.

So as with all things in technology, what is old is new again and a protocol suite that has not been implemented ubiquitously in many data centers, now can be used to provide next generation services. If you are considering using VXLAN in your environment, multicast support very well may come up as a topic. The more prepared you are, the better your chances for restful nights and peaceful weekends. This article is the tip of the iceberg, but now you have some pointers in the right direction.

For more on multicast IP and VXLAN

You can view my new video series “IP Multicast Fundamentals” in Safari now. You can also read my book, NX-OS and Cisco Nexus Switching and watch my training video series called NX-OS Configuration Fundamentals LiveLessons” in Safari. You can also follow me on Twitter @ccie5851.

Tags: Cisco, IP, multicast, nx-os, vxlan,

Comments are closed.