For years, data center design has been driven by compute density, virtualization efficiency, and east-west traffic patterns dominated by many small flows. That model breaks down entirely when you step into AI training environments. What you’re building now isn’t just a traditional data center. It’s a distributed, synchronized compute system where the network is a first-class component of performance.

The Network Is the System

In large-scale AI training, thousands of GPUs operate in tightly coordinated steps. They compute locally, then exchange gradients, weights, and parameters in synchronized bursts. This is the core of the workload. 

That means traffic is periodic, synchronized, and massive. Flows are not small and random (they’re elephant flows by design), and the network directly impacts job completion time (JCT). And, especially for AI training, JCT is the metric that matters.

In an AI training data center, we’re talking about collective patterns like all-reduce, all-gather, and parameter sync. Because training is synchronized, performance is dictated by tail latency and not averages. A single straggling flow or congested path can delay the entire training run across thousands of GPUs.

These fabrics typically rely on simple, scalable control planes, usually BGP with ECMP, to provide predictable multipath forwarding, though convergence behavior has to be carefully considered at this scale.

Think about the incredible cost of GPUs today. At scale, where individual GPUs can cost tens of thousands of dollars, even small inefficiencies translate into millions in lost utilization. The key is making sure GPUs never have to wait on the network; otherwise you’re losing performance, extending training time, and burning money. 

Switching Architecture at Scale

In data centers built for AI training, the dominant architecture is still leaf-spine, but the design principles are different.

First, symmetry is non-negotiable. Equal-cost paths must truly be equal which means identical bandwidth from leaf to spine, consistent link counts across tiers, and minimal variance in latency. 

Second, oversubscription is effectively eliminated in the training fabric. In more traditional, even some high performance computing data centers, we could get away with a 3:1 or 5:1 oversubscription. AI clusters trend toward 1:1. This is because when thousands of GPUs synchronize and transmit simultaneously, even slight oversubscription creates congestion, head-of-line blocking, and stalled computation. And though non-blocking 1:1 is the ideal for AI training fabrics, in practice some architectures introduce carefully controlled oversubscription beyond a single pod or cluster boundary.

Third, radix matters more than ever. High-radix switches reduce the number of tiers, which reduces hop count, latency, power consumption, and of course, operational complexity. In other words, the more ports per switch, the flatter and faster the network will be. That translates to reduced latency and power. 

There are inherent tradeoffs between fixed form factor and chassis switches, Clos and butterfly designs, and so on, but the underlying goal is always the same: make the network as flat as possible and as fast as possible.

Now, keep in mind this is conceptual because at scale, these fabrics are often built as modular units (pods or clusters) with well-defined boundaries to contain failure domains and maintain predictable performance.

Elephant Flows and Microbursts

Traditional data centers deal with many short-lived flows, whereas AI data centers do not. Instead, typical traffic during training runs in an AI data center is long-lived, high-bandwidth elephant flows in synchronized bursts at line rate. On top of that, you still have to deal with incast and microburst conditions.

TCP incast is where multiple, synchronized servers send data to a single receiver at the same time which causes switches to drop packets. In an AI data center much of the traffic is RDMA (RoCEv2), which means incast still happens, but it’s different under PFC and ECN. Instead of pure packet drops, you’ll see an increase in pause events and queue buildup. 

This creates two major problems. 

First, hash polarization, where large elephant flows are pinned to a subset of ECMP paths due to limited hashing entropy, can lead to uneven link utilization. 

Second, buffer pressure and microburst loss can happen when queues fill in microseconds. 

Even a single congested path can stall an entire training run, which is why modern fabrics increasingly rely on advanced load balancing, congestion-aware routing, very precise queue management, and in some designs packet spraying, which can be controversial because of potential issues with RDMA behavior.  

Lossless Ethernet and Congestion Control

AI fabrics are pushing Ethernet into territory historically (but decreasingly) owned by InfiniBand. To support GPU-to-GPU communication, networks are designed to be effectively lossless, using technologies like: 

• RoCE (RDMA over Converged Ethernet)
• Priority Flow Control (PFC)
• Explicit Congestion Notification (ECN)

These mechanisms shift the behavior of the network so that drops are minimized, congestion shows up as queue buildup and pause frames, and latency becomes the dominant signal. But it does also introduce some challenges. 

While no Ethernet network is truly lossless, these fabrics are engineered to minimize drops under expected conditions using mechanisms like PFC. However, this shifts the problem. Poorly tuned PFC can actually propagate congestion, introduce head-of-line blocking, and in worst cases lead to deadlock conditions.

Observability

At this scale, traditional monitoring is almost useless. Polling every 30–60 seconds won’t catch microbursts, short-lived congestion, or transient link issues. Instead, AI data center networks rely on high-resolution, streaming telemetry focused on the signals like queue depth and buffer utilization, PFC “time-in-pause” and ECN marking rates. 

Remember that the real failure mode in AI networks isn’t obvious outages. It’s silent performance degradation. A slightly degraded link can increase JCT by minutes or hours. And even worse, link failures can force checkpoint rollbacks, undoing large portions of a training run. 

Some of the important metrics we need for an AI data center include:

• Queue depth
• ECN events
• PFC events
• FEC correction
• Link-level retransmits
• NIC-level telemetry 

Designing for Job Completion Time

Everything in an AI data center ultimately ties back to JCT. You’re not so much optimizing for link utilization, average latency, or throughput in isolation. Rather, you’re optimizing for maximizing GPU utilization and minimizing communication overhead.

That leads to a different set of priorities. For an AI data center, the goals are to eliminate unnecessary hops, minimize retransmissions, and treat observability as a first class citizen so we can prevent congestion before it occurs and remediate issues immediately. 

Even a 10–20% reduction in JCT translates directly into lower GPU cost, faster model iteration, and in business terms, a real competitive advantage. 

Beyond Networking

One of the biggest mistakes engineers make is isolating the network from the rest of the system. AI data centers are tightly integrated systems involving compute (GPUs, CPUs, accelerators), storage and data pipelines, the network fabric, and of course also power and cooling. 

The industry is actively evolving Ethernet to better support these workloads. Initiatives such as the Ultra Ethernet Consortium are focused on improving congestion management, transport behavior, and coordination between NICs and the network fabric, bringing Ethernet closer to the performance characteristics historically associated with HPC interconnects.

Remember that for AI data centers, you’re not simply designing a network. You’re building a system where:

• the network is part of the compute
• performance is measured in job completion time
• and small inefficiencies compound into massive cost

At this scale, architecture decisions really are business decisions and not purely technical. And the difference between a good design and a great one is measured in how fast you can turn GPUs into results.

Thanks,

Phil

It’s an interesting, ambitious, and I think logical idea that extends Cisco’s networking footprint into edge computing and AI infrastructure, which are arguably two of the fastest-moving segments in enterprise IT.

What is Cisco Unified Edge

Unified Edge isn’t just a server, at least not in Cisco’s public positioning. It’s a modular, converged system combining compute, storage, and networking in a single chassis. It supports both CPU- and GPU-based configurations for real-time AI workloads, with redundant power and cooling, SD-WAN connectivity, and pre-validated blueprints for specific industries.

The platform integrates with Cisco Intersight for centralized management, ThousandEyes and Splunk for observability, and built-in Zero Trust controls down to the device level. Basically, Cisco wants to make managing hundreds or thousands of AI-enabled edge nodes as simple as managing Meraki devices.

Under the hood, Unified Edge builds on the UCS X-Series chassis with a modular backplane supporting high-bandwidth 25Gbps interconnects, optional GPU nodes, and Intel Xeon 6 processors optimized for native AI inference. Networking and compute modules communicate through an integrated fabric interconnect that allows for unified policy enforcement and telemetry collection without needing additional hardware appliances. The design adds to UCS X’s use-cases and applications and integrates directly with Cisco Intersight for lifecycle management, firmware updates, and fleet orchestration.

Cisco emphasizes agentic AI (which they define as multi-agent systems capable of autonomous decisions and collaboration between models) as a workload that will thrive at the edge where latency and data locality matter the most. In essence, Unified Edge is meant to give those agents the hardware and connectivity they need without sending every decision back to the cloud.

From a data pipeline perspective, Unified Edge supports model deployment through containerized runtimes using Kubernetes and Red Hat OpenShift, but also with the option to use Cisco’s Intersight Kubernetes Service (IKS). For AI frameworks, Cisco and Intel officially validated OpenVINO and TensorRT for low-latency inference, the idea being that it enables edge-resident models to run independently or as agents within distributed inference tasks connected back to cloud LLMs.

Each chassis includes Cisco’s Trust Anchor module for verified boot and hardware attestation, signed firmware updates via Intersight, and native segmentation using SD-WAN policies. Unified Edge also enforces per-application zero-trust profiles, integrating with Duo and SecureX for identity-based access control. This addresses the ever-present concern around compliance and protection even when deployed in semi-trusted environments like retail locations or healthcare.

One thing that’s interesting is how Cisco frames agentic AI. They explain that running an agentic AI system at the edge doesn’t simply mean running models locally – it means orchestrating multiple specialized agents (for example, a computer-vision model, a telemetry analyzer, and a policy agent) that collaborate in near-real time. Unified Edge’s low-latency inter-node communication and modular compute fabric allow these multi-agent systems to make distributed decisions, but at the same time maintain global context through a form of cloud synchronization.

According to Cisco, Unified Edge can reduce inference latency by up to 70% compared to cloud-round-trip execution, and lower WAN traffic by 40-60% for agentic workloads. Each chassis supports up to four AI accelerators and 400 Gbps aggregate throughput, which should be sufficient to run current high-density edge deployments.

Why this Matters to AI at the Edge

The timing makes sense since we’re all learning that AI inference traffic looks nothing like traditional workloads. It’s heavier, more continuous, and often requires sub-second response times. According to Cisco, AI agents generate up to 25X more network traffic than chatbots. That’s a big deal when trying to push inference to thousands of distributed locations.

Edge-optimized architectures like Unified Edge could help reduce latency and cost while allowing for real-time use cases like predictive maintenance on a factory line, automated fraud detection at bank branches, or personalized in-store experiences powered by local AI models. These are activities many larger, more technically progressive organizations already do, but with Unified Edge the idea is that more organizations than ever can take advantage of both traditional MLOps and modern AIOps workflows.

Is It Real or Just Marketing?

Technically, I think this is certainly viable, but it’s not really revolutionary. HPE, Dell, Lenovo, and even NVIDIA have been building edge AI nodes and converged systems for a long time. What Cisco brings to the table is not that it’s new technology, but that they provide tight integration and the fact that everyone in networking knows Cisco.

The question is whether enterprises want to buy AI-ready infrastructure from their network vendor, or continue blending specialized compute, storage, and networking products. Cisco’s unified approach may appeal to organizations that value simplicity and end-to-end support, which was definitely my experience with many enterprises over the years, but I think skeptics might see this as repackaging existing technology into a new acronym-worthy category.

That being said, the focus on agentic AI is definitely forward-looking. If enterprises begin deploying autonomous AI systems that need to act instantly and securely at distributed sites, a unified edge architecture like Cisco’s could be absolutely the best way to go. That’s why I think the success of Unified Edge will depend less on specific hardware, speeds, and feeds, and more on ecosystem execution like integrations, ease of deployment, and (of course) pricing. 

Conclusion

I don’t think Cisco Unified Edge is vaporware. It’s a thoughtful convergence of compute, networking, and security specifically designed for where we’re going with AI. Whether it becomes foundational to distributed AI or fades into tech history as not much more than a marketing play will depend on industry adoption, not press releases and announcements. 

But when we look out at the landscape of how organizations are actually implementing AI inference at scale in operational contexts, Cisco deserves credit for pushing the conversation beyond cloud-only AI. Cisco didn’t reinvent the server, but it’s one of the first credible architectures to merge networking, observability, and AI orchestration into a unified control plane which I believe is an important step if distributed AI is ever going to scale.

For further reading, here are a few of the resources I used for this post:

https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/unified-edge/unified-edge-solution-overview.html

https://www.cisco.com/site/us/en/products/computing/unified-edge/index.html

https://www.wwt.com/blog/cisco-unified-edge-powering-the-next-generation-of-intelligent-infrastructure

https://blogs.cisco.com/news/ai-isnt-waiting-for-the-data-center-the-edge-is-the-new-center-of-gravity

Thanks,

Phil

The conversation around these questions help me to figure out which SD-WAN solution is best, which hardware to go with, what sort of licensing to use, what the topology should look like, and even what the actual policies should be.

This is by no means an exhaustive list but more a conversation starter. If you have any initial planning questions or considerations you think should be on this list, please let me know in the comments, on Twitter, or on LinkedIn.

In my very earliest planning discussions with my customers, I like to ask these 10 questions:

1) What real-time applications are in use?
2) What are the latency thresholds for mission critical apps?
3) Is there a need for multicast?
4) Are there analog connections like PRIs, voice gateways
using analog circuits for failover, etc
5) How will SIP trunk terminations work in the the SD-WAN?
6) What is the org’s cloud footprint and future initiative?
7) What SaaS applications are in use?
8) What are the organization’s security requirements both as
as a whole and specifically for branch security?
9) What are the current providers in place and what is
available at each of the branches?
10) How will the new SD-WAN work with the
organization’s Disaster Recovery and Business Continuity
plans?

Thanks,

Phil

Below is a list of the technology-related podcasts that have been my go-to when driving the many miles I travel per month. Most are networking-related, but some cover information on security, cloud, virtualization, etc.

There are more out there, but these are the handful I always seem to gravitate to. I’d love to get your recommendations for podcasts I should be listening to, so please let me know in the comments.

Heavy Networking on Packet Pushers (networking)

A standard for the industry, the weekly Heavy Networking podcast from the Packet Pushers dives deep into networking technologies from both an engineering perspective and with a practical mindset.

Clear To Send (wireless)

A podcast devoted completely to wireless networking including shows discussing real-world scenarios, emerging wireless technology, and wireless networking industry news.

Network Collective (networking)

A collection of mostly technical podcasts centered on networking, whether that be technical deep dives into a beloved routing protocol, industry news, historical accounts, and career advice. Industry experts are brought in regularly to share their experiences, their knowledge, and their perspective on industry news.

Security Now (security)

Under the This Week in Tech (TWiT) umbrella, this podcast with Steve Gibson focuses on various aspects of information security including both consumer and enterprise, regulatory issues and industry news.

The CTO Advisor (virtualization, cloud, etc)

A series of podcasts with a distinct business-oriented perspective on cloud technologies, virtualization, big data, and the occasional career show. Keith provides a very practical perspective to applying technology in the real world.

Zigbits (networking)

A solid podcast on networking, careers, and emerging technologies led by Zig Zsiga, a high level networking pro, who takes deep dives seriously and discusses networking topics in the context of their real-world applications.

Virtually Speaking (virtualization)

A weekly show with industry experts focused primarily on virtualization with VMware.

On-Premise IT (variety of technology areas)

A shorter show, the On-Premise IT podcast brings in voices from the community to discuss a wide range of technology including networking, storage, cloud, emerging technology, careers, and industry news.

Cisco Champion Radio (Cisco networking)

A podcast series sponsored by Cisco and the Cisco Champions program on Cisco networking, emerging technologies, Cisco cloud, infosec, etc. Episodes are found on iTunes.

The Cloudcast (cloud technologies)

A podcast focused on cloud technologies including integration, emerging technologies related to private and public cloud, security, tools, and careers.