This blog is a summary of the session, moderated by WWT’s Chief Technical Advisor Brian Glibert  at the recent AI Networking Summit in NYC. To view the complete presentation you may visit here. 

The rise of Artificial Intelligence, particularly Large Language Models (LLMs) and autonomous agents, is generating a “data tsunami” that threatens to overwhelm existing enterprise networks. As companies shift AI workloads from the cloud back to on-premises data centers for cost and control, they are confronting a massive technical debt. This recent panel discussion explored the complex network re-architecture required to support this new reality, touching on core hosting, the middle mile, and the network edge.

The Core: The High-Performance Data Center

The traditional data center network is ill-equipped for AI. Unlike typical enterprise traffic, AI traffic is fundamentally multicast, meaning one source transmits to multiple receivers simultaneously and demands unprecedented bandwidth. Current network architectures simply cannot handle the load, leading to a dramatic shift in core hosting strategy.

We are seeing a rapid push toward 800G and 1.6T network speeds. This capacity increase is essential for minimizing job completion time (JCT), the critical metric for AI training. This pursuit of performance has fueled the “InfiniBand vs. Ethernet” debate. While InfiniBand was initially preferred for its low latency, the conversation is rapidly shifting toward optimized Ethernet.

The emergence of standards like Ultra Ethernet (UEC) signals a clear future for Ethernet. Our testing shows that high-speed Ethernet can now match or exceed the performance of InfiniBand at massive scale. Crucially, Ethernet offers a more flexible investment, as equipment can be repurposed for non-high-performance use, extending its lifecycle. For enterprises, vendor lock-in with a single InfiniBand provider is a significant risk, making a standards-based, scalable Ethernet approach far more palatable for long-term planning. The focus has moved from simple latency to achieving a “lossless fabric” with low jitter, ensuring that all data packets arrive consistently and minimizing the need for costly retransmissions.

The Middle Mile: Beyond the GPU Farm

The challenge extends far beyond the core cluster. Enterprise users, scientists, and AI agents are rarely sitting right next to the GPU farm. This introduces a critical need for efficient “middle mile” connectivity—from the data center to the metro, the cloud, and machine-to-machine.

The key to navigating the middle mile is what we term “Model Routing.” The network must become highly intelligent, deciding where to process data based on context and cost. For example, processing a massive video stream at the edge for image recognition, and only sending a small alert back to the core, is vastly more efficient than shipping the entire video feed across the globe. This approach minimizes data movement, turning the problem into a caching issue—where to cache contextual data to serve the local AI model most effectively.

The Edge and the Security Crisis

The true scale of the network re-architecture is yet to hit the edge, but it is imminent. The widespread deployment of Small Language Models (SLMs) and Ultra-Small Language Models (USLMs) on devices, phones, and local servers will put immediate pressure on campus and branch networks.

While the network is not yet a broad blocker for AI deployment, it will be soon. Enterprises must start planning today, aligning network depreciation cycles (the typical 7-10 years for core, 3-5 years for campus) with the expected arrival of full-scale AI agent workloads. This means proactively upgrading Wi-Fi infrastructure, considering 5G at the edge, and solidifying the ‘scale across’ design—the new, intelligent routing between campus, data center, and cloud.

Finally, no AI network architecture is complete without addressing security. The AI attack surface is exponentially expanding, with new attack vectors becoming cheap and easy to deploy. The network must be redesigned not just for performance, but as a robust, intelligent defense layer that can protect state and manage agents across increasingly complex trust boundaries. The future of the enterprise network is not just about moving packets faster; it’s about intelligence, efficiency, and integrated security.