Your AI infrastructure is likely wasting half its GPU power right now. The problem isn’t your GPUs; it’s the network. Traditional networks are built for moving data, leaving compute idle while packets travel. New solutions are emerging that allow AI networks compute directly within the fabric, which can boost GPU utilization by 5-10 percentage points and significantly cut operational costs.
For decades, we’ve seen vendors try to solve networking bottlenecks by just throwing more bandwidth at the problem. I’ve deployed everything from ATM switches in the ’90s to 100GbE in the 2010s, and the core issue often remained: the network was a dumb pipe. It connected endpoints, sure, but it didn’t participate in the workload. This changes with architectures like Cornelis Networks’ Active Compute Fabric, which embeds programmable compute, specifically RISC-V cores, directly into the network interface cards (NICs) and switches. This isn’t just a smart NIC offloading overhead; it’s real compute, right there in the wire.
Think about what that means for your AI and High-Performance Computing (HPC) workloads. Functions like KV cache acceleration and Mixture of Experts (MoE) routing, which typically hog valuable GPU cycles, can now run inside the network itself. We’ve watched clients pour millions into GPUs, only to find their utilization hovering around 40-50%. According to Cornelis CEO Lisa Spelman, their approach aims to “give you your GPUs back,” pushing utilization up by 5-10 points. That’s a massive return on investment for hardware you already own, or are about to buy.
Why AI Networks Compute Matters for Your Bottom Line
This isn’t just theoretical; it’s a practical shift with serious implications for enterprise AI adoption. When I started out installing structured cabling for Fortune 500 companies, the network was purely about connectivity. Then came VoIP, where Quality of Service (QoS) protocols like DiffServ and MPLS started adding intelligence. Now, with AI, the network needs to be an active participant. Cornelis, which spun out of Intel in 2020 from the Omni-Path team, has been building this capability. Their CN6000 SuperNIC, for example, is multimode silicon that can run either their proprietary Omni-Path protocol or RoCEv2 Ethernet. Even in Ethernet mode, it uses an internal encapsulation layer that maintains Omni-Path’s benefits like credit-based flow control and congestion management for low latency.
Here’s what nobody is talking about yet: this isn’t just for the hyperscalers. While companies like NVIDIA have their proprietary NVLink ecosystems, Cornelis is targeting the vast majority of organizations that need high performance but want to avoid vendor lock-in. Their scale-up Active Compute Fabric relies on open standards like UALink and ESUN. We’ve seen this play out for 30 years – proprietary systems often offer initial advantages, but open standards eventually win out for broader enterprise adoption and flexibility. If you’re building out your own AI infrastructure, especially for inference workloads, this is a critical consideration. You need options beyond a single vendor’s stack.
So, what should you do about it?
- Audit Your GPU Utilization: Use tools like NVIDIA-SMI or Prometheus exporters to get a baseline on how efficiently your GPUs are running. If it’s below 70-80% for active workloads, you have a problem.
- Evaluate Network Fabric Options: Don’t just default to standard Ethernet. Look into specialized AI network fabrics that offer embedded compute capabilities. Understand the difference between a SmartNIC for offloading network tasks and one with actual RISC-V cores for AI acceleration.
- Demand Open Standards: If you’re not locked into a specific ecosystem, prioritize solutions that leverage open standards like UALink or ESUN for scale-up interconnects. This prevents future vendor dependency and gives you more flexibility.
- Consult a Specialist: Understanding these new AI network architectures and how they integrate with your existing compute and storage is complex. We at CTS offer AI infrastructure consulting to help you navigate these choices and optimize your deployments without overspending.
The days of the network being a passive conduit are over. Your network needs to compute.
Source: Cornelis lands $205M to make AI networks compute, not just connect
Frequently asked questions
What is Active Compute Fabric?
Active Compute Fabric is an architecture that embeds programmable compute, like RISC-V cores, directly into network devices (NICs and switches). This allows the network to perform AI acceleration tasks directly, rather than just moving data between endpoints.
How does embedded compute in the network improve GPU utilization?
By offloading specific AI acceleration tasks, such as KV cache acceleration and MoE routing, from the GPUs to the network, the GPUs are freed up to focus on their primary computational roles. This can increase their utilization by 5-10 percentage points, making your existing hardware more efficient.
Is this technology only for large enterprises or hyperscalers?
While beneficial for large-scale deployments, the goal is to make these efficiencies accessible to a broader range of enterprise, government, and academic customers. Solutions are emerging that support open standards, offering alternatives to proprietary ecosystems and making high-performance AI infrastructure more widely available.
Can these new AI networks integrate with existing Ethernet infrastructure?
Yes, some advanced network devices, like Cornelis's CN6000, are designed to be multimode. They can operate in standard Ethernet mode (e.g., RoCEv2) while still leveraging internal architectural benefits for congestion management and low latency, even when the wire protocol is Ethernet.
Related reading
- Stop 3 AI Network Automation Risks NOW
- Stop 2 Critical Cisco IOS XR Exploits Now
- Broadcom VMware: 3 Moves to Avoid Disaster
Ready to upgrade your technology?
Complete Tech Solutions designs, installs, and supports IT, cabling, security, and network infrastructure for businesses across Grand Rapids, West Michigan, and nationwide. Schedule a free site assessment and we’ll map out the right solution for your space and budget.
Learn more about our Consulting services.
