Generic Ethernet input
– Getty Images

As data center operators build out their AI data centers, high-bandwidth, low-latency networks are just as important as GPUs themselves to keep AI training and inference running fast and reliably.

Three layers make up AI networking: scale-up, which connects hardware within a data center rack; scale-out, which connects racks within a data center; and scale-across, which connects separate data centers.

Here is how each AI networking layer works, the competitive landscape, and the many choices enterprises, hyperscalers, and cloud providers face as they deploy their AI networking infrastructure.  

What is scale-up networking?

Scale-up networking, which is increasingly critical for AI inference workloads, connects up to hundreds of accelerators within a single rack or across a few racks, enabling them to operate as a single, tightly coupled compute domain.

Nvidia's NVLink is a proprietary high-speed, low-latency GPU-to-GPU interconnect that currently dominates scale-up networking because Nvidia dominates the GPU market.

But Dell’Oro Group VP Sameh Boujelbene contends market forces are driving adoption of open alternatives, particularly Ethernet-based solutions, such as Ethernet for Scale-up Networking (ESUN) and Scale Up Ethernet (SUE), which are both overseen by the Open Compute Project Foundation.

In fact, according to Broadcom, more than five hyperscalers use Ethernet scale-up in production, while more than 10 vendor partners are building racks based on scale-up Ethernet.

To compete against NVLink, the UALink Consortium, a group of companies that includes AMD, Astera Labs, Intel, AWS, Meta and Microsoft, is building an open alternative called UALink.

It will offer similar performance to NVLink and is designed to be more scalable, but it’s not yet available, reminded Matt Kimball, vice president and principal analyst at Moor Insights & Strategy. In spring of this year, the UALink Consortium said it was targeting commercial deployments of UALink in 2027.

Organizations using non-Nvidia GPUs can also connect them using PCIe switches, but it’s a slower option, Kimball said.

Moving forward, the Dell’Oro Group forecasts that Ethernet will become the dominant scale-up networking technology by 2029. Boujelbene said Ethernet adoption is driven by three market forces:

  • The need for vendor diversity as AI clusters grow in scale and strain supply.
  • The proliferation of non-Nvidia GPUs and custom AI accelerators, which drive demand for open networking architectures.
  • Continuous improvements in Ethernet performance, congestion management, and reliability.

Scale-out networking

Scale-out networking connects up to hundreds of thousands of GPUs across multiple racks within a single data center, allowing them to operate as a large-scale AI compute cluster, Boujelbene said.

Scale-out has undergone a dramatic transformation. In 2023, InfiniBand dominated with more than 80% market share. By 2025, Ethernet became the dominant networking technology because of the same market forces that are leading Ethernet’s rise in scale-up: vendor diversity, demand for more open-networking architectures, and continuous Ethernet improvements, she said.

Ethernet options include Nvidia’s Spectrum-X, which is optimized for Nvidia hardware and software, RDMA over Converged Ethernet Version 2 (RoCEv2) and the emerging Ultra Ethernet Consortium (UEC) specification.

Network equipment makers that compete in the scale-out space include Arista Networks, Cisco, Dell, and Hewlett Packard Enterprise (HPE), Kimball said. Cisco and Dell, for example, support all three Ethernet standards in their network gear. On the silicon side, chipmakers like Broadcom, Marvell, and Cisco support RoCEv2 and UEC.

Scale-across networking

Scale-across networking connects hundreds of thousands to potentially millions of GPUs across multiple data center campus buildings or further afield across vast distances, enabling operators to build sizable geographically distributed AI compute clusters.

Scale-across networks are mainly built by hyperscalers and neoclouds that run large AI training workloads, Kimball said. Large enterprises might need them if they operate like hyperscalers. Vendors competing in scale-across are the same players as in scale-out, such as Arista, Cisco, and HPE/Juniper. Many hyperscalers use their own custom solutions as well, he said.

How should enterprises build an AI network?

According to Boujelbene, large enterprises should start with AI workload requirements when building out AI networking infrastructure, not networking technology.

They should take a phased approach: For scale-up, determine how much tightly coupled GPU compute is needed. For scale-out, build a high-performance Ethernet fabric with room to expand as AI demand grows. Then enterprises should evaluate the need to connect AI resources across multiple buildings, data centers, or cloud environments, she said.

Where is the AI networking market headed?

Scale-out networks make up more than two-thirds of today’s AI back-end networking market, while scale-up accounts for less than one-third, Boujelbene said.

But scale-up is projected to account for more than half the market by 2030, as data center switch spending for back-end networks approaches $1 trillion from 2026 to 2030, according to the Dell’Oro Group.

Bandwidth speeds will also double and then quadruple in the coming years.

Today, most switch ports are 800 Gb/s across the three layers: scale-up, scale-out, and scale-across. Switches with 1.6 Tb/s speeds have begun to ship in the second half of 2026 and will comprise more than half of the ports by 2027. Switches with 3.2 Tb/s speeds will start shipping in 2028 and 2029 and will account for the majority of the ports by 2031, Boujelbene said.

Overall, faster networks mean faster AI performance.

“The network needs to operate at 100% to maximize the utilization rate of the expensive GPU resources,” she said. “Therefore, increasing network speed would have a direct impact on AI application performance.”