Data centers are increasingly straining under artificial intelligence (AI)-fueled data loads that are over powering the physical infrastructure put in place to handle more modest expectations, a scenario that fiber builder Corning thinks is about to hit an inflection point that will pave the way for a long-awaited shift away from legacy connection methods and toward co-packaged optics (CPO).

A recent McKinsey report predicts the “demand for AI-ready data center capacity will rise at an average rate of 33 percent a year between 2023 and 2030 in a midrange scenario. This means that around 70 percent of total demand for data center capacity will be for data centers equipped to host advanced-AI workloads by 2030.”

Benoit Fleury, director of CPO at Corning, notes that this demand is quickly turning the tide on how data center operators view CPO.

“AI has really sparked a huge need for increased data processing, which is pushing the speeds and the bandwidth of the data networking inside data centers,” Fleury said, adding that “maybe the pain point was not high enough and now that has started to change.”

This traffic is currently handled by copper, which has enough history behind it to have worked out most manufacturing and deployment issues. However, Fleury notes that copper can’t continue to meet traffic demand.

“Copper has inherent limitations in how high data rates are handled over distance, including within data centers,” Fleury said. “As AI is accelerating the speeds of transmission, accelerating the capacity of the switches, and the density that goes along with that, copper starts to become really less and less viable in terms of efficiently transmitting those data rates even inside the box.”

CPO is not a new technology, but its capabilities have been overshadowed by an already established base of copper-based intra-data center connectivity.

“Our current forecast for CPO doesn’t show significant volume adoption within our forecast period,” Dell’Oro Group noted in a report that was released in early 2023 and with a forecast range through 2027.

AI is having an impact

However, AI is changing that narrative.

Fleury explained that as the speed of and demand of data hitting the GPU starts toward 200 Gb/s, “it starts to become very difficult [to transmit] that effectively using copper traces.”

“The idea then is to move that optical-electrical conversion much closer to the chips inside the box,” Fleury said. “In essence, the function of the transient function, which is what in both directions converts optical to and from an electrical signal, is still required but it's just done much, much closer in. Instead of being in the form of a pluggable transceiver on the face plate of equipment, it now becomes a photonic IC, or a chiplet, which is co-packaged with the target IC on the same substrate.”

This also can support higher densities the front plate of a server are now just optical-to-optical connectors instead of needing larger pluggable transceivers.

Others have also pointed to the ability for CPO to produce a meaningful reduction in power consumption.

“To the extent we can save 30 or 50 percent of the interconnect power by moving from traditional optics to CPO, that’s the driving force,” Kevin Deierling, SVP of networking at Nvidia, recently told DCD. “We're talking about 10s of megawatts of power savings on a giant data center.”

Despite the advantages, CPO is still challenged by manufacturing and deployment concerns. CPO is constructed using a large number of optical fibers densely packaged within cables. Fleury said that Corning was using 32 fibers in each cable.

“There's an awful lot of fibers that are packed pretty tightly, but pretty neatly, into this,” Fleury said.

That tight package also means that proper deployment is essential, especially, “if the volumes are very high, and especially if these are assembled by folks that may not have a lot of experience handling fiber, because today, inside the box, there is no fiber.”

“You want to make sure that this is well thought out in terms of laying out the fibers,” Fleury said. “You don't want to bend them too tightly, because then there's going to be some degradation effects, maybe even some reliability effects. And so, all the thinking that goes behind this and how you do this so it's easy to assemble, it's easy to service if you need to replace maybe just one of the cables without disrupting the adjacent assemblies.”

The industry’s work toward tipping the CPO value ratio is starting to gain traction.

Corning is working with data center silicon vendors like Nvidia and Broadcom to better integrate CPO into those chip platforms. This includes support for Nvidia’s latest Quantum-X and Spectrum-X Photonics and Broadcom’s latest third-generation CPO systems.

While CPO is viewed as the next step in intra-data center connectivity, Fleury also pointed to ongoing work with more advanced glass technologies.

Corning is also working on next-generation connectivity options that include using a glass substrate to act as a bridge between fiber connections and photonic ICs. This involves replacing the current silicon substrate with glass that can house multiple chips. That glass can in turn be inscribed with wave guides to act as the connections between the fiber and the chips.

Fluery said this model can support higher densities “right at the photonic IC in ways that even very thin fibers cannot really address.” This includes those fiber production and deployment challenges, “because it’s something you can do at very high scale and in a way that is relatively much easier to assemble.”