The year is 2023: OpenAI is on top of the world, riding its ChatGPT wave with revenue and valuations soaring. But under the surface, a tension began to boil – as leaders at the startup wanted to shore up access to high-end hardware to power its AI revolution.

It’s an idea that’s long been in the works. From previous mulling over potentially buying Cerebras in the Elon Musk days, to working with Microsoft on its custom Athena project, OpenAI has long sought chip-dependence.

This week it got its wish, signing a deal with Broadcom to jointly develop custom GPUs. The ChatGPT developer will design its own GPUs with Broadcom providing assistance as well as its networking solutions for OpenAI’s racks.

Despite the anticipation regarding the project, it was humorous when one Hock Tan, Broadcom’s CEO, may have let the cat out of the bag early.

Alexander Harrowell, principal analyst for advanced computing at Omdia, noted that during the chipmaker’s recent Q3 earnings call, Tan said its latest Tomahawk switch was deployed “with OpenAI, sorry, open Ethernet.”

The plot thickens, then, when during that same earnings call, Tan told analysts that Broadcom had bagged a mystery $10 billion customer. However, Charlie Kawwas, the chip firm’s semiconductor solutions group president, told CNBC that OpenAI wasn’t the mystery customer.

“I would love to take a $10 billion [purchase order] from my good friend Greg [Brockman],” Kawwas said. “He has not given me that purchase order yet.”

Why Broadcom?

Despite not being on the receiving end of $10 billion, though, Harrowell told SDxCentral that the deal was “good business” for Broadcom.

“OpenAI gets an alternative to the GPU vendors, a product it can tune exactly as it wants and it can also reduce the amount of margin that gets stacked into its costs,” the analyst said. “Nvidia’s gross margin is over 75% and Broadcom’s is in the thirties. That on its own is good for the unit economics and relieves the 'Premier League' effect where all the money goes to the (chip) players.”

OpenAI’s decision to join up with Broadcom makes sense on paper.

The chip giant has a history of working with big-name firms developing custom hardware for AI, given its prior support for Google’s development of its Tensor Processing Units (TPUs).

That alone lines up considering that OpenAI’s hardware team is made up of engineers who worked on Google’s TPU projects. Richard Ho, who previously led TPU development, now heads up OpenAI's hardware team.

The AI startup is believed to have a team of some 40 engineers working on its custom silicon project. Its latest addition saw another former Google TPU engineer in Safeen Huda, jump across. The new hire had previously worked on hardware-software co-design and modeling of Google's TPUs, and also brought with him experience working on Intel’s eFPGAs.

The rationale behind OpenAI’s Broadcom team-up adds to suggestions that the pair have been quietly working behind the scenes on this project for some time.

Reports surfaced in February that a partnership was at play, with TSMC’s 3-nanometer technology set to form the basis of hardware that would be produced beginning in 2026.

Previous reporting also suggested that those very same chips would feature a systolic array architecture with high-bandwidth memory not too dissimilar to Nvidia-developed GPUs.

While OpenAI has moved to designing its own chips to shore up its supply chain in a bid to keep its crown, for Nvidia, the deal comes as rivals are circling.

Prior to the Broadcom deal, Nvidia watched as long-time partner OpenAI struck a seismic deal with bitter rival AMD.

While OpenAI would also sign a new agreement with Nvidia, its long-awaited bid to diversify its chip supply and subsequent signing on with AMD and now Broadcom has helped to shape what looks to be a period of intense competition in the semiconductor industry.

“AMD’s own deal with OpenAI is a stamp of approval of their credibility and their roadmap is very strong,” Harrowell told SDxCentral. “But the two of them have to worry that the total GPU profit pool is threatened by the custom ASICs, especially because so much of the profit pool comes from a relatively small number of hyperscalers and cloud builders.

“It’s analogous to the way Intel has to both compete tooth and nail with AMD over x86 CPU orders, but they both have to worry about the x86 profit pool leaking away to Arm, Ampere, and custom Arm CPU projects,” Harrowell added.

In essence, OpenAI's multipronged chip strategy may have come out of a desire for sustainable access to high-end hardware. But it also signals a maturation of the AI industry's hardware landscape.

Rather than betting everything on a single vendor, the ChatGPT maker is hedging its bets across multiple suppliers while simultaneously developing proprietary silicon, a familiar pattern for anyone watching the cloud computing space over the past decade.

Larry Dignan, editor in chief at Constellation Research's Constellation Insights, captured this view in an analyst note that with its Broadcom deal, OpenAI is “following the hyperscale cloud playbook for AI that includes big doses of Nvidia as well as custom-made GPUs such as Google's TPUs and AWS' Trainium.”

It seems that Altman got his wish for OpenAI to have its own hardware. And with the stakes of the AI market (and its subsequent bubble) growing ever higher, it’s safe to say the era of single-vendor dependence is gone. In its place is a more diversified and competitive chip ecosystem.