The memory shortage. Component crunch. RAMageddon. Whatever you call it doesn’t matter; you’ve been affected by it in the past year. Your new PC parts and gaming consoles are more expensive than ever before. Budget and entry-level phones have soared in price. And your infrastructure project costs have likely never been higher.
Bean counting was never the sexy part of the coveted AI buildout many of us envisioned when we began this journey, but today, it’s more prevalent than ever.
Right now, if the digital infrastructure industry’s costs were on a graph, they’d belong to Disco Stu: “Record sales were up 400% for the year ending 1976, ‘If these trends continue… A-y-y-y!”
Poor Stu failed to grasp the concept of linear extrapolation, falsely assuming that a temporary trend would last forever. Disco music went on to crash harder than the crates blown up at Comiskey Park in July 1979 (we only do references for the young’uns here).
Now you might say, well, the memory prices will go the way of Stu’s rhinestone jumpsuit; things will be cheaper soon. What you fail to grasp is that things can get far worse before they get better. SK Hynix had thought memory woes would alleviate by around 2028. That project has now been pushed back to 2030. And even then, that’s a best-case scenario.
The reality is that supply chain crises will spill over. Already, we’re tracking electroabsorption modulated lasers used in optical transceivers as the next potential crisis, along with glass substrates for chip packaging, and programmable logic devices for networking and storage controllers. And that’s before we even bring in power, fiber connections, and future headaches like copper in the early to late 2030s.
If you think costs are biting now, just you wait and see. And what’s worse is the unintended consequences of regulations designed to improve the industry. A purported ban on Chinese optical transceivers – of which most of the big boys rely on to keep their high-speed networks running – would send costs for “trusted” optics skyrocketing. Do you really think Google or Microsoft are going to be happy that the transceivers they’ve used for years are suddenly unavailable, and the only alternatives available to them are way more expensive?
The industry is going to have to get comfortable with the hardware we’ve got, avoid pitfalls of unserviceable demands from higher-ups to have the next shiny thing, and get innovative. Meta’s storage rearchitecture, for example, is a testament to what engineering minds can do to beat rising costs while improving infrastructure efficiency. This writer also recently sat down with folks at Recursion, a drug discovery firm that uses Nvidia A100s and H100s to power their on-prem AI workloads. Hardware that, at least when compared to the apparent cutting edge of Nvidia today, is a ways behind, but for the “TechBio” it’s happily powering workloads that could truly benefit humanity.
Which is why when OpenAI’s Sam Altman chastises the human race simply for having RAM at home in their PCs and not in his data centers, the whole worldview of the actual shortage gets worse.
Optics are everything, and at a time when the AI market rests precariously, more so now than ever, getting the public onside might just be an extra pillar of support. We only need to think back to the pandemic-induced semiconductor shortage to feel the wrath of folks unable to get their hands on a PS5 or a new car. And when the inevitable step for GTA VI to come to PC in 2027 comes, are we really going to deprive them of that just to keep your agent spinning that little bit more? Heck, you know you’ve said something silly if even Elon is joining the pile-on.
The ultimate threat to the AI bubble then is not Nvidia overpromising on performance or the wider industry’s shenanigans of the rather dubious financial merry-go-round; it’s fundamentally having access to components to keep that money train chugging. And if you think that’s going to be affordable for gamers and IT leaders alike in the coming years, then you may as well be the fish trapped in Disco Stu’s platform shoes.
Comments