Few online phenomena have divided online opinion more than 100 men versus a gorilla or that darned dress (it was white and gold). But in late March, it was the turn of a Google-authored paper on an advanced compression algorithm. While it didn’t induce as seismic discussions as the aforementioned examples, to some, it caused a slight stock crash among memory makers and even caught the attention of certain tech CEOs.

This is, of course, in reference to TurboQuant, an algorithm capable of reducing the memory footprint of AI systems by compressing the key-value (KV) cache.

Some read the paper and related coverage as the dawning of a new age of compressed model weights that could smash the memory wall. Others were naturally skeptical.

The reality is a bit murky, with the industry looking for any such means to get the better of the ongoing memory shortage.

But first, what is TurboQuant?

Despite going viral in March, TurboQuant was originally detailed in an April 2025 paper, emerging ahead of being presented at ICLR 2026 in Brazil following a Google blog post detailing its powers.

It’s an algorithm designed to reduce the memory footprint of AI systems by compressing the KV cache, a process technically known as KV cache quantization. Per the original paper, it’s designed to be lightweight to ensure it’s suitable for use supporting online applications, and also “highly accelerator-friendly.”

It compresses AI models while maintaining their core makeup, without the need for preprocessing or specific calibration data. More technically, it uses a “vector quantizer” that’s paired with a one-bit quantizer, which results in what Google’s researchers describe as a “unbiased and low-distortion inner product quantizer.”

The benefit of such an algorithm is simple: reducing key-value bottlenecks without sacrificing model performance. Rather than building network systems to enable utilization and intricate compute express link interconnects or hybrid memory architectures, the team at Google instead views it as a "keep calm and carry on" alternative to the memory crisis, allowing AI model developers to minimize KV memory footprints.

After bouncing about on computing nerd social media forums, it found its way across the desk of none other than Cloudflare CEO Matthew Prince, who declared the release was “Google’s DeepSeek.”

“So much more room to optimize AI inference for speed, memory usage, power consumption, and multitenant utilization,” he said in a post on X. “Lots of teams at Cloudflare focused on these areas.”

A bold claim for sure. And things got bolder still as while TurboQuant was becoming ever more communicable, memory makers like Micron and Sandisk saw their stock price dips, leading to Futurum analyst Shay Boloor proclaiming: “The market is reading it as a potential headwind for memory names because long-context AI inference may now need far less memory per workload.”

Such speculation may have some truth. But it’s important to note that as the TurboQuant talk was ongoing, Sandisk announced plans to invest $1 billion in Taiwanese dynamic random access memory (DRAM) manufacturer Nanya Technology, which drew ire from some corners over fears of capital misallocation. Micron, meanwhile, had a storied history of being an unpredictably volatile stock – though in the weeks since TurboQuant, it turned a complete circle as its market cap surpassed the $1 trillion mark for the first time amid unprecedented demand for its chips.

Regardless of its status as a short-lived stock slayer or not, TurboQuant had brought algorithmic compression into the minds of the compute-focused masses.

Mathmagicians or bubble bursters: What’s the reality?

In the minds of the semiconductor analysts, however, TurboQuant represented just another market hyperbole.

Jim Handy, founder and general director at Objective Analysis, said that as the paper wormed its way around the web, investors called him, desperate to determine if their investments would be impacted. He took a measured approach in responding to investor concerns, reminding them of the same sort of panic seen more than a year ago with DeepSeek.

“These are just things the hyperscalers do; they put together algorithms to find ways to better use resources, but it doesn’t change the spending patterns of hyperscalers,” Handy told SDxCentral. “What the investors were worried about was that the investments, the capital spending, would drop as a result of TurboQuant. And that didn't happen.”

But why didn’t that happen? Handy explained it had everything to do with tokens: the economic measurements for AI infrastructure. Given hyperscalers measure their buildout efforts in token terms – specifically, how much potential they have to process.

If anything, TurboQuant and its ability to crush AI model weights through impressive KV cache quantization while maintaining their core makeup would encourage these hyperscalers to spend more on tokens.

“These brilliant people are putting together algorithms; they're trying to reduce memory usage and reduce processor usage. And what ends up happening is that if everybody says we're going to be spending $10 billion this quarter on capital expenditures for our data centers, then they say, "Oh, good, our $10 billion will go farther, but we're still going to be spending $10 billion.”

Like a volcano erupting or an adverse weather effect, TurboQuant ended up being just another little blip, at least to those investing in the AI space, with Micron and Sandisk ending up victims of nervous stock traders, rather than substantial, lasting changes to underlying market fundamentals.

But model makers are always finding new ways to reduce weights. The bigger equals better adage from the early days of AI is far behind us, with developers instead wanting smarter reasoning capabilities and modular specialization. Take the rise in popularity of open-source distillations of popular frontier models, where popular models are essentially imitated for specific use cases or applications, without needing the massive compute of the original model, like GPT-4o mini.

Developers will always find a way. Handy referenced BitNet, a Microsoft line of AI models with weights of 1.58 bits. For reference, Nvidia hardware is used to process four-bit, eight-bit, and 16-bit workloads.

Not only could that drastically reduce the amount of computing power GPUs would need in order to run such small models – which leaves developers to run even more. But it also unlocks the ability to run inference on the CPU, which, for more inherently complex, agentic-style workloads, is essential.

“Algorithm developers compete to see who can squeeze down [weights] the most, but that doesn't mean that in a production environment,” Handy outlined. “In a production environment, things like that are still viewed as being experimental. And they'll roll out, but they'll roll out gradually. And when that happens, you’ll see a gradual change, but I still don't expect to see a huge difference.”

What it also doesn’t do is eliminate the need for the GPU, as the analyst explained that companies are buying GPUs to help with their memory needs amid the ongoing shortage.

“It sounds like they’re pulling the GPU off the board to use the high-bandwidth memory (HBM), but they’re not doing that. What they’re doing is taking an idle GPU, let’s say 50% of the time, and during that idle time, using it,” Handy said. “GPUs are so wretchedly expensive that anybody will do anything to try to get more performance out of the same amount of memory that's in them.”

What Google's paper didn't mention

Shortly after TurboQuant’s overnight success, Jianyang Gao, a postdoctoral researcher at ETH Zurich and first author of RaBitQ, a rival vector quantization method published at SIGMOD in 2024, accused the TurboQuant team of misrepresenting his prior work.

RaBitQ and TurboQuant share a core technique: both apply a random rotation (or Johnson-Lindenstrauss transform) to vectors before quantizing them. But in a scathing blog post, Gao argued this is the central design idea the two methods have in common, and that TurboQuant's authors knew it.

His evidence was that in January 2025, months before TurboQuant appeared on arXiv, one of its authors had reached out to Gao's team for help debugging a Python implementation translated from RaBitQ's own code. Despite that, the TurboQuant paper described RaBitQ only as "grid-based" quantization, without mentioning the rotation step, and according to Gao, even after a reviewer flagged the similarity and asked for a fuller comparison, the final version moved its (still incomplete) description of RaBitQ into the appendix rather than expanding it.

Gao also disputes TurboQuant's characterization of RaBitQ's theoretical guarantees as “suboptimal,” pointing to a proof in the extended RaBitQ paper showing it achieves an asymptotically optimal error bound. And he raises questions about the benchmark comparison itself: TurboQuant was evaluated on an Nvidia A100 GPU, while RaBitQ was tested using a translated Python version on a single CPU core with multithreading switched off – conditions Gao says don't reflect RaBitQ's publicly available, multithreaded implementation.

According to Gao, his team raised all three issues with TurboQuant's authors by email before the ICLR submission. One author reportedly agreed to address two of the three points but declined to revisit the question of methodological overlap, and only after the conference had concluded. Gao's team has since posted a public comment on OpenReview and filed a complaint with ICLR's ethics chairs.

What might appear on the surface to be a matter of academic etiquette, but the separate technical note posted to arXiv in late April, which re-ran the comparison under matched hardware and software conditions, found that TurboQuant actually underperformed RaBitQ in most of the tested settings for inner-product estimation, nearest-neighbor search, and KV cache quantization.

So, is compression all you need?

Maybe, but not necessarily this compression, and not necessarily on the terms it was sold to the public. The TurboQuant saga has, in miniature, almost every ingredient of an AI hype cycle: a dramatic headline result; a viral endorsement from a high-profile CEO; a same-day market reaction worth billions; and then, once the noise died down, a far messier reality involving disputed benchmarks, contested attribution, and a research community quietly arguing about whose idea this actually was.

Handy's framing still holds up: hyperscalers aren't going to spend less simply because an algorithm got a bit more efficient; they're going to do even more with the very same budget. The extra drama from the RaBitQ dispute only adds to the irony here.

In sum, the paper that briefly knocked billions off memory stocks and lauded as Google's answer to DeepSeek, is now itself the subject of the kind of scrutiny it was supposed to deliver to the rest of the industry.

If there's a lesson to be learned from TurboQuant beyond memory and tokens, it might be that this industry is moving so fast that the gap between a paper going viral and a paper being right can be measured in weeks. And by the time anyone’s scratched under the surface, stock prices and investor cheerleaders have already moved on.

This article first appeared in the SDxCentral Memory & Storage Supplement.

To read for free, simply register.