A startup specializing in AI inference pulled in $20 million in funding from investors including AMD, Nvidia, and neocloud CoreWeave.

San Francisco-based Tensormesh has now collected investments totaling $24.5 million. Founded last year by faculty and researchers from the University of Chicago, University of California-Berkeley, and Carnegie Mellon, the firm specializes in caching-accelerated technology for inference with its flagship Tensormesh Inference platform.

The platform promises a 10-times reduction in latency and graphics processing unit (GPU) spend, with zero-cost cached input tokens as a permanent pricing model. These bold claims come from how the platform stores and reuses computed results through key-value (KV) caching. With the rise of AI agents, the repeated token reuse and intensive agentic workloads inherent with AI inference are rapidly increasing demand for prompt, always-on storage, with the industry increasingly employing KV caching in AI-related workflows to eliminate the bottleneck of redundant compute.

Under the Tensormesh hood

Tensormesh Inference touts prompt API access from various frontier models without the need for infrastructure to provision or manage, as well as reserved deployments for those enterprises running AI at scale and which are in need of dedicated capacity, predictable performance, and custom service level agreements (SLAs).

Open-source KV caching project LMCache underpins the operation, which speeds up large language model (LLM) inference by caching and reusing KV computations. Tensormesh added the platform allows for direct control over how much cache backend storage is allocated to deployments and can surface metrics to gauge exactly how the storage is performing.

Tensormesh CEO Junchen Jiang also created LMCache and noted in a statement on the funding round that Tensormesh offers “a new vision on the significance of the intermediate data that LLM generates when processing the prompt.”

“Behind the term KV cache is a whole concept of AI interpretation of the question it is asked," Jiang said. "This makes it a whole new class of data and a category Tensormesh is uniquely positioned to define."

Tensormesh claimed AMD, CoreWeave, and Nvidia hold “a shared conviction that KV caching is a foundational layer of the AI infrastructure stack.” Indeed, Nvidia this year debuted its approach to KV caching in the form of Inference Context Memory Storage (ICMS) platform via the chip giant’s BlueField-4 DPU. Specifically, Nvidia’s stack sees BlueField data processing units (DPUs) as the storage engine handling context movements close to the GPU, while Spectrum-X Ethernet scale-across networking keeps access predictable under load. Nvidia's Dynamo software orchestrator coordinates inference behavior and ICMS defines context as system memory.

The implications of this saw Nvidia partner and storage vendor Vast Data claim the market needs an additional capacity of 100 exabytes, which “doesn't necessarily exist” according to Vast cofounder Jeff Denworth due to ongoing memory supply constraints.

Tensormesh’s funding reflects a growing clamor for solutions to such bottlenecks as well as AI inference tools in the market. The South Korea-founded FriendliAI, for example, saw funding late last year of $20 million for its own inference solution, while chip designer Cerebras Systems this month bagged the largest IPO for a U.S. tech company since 2019 off the back of its wafer scale engine (WSE)-3 semiconductors and AI inference offering.

Article updated with further information from Tensormesh that was unavailable prior to publication.