Modular founders Tim Davis (L) and Chris Lattner (R)

AI infrastructure startup Modular secured $250 million in Series C funding, nearly tripling its valuation to $1.6 billion as it builds what it calls “AI's unified compute layer.”

The Palo Alto, California-based company's latest round was led by Thomas Tull's U.S. Innovative Technology (USIT) fund, with DFJ Growth joining as a new investor. Existing backers, including GV (Google Ventures), General Catalyst, and Greylock Ventures, also participated.

The funding brings Modular's total capital raised to $380 million across three rounds since its 2022 founding by Chris Lattner, the creator of Apple's Swift programming language and former Tesla Autopilot head.

Modular is tackling what it describes as massive underutilization in AI compute infrastructure, fragmented by proprietary software stacks from different hardware vendors. It has developed what amounts to a hypervisor for AI workloads, allowing them to run seamlessly across diverse silicon from Nvidia and AMD, as well as emerging chip architectures.

“Strategic AI implementation is the most important competitive factor in today's economy," Tull noted as part of the latest funding round. "Modular is foundational in this new era of diverse AI infrastructure, providing a unified AI compute layer that maximizes efficiency, resilience, and cost reduction.”

Since launching its platform in 2023, Modular claims it has been downloaded tens of thousands of times every month, powering trillions of tokens daily.

The startup touts its unified stack as a way to deliver up to 70% latency reduction and 80% cost savings compared to vendor-specific runtimes like CUDA and ROCm. In its latest 25.6 release, Modular reports up to 50% performance gains over popular frameworks like vLLM and SGLang on hardware like Nvidia’s B200 and AMD's MI355.

Modular’s platform is made up of three core components:

• Mammoth: A Kubernetes-native control plane for large-scale distributed AI serving
• Max: A high-performance generative AI inference framework with OpenAI-compatible endpoints
• Mojo: A systems programming language blending Python syntax with C/C++ performance

All three come together to maximize compute utilization – a growing digital infrastructure bottleneck that engineers are hoping to overcome to get the best out of hardware handling intense AI workloads.

Modular's approach has earned it an array of partners, including enterprises like Inworld and SF Compute, research teams like Jane Street, and cloud giants including Oracle, Amazon Web Services (AWS), Lambda Labs, and TensorWave.

The new capital will fund scaling of its platform for cloud-native environments while also extending support across edge hardware platforms. The company also plans to expand beyond traditional data center GPUs, with a view to supporting Apple GPUs, consumer-grade hardware, and upcoming ASICs.

“When we founded Modular, we believed that the world needed a unified platform for AI, and today, that vision is more important than ever,” Lattner explained. “This funding will enable us to realize that vision for developers, enterprises, and hardware companies around the world.”